# CompletionKit > CompletionKit is a prompt evaluation and improvement loop for AI apps. Define what good means with a 1-5 rubric, run prompts against a fixed dataset, score every output with an LLM judge, and see per-row scores plus the judge's rationale before you ship. CompletionKit Cloud is the hosted multi-tenant version. The same product also ships as a Rails engine (`completion-kit` gem) you can mount in an existing Rails app, and as a bundled standalone Rails app for self-hosters. It is source-available (BSL 1.1), not open source. ## Start - [Sign up](https://completionkit.com/registration/new): create an account and run your first evaluation. Self-serve, no sales call. - [Pricing](https://completionkit.com/pricing): the plan tiers and what each unlocks. - [About](https://completionkit.com/about): who makes CompletionKit and why it exists. ## Docs - [Overview and concepts](https://completionkit.com/docs): the four moves of the loop, the glossary, and where to start. - [REST and MCP reference](https://completionkit.com/docs/api): every API endpoint, every MCP tool, with copy-paste examples. Point a coding agent (Claude Code, Cursor) at the MCP server and it can drive the whole evaluate-and-improve loop. - [Glossary](https://completionkit.com/glossary): one-page definitions for prompt evaluation, LLM judge, prompt versioning, regression testing, eval dataset, metric, judge calibration, and the automated prompt optimization loop. - [Changelog](https://completionkit.com/changelog): what shipped, dated. ## Blog (the Prompt Loop campaign) - [LLM application requirements for production-ready AI apps](https://completionkit.com/blog/llm-application-requirements): Most LLM application requirements are about outputs. Write each one so it can fail, then score it with a deterministic check or a calibrated LLM judge. - [AI quality assurance: best practices for testing prompts and outputs](https://completionkit.com/blog/ai-quality-assurance): AI quality assurance best practices for prompts and outputs: a fixed dataset, checks plus an LLM judge, a baseline, CI gates, and a judge you can trust. - [How to test AI prompts before deploying them to production](https://completionkit.com/blog/how-to-test-ai-prompts): How to test AI prompts before they ship: run them on a fixed dataset of real inputs, score each output with checks and a 1-5 judge, and gate on regressions. - [How to reduce AI hallucinations through better prompt testing](https://completionkit.com/blog/how-to-prevent-ai-hallucinations): You can prevent AI hallucinations from shipping by testing the prompt on hard cases, scoring groundedness with a judge and checks, and comparing every change. - [Agent prompt testing: how to test agent prompts and outputs](https://completionkit.com/blog/agent-prompt-testing): Agent prompt testing catches failures before they ship. Here is how to test agent prompts and evaluate agent outputs on real inputs with checks and a judge. - [CI/CD for LLM applications: testing and deploying prompt changes](https://completionkit.com/blog/ci-cd-for-llm-applications): How CI/CD for LLM applications gates prompt and model changes on a scored eval, catches per-row regressions, and publishes only the prompt revision that passed. - [Let your coding agent improve the judge itself](https://completionkit.com/blog/improve-the-judge-with-mcp): How to let a coding agent improve the LLM judge itself over MCP: read disagreements, draft rubric rewrites, compare against human verdicts, publish only on a recommend verdict. - [Your eval tool shouldn't work for the model it's grading](https://completionkit.com/blog/eval-tool-independence): Why evaluation infrastructure should be independent of the model labs it scores, and what to ask any eval vendor about ownership, defaults, and data. - [The 8 best prompt evaluation tools for testing LLM prompts in 2026](https://completionkit.com/blog/best-prompt-evaluation-tools-2026): Compare the 8 best prompt evaluation tools for testing LLM prompts in 2026: who each one fits, its main tradeoff, and what three acquisitions changed. - [CompletionKit vs Braintrust, Langfuse, Promptfoo, Galileo, DeepEval: prompt evaluation tools compared](https://completionkit.com/blog/completionkit-vs-galileo-vs-braintrust-vs-promptfoo): Honest comparison of the eight prompt evaluation tools that matter in 2026 (CompletionKit, Galileo, Braintrust, Promptfoo, Langfuse, OpenAI Evals, Anthropic Workbench, DeepEval) on seven axes. - [What is a judge-only run?](https://completionkit.com/blog/what-is-a-judge-only-run): A judge-only run scores outputs you already have, with no generation step. Use it on production logs, gold sets, or another system's results. - [Claude Code and Cursor setup: an MCP server for prompt evals](https://completionkit.com/blog/mcp-server-for-prompt-evals): MCP server setup for prompt evals in Claude Code and Cursor: connect your coding agent to CompletionKit with one command or one mcp.json entry. - [How to build an evaluation loop for automated prompt optimization](https://completionkit.com/blog/automated-prompt-optimization-loop): How to build an evaluation loop that lets Claude Code or Cursor optimize your prompts: revise, run, score, read the misses, stop when it clears your bar. - [Prompt versioning: how to track, test, and improve LLM prompts](https://completionkit.com/blog/prompt-versioning): Prompt versioning gives each prompt change a fixed id, a readable diff and a rollback with no deploy. See how AI prompt versioning fits prompt testing. - [What is prompt regression testing, and how does it work?](https://completionkit.com/blog/prompt-regression-testing): Prompt regression testing, or LLM regression testing, re-runs a fixed dataset on every prompt change and diffs each row's score so working cases stay working. - [How to build a prompt evaluation dataset](https://completionkit.com/blog/how-to-build-a-prompt-eval-dataset): An evaluation dataset is the fixed set of inputs every prompt version is scored on. How to build one from real traffic, size it, and keep it fixed. - [What is prompt evaluation? How to test and improve LLM prompts](https://completionkit.com/blog/what-is-prompt-evaluation): Prompt evaluation scores a prompt against a fixed dataset before you ship it. Here is how it works and how to use it to test and improve LLM prompts. - [LLM-as-a-judge: a practical guide to scoring prompt outputs](https://completionkit.com/blog/llm-as-a-judge): LLM-as-a-judge scoring uses one model to grade another's output against a rubric. Here is how to set it up and calibrate it so you can trust the score. - [Why we built CompletionKit](https://completionkit.com/blog/why-we-built-completionkit): Shipping a prompt is the easy part. The hard part is the year afterward without quietly breaking what already worked. Why we built CompletionKit for that. - [CompletionKit Cloud is live](https://completionkit.com/blog/completionkit-cloud-is-live): CompletionKit Cloud is open for signups: hosted prompt evaluation with a free tier, no install, and a real discount for early adopters. ## Compare - [CompletionKit vs Promptfoo](https://completionkit.com/compare/promptfoo): how the integrated evaluate-and-improve loop differs, and where each fits. - [CompletionKit vs Braintrust](https://completionkit.com/compare/braintrust): feature, focus, and pricing comparison. - [CompletionKit vs Langfuse](https://completionkit.com/compare/langfuse): feature, focus, and pricing comparison. - [CompletionKit vs DeepEval](https://completionkit.com/compare/deepeval): feature, focus, and pricing comparison. ## Code - [Engine on GitHub](https://github.com/homemade-software-inc/completion-kit): the source-available Rails engine. - [completion-kit on RubyGems](https://rubygems.org/gems/completion-kit): the gem releases. ## Legal - [Privacy](https://completionkit.com/legal/privacy), [Terms](https://completionkit.com/legal/terms), [DPA](https://completionkit.com/legal/dpa), [Security](https://completionkit.com/legal/security), [Accessibility](https://completionkit.com/legal/accessibility).