November 14, 2026 · The CompletionKit team

How to test AI prompts before deploying them to production

To test an AI prompt before it reaches production, run it on a fixed set of real inputs, score each output against criteria you wrote down in advance, and compare those scores row by row with the version you have live. If the candidate clears the bar you set and breaks nothing that used to work, it ships. If not, you read the rows that failed, change one thing, and run it again. AI prompt testing replaces "this reads better" with a score per row that you can check against the version you already trust.

Most prompts ship on far less. You try it on five examples while you write it, tweak a sentence, try the same five again, and they look better, so out it goes. Then a customer asks something you didn't think of, the model drops a formatting rule, or a support reply promises a refund your policy doesn't allow. The prompt passed your tests. Your tests were five examples you made up while you already knew what the prompt was meant to do.

Software people learned this a long time ago. At a NATO software engineering conference in Rome in October 1969, Edsger Dijkstra put it in one line: testing shows the presence, not the absence of bugs. A test suite cannot prove a prompt is right. What it can do is find the failures you can find before a customer does, and keep finding them on every later change. That is the whole job.

What testing an AI prompt means

A prompt test has three parts that stay put and one that moves. The dataset of inputs stays put. The metrics that define a good output stay put. The baseline, the scores from the version you trust, stays put. The prompt is the part that moves, and the test tells you whether the move helped.

It differs from a unit test in one way that matters. A unit test can assert that a function returns 42. A support reply can come out ten different ways and all ten can be fine, so an exact match is the wrong test for most of what a prompt produces. You need two kinds of scoring side by side. A deterministic check handles anything a rule can decide (valid JSON, a required field, a label from a fixed list, a phrase that must never appear) and gives the same pass or fail every time. An LLM judge handles the qualities that need reading (did it answer the question, did it stay inside policy, would a customer know what to do next) and scores each one against a written rubric on a 1 to 5 scale, with 5 the top mark. Neither replaces the other. For the longer definition, see what prompt evaluation is.

How to test AI prompts in seven steps

The loop is the same for one support prompt or forty prompts spread across an app. None of it needs special infrastructure to start; it needs a dataset, a definition of good, and the discipline to compare like with like.

1. Write down what the prompt is for. Before you look at a single output, list what a good one does. For a support assistant that might be accuracy (no invented order details), clarity (the customer knows the next step), tone, and policy (no promises the policy doesn't make). For an invoice extraction prompt it would be a correct total, the right invoice number and valid JSON. A generic "quality" score can't tell a friendly answer from a correct one, so pick metrics that match the prompt's actual job.
2. Build a dataset from real inputs. Pull them from production logs, support tickets or whatever the prompt already receives, because inputs invented by the person who wrote the prompt are the ones it already handles. Thirty to fifty well-chosen rows is a respectable start. Weight it toward the awkward cases, and keep a stable version of it, since a score change between two datasets tells you nothing about the prompt. The guide to building a prompt eval dataset covers sourcing and coverage.
3. Run the current prompt over every row. Each row fills the template's variables, the model generates an output, and you keep all of them. Record the model and its settings alongside, because if you change the prompt, the model and the temperature at once you will never know which one moved the score. This first run is the baseline every later version has to beat.
4. Score every output. Checks go first, since they are cheap and certain. The judge scores the rest, one metric at a time, and returns a rationale for each row along with the number. The rationale matters more than the average, because it tells you why a row failed.
5. Change one thing and run it again. Same dataset, same metrics, same judge. Then compare the two runs row by row, not just their averages, and read every row whose score went down.
6. Make the fix targeted. If the new version is friendlier but now promises refunds, the answer is not to make it less friendly. Add the constraint that is missing ("never promise a refund, replacement or account credit unless the supplied policy allows it") and run the comparison again.
7. Ship against a bar you set in advance. Decide before the candidate runs what it has to clear: an average on each metric, no failures on the checks that guard hard rules, and no row worse than the live version by more than a set margin.

A real test run, and what it didn't tell us

Here is one pass of that loop from our own workspace. The prompt was the generic first draft most people write: a support agent for an online store, asked for a short, friendly reply. The dataset was eight customer messages, each carrying an order number. One metric, acknowledges-order-number, scored whether the reply referred to the customer's specific order. The first run averaged 3.88. On the lowest-scoring row, the judge explained that the reply never mentioned the order at all. Adding one sentence, "Acknowledge their specific order.", lifted the second run to a perfect 5.0, with the dataset and judge unchanged. The full rubric and the row-level rationale are in the LLM-as-a-judge guide.

(Real runs from June 12, 2026: gpt-4.1-mini wrote the replies and claude-sonnet-4-6 scored them.)

That is a clean result and an incomplete test. A single metric that climbs to 5.0 tells you the reply now mentions the order. It says nothing about whether the reply got longer, drifted in tone, or started promising things, because nothing was measuring those. One metric is enough to show the mechanics. A prompt you are about to ship needs a metric for each thing you listed in step one, so that the edit that fixes one row has nowhere to hide the damage it does to another.

Why the average hides the regression

The failure this whole process exists to catch looks like a success from a distance. Take a support prompt that is accurate but stiff. You add a line asking it to sound warmer. The replies get friendlier and clearer, and the overall average goes up. Somewhere in the dataset, three rows that used to say "I'll pass this to our billing team" now say "I've gone ahead and refunded you". The policy score on those rows fell through the floor, but the gains everywhere else pulled the average up past the old version.

An average treats a lost policy row and a slightly stiffer greeting as the same kind of loss. They are not. That is why you read the per-row comparison, and why a hard rule belongs in a check or a metric of its own instead of being folded into a general quality score. A small drop on an odd phrasing might be a fair trade. A new refund promise is not, whatever the average says. Catching that kind of drop on every change is prompt regression testing, and it is the half of prompt testing that keeps paying after launch.

Which test fits which output

Most prompts need more than one kind of test. Match the method to whether the answer can be decided by a rule.

You don't need a language model to decide whether $125.00 equals $152.00, so don't use one. You also shouldn't fail a support reply for using different words from a reference answer. For mixed outputs, use both: check the fields with definite answers directly and let the judge score the rest. A judge-only run is the quickest way to start if the prompt is already live, because the outputs are sitting in your logs.

The judge is a measuring instrument, and it can be wrong. It can favour longer answers, misread a vague rubric, or score two near-identical replies differently. Before you let a judge gate a release, have someone mark a sample of its scores as right or wrong, tighten the rubric where they disagree, and only then trust the number.

Set the bar before you run the candidate

The release decision should be made before you see the candidate's scores. If you set the threshold afterwards, or nudge it down each time a version narrowly misses, you don't have a gate. You have a way of explaining why this one is fine to ship.

There is no universal pass mark. A prompt that suggests blog titles and a prompt that answers billing questions shouldn't share one. A workable pattern is two tiers. Hard rules (policy, schema, banned phrases) are checks that must pass on every row, and any failure blocks the release. Softer qualities (tone, clarity) are judge metrics with an average to clear and a per-row tolerance against the live version, wide enough to absorb a judge's normal variance between runs. The CI/CD guide covers how to pick those tolerances so a good change doesn't fail on noise.

Automate AI prompt testing before deployment

Running the loop by hand works with one prompt and one person. It stops working when several people edit prompts, models get upgraded, and changes ship every week. At that point testing belongs in the pipeline, not in a spreadsheet someone remembers to update.

Gate prompt changes in CI. The same pipeline that stops untested code from merging can stop an untested prompt. A change to the prompt, the model or the sampling settings starts a run on the candidate, the job compares its scores with the baseline, and the build fails on a drop you didn't sign off. The CI/CD for LLM applications post walks through a working GitHub Actions job.

Ship the version you tested. A common gap is that the prompt you tested is not quite the one you deployed: someone makes one more small edit after the run, or two people edit from different starting points. Prompt versioning gives each revision a fixed identity, so every score points at the exact text that earned it, and rolling back means picking an earlier tested version rather than rebuilding it from memory.

Turn every production failure into a row. No dataset covers everything, and real users will find the gaps. When they do, fix the prompt and also add the input that broke it to the dataset, so every later version has to pass that case before it ships. Over a year the dataset becomes a record of everything the prompt has to keep doing.

Testing AI prompts in CompletionKit

The seven steps above map onto CompletionKit directly. You upload a CSV dataset whose columns match the variables in your prompt template, define your metrics (LLM judge metrics with a rubric scored 1 to 5, and deterministic checks such as valid JSON, contains, regex and numeric bounds), and run the prompt over every row. Each output comes back with its scores and the judge's rationale. Two runs compare row by row, so you can go straight to the rows that got worse. Prompts are versioned, so once a prompt has runs, editing it creates a draft that stays out of production until you publish it. Your app then pulls the live version from the API. When you want the judge checked, you mark its scores as agree or disagree and correct the ones it got wrong, which shows how far the judge tracks your team's verdicts.

Everything above also works through the REST API for CI, and through the MCP server for a coding agent, which can then run the improvement loop itself. Runs can generate with OpenAI, Anthropic, Azure AI Foundry, OpenRouter or a local Ollama model, using your own provider keys. CompletionKit is independent of every model lab it scores, the engine is source-available and self-hostable, and pricing is a flat price for the team, not a fee per seat, with a free tier to start on. The docs walk through a first dataset, run and metric.

FAQ

Test your prompts free   Prompt regression testing

Built by Homemade Software. Disagree with any of this? support@completionkit.com or r/completionkit.

← All posts