June 7, 2026 · The CompletionKit team

Best prompt evaluation tools in 2026

The credible prompt evaluation tools in 2026 are Anthropic Workbench, Braintrust, CompletionKit, DeepEval, Galileo, Langfuse, OpenAI Evals, and Promptfoo. The list below is alphabetical, not ranked. Each one is built around the same loop (a dataset, a prompt, a score, a comparison to last week) and each one makes different tradeoffs about where it lives, how it grades, and what it leaves to you.

A roundup is only useful if it is honest about who each tool is for, so that is what every entry tries to do: one paragraph, the standout strength, the main tradeoff, the kind of team it fits. We make one of these tools, and we have tried to write its paragraph the same way we wrote the others. If you want the concept rather than the catalogue, the what is prompt evaluation piece is the pillar, and our head-to-head comparison of these tools puts them on one feature table; this list is the directory.

How the category formed

Prompt evaluation as a named, packaged thing barely exists before 2023. People scored model outputs against held-out test sets for years (the move is borrowed straight from classical machine learning), but the activity ran in ad-hoc Python scripts and the occasional MLflow project. The category got its short name, its first reference framework, and its first big public push on March 14, 2023, when OpenAI shipped GPT-4 and open-sourced a framework called Evals alongside it. They even paid for high-quality eval contributions with GPT-4 access. The signal was loud: the next bottleneck was not the model, it was a trustworthy way to measure it.

Three months later, in June 2023, the Berkeley group behind Chatbot Arena published "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" and gave the field its other foundational move: use one model to grade another, calibrated against human-to-human agreement rather than absolute correctness. Most of the platforms below score open-ended outputs by following that recipe in some form. We unpack it further in our LLM-as-a-judge guide.

The two years after that were the build-out. Galileo (founded 2021) had already framed the problem for enterprises. Braintrust, Langfuse, and several others arrived in 2023; DeepEval and Promptfoo in 2024; CompletionKit later. Then came consolidation. In the first four months of 2026, three of the tools on this list changed hands: ClickHouse acquired Langfuse on January 16; OpenAI announced its acquisition of Promptfoo on March 9 (the project stays open source); Cisco announced its intent to acquire Galileo on April 9, folding it into Splunk Observability Cloud. The 2026 list still has eight credible tools, but the ownership map looks very different than it did a year ago.

The list

1. Anthropic Workbench. The prompt sandbox inside the Anthropic Console. You draft a Claude prompt, send it, inspect the response, tweak settings, version it, and compare versions side by side. There is a built-in evaluation surface with a 5-point grading scale for subject-matter experts. It is the fastest tactile feedback loop for building a Claude prompt and the easiest entry point on this list, especially for a team that already lives in the Console. The tradeoff is that it is Claude-only, hosted-only, and aimed at the build phase rather than the year of maintenance after it. There is no production serving layer for your app to fetch from, and the loop ends at the Console boundary.

2. Braintrust. A hosted platform that does eval, observability, prompt management, and serving as one product. Founded in 2023 by Ankur Goyal, who spent years at MemSQL (eventually VP of Engineering) and founded Impira (acquired by Figma) before this. The standout strength is completeness: most of what a serious LLM team wants on day 200 is in one place, with a polished UI, real data tooling, and a thoughtful experiment surface. Closest direct peer to CompletionKit in scope. The tradeoff is that it is hosted-first (self-hosting is reserved for an enterprise tier), it is the heaviest commitment on the list, and the pricing reflects that. Strong pick if you want the whole platform under one roof and SaaS is the right shape for you.

3. CompletionKit. The tool we build. A Rails engine you can mount in your own app or run standalone, plus a hosted version (CompletionKit Cloud) for teams that would rather not. Built around the idea that the prompt your app fetches in production should be the same versioned artifact you tested against, with an MCP server so a coding agent can drive the whole loop. Scoring is LLM-judge-first on a 1–5 rubric you write per metric, with a calibration surface for human labels. The engine is source-available under BSL 1.1, free for any use including production. Tradeoffs to be honest about: the scoring model assumes you are happy grading open-ended outputs with an LLM judge (it can do exact-match too, but the judge is the spine), and we are younger and smaller than the platforms above. Why we built it covers the reasoning in full.

4. DeepEval. The open-source Python testing framework from Confident AI, founded in 2024 by Jeffrey Ip and Kritin Vongthongsri (YC W25). The standout strength is ergonomics for engineers who already live in pytest: assert_test(case, [Metric()]) reads like a normal unit test, and the metric library is broad (G-Eval, hallucination, summarization, faithfulness, plus your own). Apache 2.0 licensed, runs anywhere. Confident AI is the paid platform layered on top, for tracing and team-level reporting. The tradeoff is that DeepEval alone is a framework, not a system of record: there is no UI for non-engineers, no prompt-versioning store, no serving layer; bring those yourself or pick up the paid product.

5. Galileo. Founded in 2021 by Vikram Chatterji, Atindriyo Sanyal, and Yash Sheth, with backgrounds spanning Google AI, Apple Siri, and Uber AI. The standout strength is scope: full agent observability and evaluation over multi-step agent traces, with guardrails for production. Built for enterprises that need real procurement-grade governance around an LLM rollout. Cisco announced its intent to acquire Galileo on April 9, 2026, with the team folding into Splunk Observability Cloud. The tradeoff is what comes with that posture: it is closed-source SaaS (with VPC and on-prem for enterprise), pricing is not on the website, and the buying motion is enterprise, not a swipe-and-go signup.

6. Langfuse. Founded in 2023, acquired by ClickHouse on January 16, 2026. The standout strength is open-source LLM observability done well: tracing, costs, latency, prompt management, and a real evaluation suite that scores how often the LLM judge agrees with your own labels. Self-hostable, Docker-first, with a huge community. The tradeoff is that the center of gravity is observability; evaluation is a feature, not the spine. If your dominant question is "what happened in production yesterday" it is one of the best picks on this list. If your dominant question is "is this prompt change safe to ship," the eval-first tools take the loop further.

7. OpenAI Evals. The original. Open-sourced by OpenAI on March 14, 2023, alongside the GPT-4 launch; MIT licensed. The standout strength is that it is dependable, well-understood, and where many people first learned the vocabulary of datasets, runs, and graders. Custom evals are straightforward Python; the registry has a lot of public examples to copy from. The tradeoff is that it is a dev-time harness rather than a system of record: it has no UI, no managed dataset versioning, no serving layer, and its defaults assume the OpenAI ecosystem. Strong pick when you want code-first evaluation for OpenAI models and don't need a platform around it.

8. Promptfoo. A beloved open-source CLI for prompt evaluation in CI, founded in 2024 by Ian Webster and Michael D'Angelo. Acquired by OpenAI on March 9, 2026; the project remains open source under its current license. The standout strength is the CI fit: a YAML config, a row of providers, deterministic and LLM-based graders, and a clean diff between runs. Fast, scriptable, easy to drop into an existing pipeline, especially strong for red-teaming and adversarial test suites. The tradeoff is that it is a test runner, not a system of record: no prompt-of-record, no serving layer, no UI for non-engineers; the lifecycle ends at "the suite passed." Pairs well with anything from this list that does the serving and UI side.

How to pick

Start from how your team actually works, not from the feature matrix.

1. Need a CLI in CI. Promptfoo, then DeepEval. Both drop into a pipeline cleanly; Promptfoo is closer to a turnkey runner, DeepEval is closer to pytest.
2. Need non-engineers in the loop. Braintrust, CompletionKit, Galileo, or Langfuse. CLIs and frameworks leave PMs and content folks out; these four have a UI built for them.
3. Need the prompt in your app to be the artifact you tested. Braintrust or CompletionKit. Both have a serving layer; most of the others don't.
4. Self-hosting matters. CompletionKit (source-available, free for any use), DeepEval (Apache 2.0), Langfuse (MIT), OpenAI Evals (MIT), or Promptfoo (open source under its current license).
5. Already on a single provider for a sandbox. Anthropic Workbench (Claude) or OpenAI Evals (GPT). Lowest setup cost when you are deep in one ecosystem.
6. Enterprise procurement, multi-step agents, governance. Galileo, headed into the Splunk Observability portfolio. Braintrust is also in that conversation; the buying motions look different.
7. The scoring model matters most. If outputs are open-ended, an LLM-judge-first tool with a calibration surface (CompletionKit, Braintrust, Langfuse, DeepEval) saves the most pain. If outputs are exact-match (JSON keys, tool calls), the deterministic graders in any of these will do.

One last note. The thing that determines whether a prompt evaluation tool earns its keep is not its launch-day demo; it is whether the loop still runs on day 200, after a model swap, a customer-driven edge case, and three rounds of tone tweaks. Regression testing is the muscle that turns the loop into a habit, and any of the tools above can host it. The differences are in which steps they automate, where the artifact lives, and how much of the work stays with you.

FAQ

Try CompletionKit free   Why we built CompletionKit

Built by Homemade Software. Got a tool right that we got wrong? support@completionkit.com or r/completionkit.

← All posts