May 11, 2026
CompletionKit Cloud is live
CompletionKit Cloud is open. The hosted version of the prompt-evals engine we've been building — no install, no Rails app of your own to maintain, a free tier you can start on today, and your prompts versioned behind stable URLs your app calls directly.
If you've been treating prompts like configuration — change the string, ship, hope — this is the thing that turns "hope" into a number. Pick real inputs from production, run your prompt against them, score the outputs on the criteria you actually care about (tone, accuracy, format compliance, whatever matters), and gate every future change on "doesn't regress the baseline." That's the whole loop. Cloud just makes it fast enough to run on every edit.
What's in the box
- Prompts as endpoints. Every published version has a URL. Your app fetches the template + model and runs the LLM the way it already does. No SDK.
- LLM-as-judge scoring. Write metrics with 1–5 rubrics; a judge model applies them to every response in a run.
- Multi-model. OpenAI, Anthropic, Ollama or any OpenAI-compatible local endpoint, and 100+ models via OpenRouter.
- Runs, datasets, versioned history. See exactly what got better and what broke, version over version.
- An MCP server per organization so you can drive prompts, runs, and datasets from an LLM agent.
The engine stays source-available and free to self-host — on GitHub, on RubyGems. Cloud is for the people who'd rather not run it themselves.
Early adopters get a real discount
We're launching, which means we want the first wave of teams who'll push the product and tell us where it bends. If that's you, there's a meaningful discount on the table — well beyond the published pricing.
Two ways to get the details:
- Email support@completionkit.com and tell us a bit about what you're building.
- Or hit the feedback widget (bottom-right corner once you're signed in) and say "early adopter" — we'll come back to you.
No fine print, no "limited time" countdown timer. We just want to work with people early, and we're happy to price for it.
Start now
Built by Homemade Software. Questions, gripes, feature wishes — support@completionkit.com.