May 11, 2026 ·
CompletionKit Cloud is live
CompletionKit Cloud is open. The hosted version of the prompt-evals engine we've been building: no install, no Rails app of your own to maintain, a free tier you can start on today, and your prompts versioned behind stable URLs your app calls directly.
If you've been treating prompts like configuration (change the string, ship, hope), this is the thing that turns "hope" into a number. Pick real inputs from production, run your prompt against them, score the outputs on the criteria you actually care about (tone, accuracy, format compliance, whatever matters), and gate every future change on "doesn't regress the baseline." That's the whole loop. Cloud just makes it fast enough to run on every edit.
What's in the box
- Prompts as endpoints. Every published version has a URL. Your app fetches the template + model and runs the LLM the way it already does. No SDK.
- LLM-as-judge scoring. Write metrics with 1–5 rubrics; a judge model applies them to every response in a run. New to the idea? The LLM-as-a-judge guide covers it properly.
- Multi-model. OpenAI, Anthropic, Ollama or any OpenAI-compatible local endpoint, and 100+ models via OpenRouter.
- Runs, datasets, versioned history. See exactly what got better and what broke, version over version.
- An MCP server per organization so you can drive prompts, runs, and datasets from an LLM agent.
The engine stays source-available and free to self-host: on GitHub, on RubyGems. Cloud is for the people who'd rather not run it themselves.
Early adopters get a real discount
We're launching, which means we want the first wave of teams who'll push the product and tell us where it bends. If that's you, there's a meaningful discount on the table, well beyond the published pricing.
Two ways to get the details:
- Email support@completionkit.com and tell us a bit about what you're building.
- Or hit the feedback widget (bottom-right corner once you're signed in) and say "early adopter" and we'll come back to you.
No fine print, no "limited time" countdown timer. We just want to work with people early, and we're happy to price for it.
Start now
Built by Homemade Software. Questions, gripes, feature wishes: support@completionkit.com or r/completionkit.
FAQ
Is the free tier really free, or is it a trial?
Free for keeps. 20 runs a month, 1,000 prompt fetches a month, three seats, 30 days of run history. No card required, no countdown timer. If you outgrow it, the paid tiers lift the caps; if you don't, stay on free.
If I'm already self-hosting, do I have to move to Cloud?
No. The engine stays source-available and free for any use, including production. Cloud is for people who'd rather not run the infrastructure. Same product on both paths; pick the one that fits your team.
Is my data isolated from other organizations on Cloud?
Yes. Every engine table carries an organization_id, and PostgreSQL row-level security policies enforce that the cloud_app database role can only see rows for the current tenant. The isolation is enforced at the database layer, not just in application code.
What does the early-adopter discount actually look like?
It varies by use case and we'd rather price it for you than post a number. Email support@completionkit.com with a couple of sentences on what you're building and we'll come back with something well below the published pricing.