Promptfoo alternatives
Prices checked 2026-10-17 on each vendor's own pricing page.
Short answer: Promptfoo is now part of OpenAI and still MIT-licensed. For another open-source test framework, DeepEval (Apache-2.0, pytest-style Python tests) and Ragas (Apache-2.0, RAG and agent metrics) both run locally and in CI. If you also want tracing, datasets and human review in one place, Langfuse, Comet Opik and Helicone are open source and self-hostable, while Braintrust, LangSmith and Confident AI are hosted.
For reference, Promptfoo: free tier: Community: free, all LLM evaluation features, red teaming up to 10k probes/month [source, checked 2026-10-17]; entry paid: Enterprise: custom pricing (no public price) [source, checked 2026-10-17]
How we rank: Tools are grouped by type (open source, hosted, frameworks) and listed alphabetically within each group, not ranked. Every entry gets the same fields, taken from vendor pricing pages, docs and official repositories on the checked date, with a source link; anything we couldn't confirm says “not verified”. Commission and sponsorship don't affect order or what we say. We show no star ratings or user scores. How we review
Open source and self-hostable
Code you can run on your own infrastructure.
| Tool | Best for | Licence | Free tier | Entry paid price |
|---|---|---|---|---|
| Arize AX and Phoenix | Teams that want an OpenTelemetry-native stack: Phoenix to self-host for free, Arize AX when they want a managed service. | Phoenix: Elastic License 2.0. Arize AX: hosted service [source] | AX Free: 25k spans/month, 1 GB ingestion, 15-day retention, unlimited users [source, checked 2026-10-17] | AX Pro $50/month: 50k spans, 10 GB, 30-day retention (overage rates not stated on the pricing page) [source, checked 2026-10-17] |
| Comet Opik | Teams that want a fully open-source tracing and evaluation platform with a low-cost cloud option. | Apache-2.0 [source] | Free Cloud: 25k spans/month, up to 10 team members, 60-day retention; open source self-hosted is free [source, checked 2026-10-17] | Pro Cloud $19/month: 100k spans, up to 50 members, then $5 per 100k spans [source, checked 2026-10-17] |
| Helicone | Teams that want request logging, cost tracking and a gateway first, with scores and datasets on top. | Apache-2.0 [source] | Hobby: 10,000 requests/month, 1 GB storage, 1 seat, 7-day retention [source, checked 2026-10-17] | Pro $79/month plus usage-based pricing (usage rates are shown only via a calculator, not verified) [source, checked 2026-10-17] |
| Langfuse | Teams that want open-source, self-hostable tracing, prompt management and evals with usage-based cloud pricing. | MIT for code outside the ee/ directories; ee/ code under a separate licence [source] | Hobby: 50k units/month, 30 days data access, 2 users [source, checked 2026-10-17] | Core $29/month: 100k units included, then $8 per 100k units (graduated down to $6 per 100k above 50M) [source, checked 2026-10-17] |
Hosted platforms
Managed services; self-hosting, where offered, is on Enterprise plans.
| Tool | Best for | Licence | Free tier | Entry paid price |
|---|---|---|---|---|
| Braintrust | Product and engineering teams that run many experiments and want evals, human review and prompt playgrounds in one hosted tool. | Hosted service; its autoevals scorer library is MIT [source] | Starter $0: 1 GB processed data, 10k scores, 14-day retention, unlimited users; then $4/GB and $2.50 per 1k scores [source, checked 2026-10-17] | Pro $249/month: 5 GB and 50k scores included, then $3/GB and $1.50 per 1k scores; 30-day retention [source, checked 2026-10-17] |
| DeepEval and Confident AI | Python teams that want pytest-style LLM tests with ready-made RAG and agent metrics, with an optional hosted platform. | DeepEval: Apache-2.0. Confident AI: hosted service [source] | Free: 2 seats, 1 project, 5 test runs/week, 1 GB-month of trace spans [source, checked 2026-10-17] | Starter $200/month per organization: unlimited seats, 5 projects, 5 GB-months, then $1 per GB-month [source, checked 2026-10-17] |
| Galileo (Splunk Agent Observability) | Enterprises already on Splunk Observability that want agent evaluation and guardrails in the same contract. | Commercial product (Splunk) [source] | not verified | From $100/month per 1.2 million spans, billed annually; includes 500 million Luna tokens and 5 Luna fine-tuning runs per year [source, checked 2026-10-17] |
| LangSmith | Teams building on LangChain or LangGraph that want tracing, evals and agent deployment from the same vendor. | Hosted service; self-hosted and hybrid options on Enterprise [source] | Developer $0: 1 seat, up to 5k base traces/month, then pay as you go [source, checked 2026-10-17] | Plus $39/seat/month: up to 10k base traces/month, then pay as you go (per-trace rate not stated on the pricing page) [source, checked 2026-10-17] |
| Maxim AI | Teams testing agents with simulations and wanting experimentation, evals and observability priced per seat. | Platform is a hosted service; Bifrost gateway is Apache-2.0 [source] | Developer: free for up to 3 seats, 1 workspace, 10k logs/month, 3-day retention, no overages [source, checked 2026-10-17] | Professional $29/seat/month: 100k logs/month, 7-day retention, $1 per 10k extra logs [source, checked 2026-10-17] |
| W&B Weave | Teams already using W&B for model training that want LLM tracing and evals in the same account. | Weave SDK: Apache-2.0; the platform is a hosted service [source] | Free $0: 1 GB/month Weave data ingestion [source, checked 2026-10-17] | Pro from $60/month: 1.5 GB/month ingestion, then $0.10 per MB; for teams under 50 employees [source, checked 2026-10-17] |
Frameworks for tests in CI
Libraries and CLIs rather than full platforms.
| Tool | Best for | Licence | Free tier | Entry paid price |
|---|---|---|---|---|
| Ragas | Teams that need RAG and agent metrics as a Python library inside their own pipeline or another platform. | Apache-2.0 [source] | Free, open source [source, checked 2026-10-17] | No paid plan published [source, checked 2026-10-17] |
Head-to-head
Other alternatives pages: Langfuse alternatives · LangSmith alternatives · Braintrust alternatives · Arize AX and Phoenix alternatives · DeepEval and Confident AI alternatives · Helicone alternatives
FAQ
Is Promptfoo still open source after joining OpenAI?
Yes. Its README, read 16 Oct 2026, says Promptfoo is now part of OpenAI and remains open source under the MIT licence. The free Community version includes all LLM evaluation features and red teaming up to 10k probes a month (Promptfoo pricing page, checked 17 Oct 2026).
What is a Python alternative to Promptfoo?
DeepEval and Ragas are Python libraries. DeepEval writes tests as pytest-style Python test cases with ready-made RAG and agent metrics; Ragas is a library of LLM-judged RAG and agent metrics that you call inside your own pipeline. Promptfoo itself is configured in YAML and run from a CLI.
Which alternatives run in CI for free?
DeepEval documents unit testing in CI/CD, Ragas documents adding its metrics to CI with pytest, and Comet Opik has a PyTest integration; all three are open source. Promptfoo's own CI/CD integration is part of its free Community version.