Eval frameworks - verified October 18, 2026

Promptfoo

Promptfoo is listed under eval frameworks. Promptfoo's README says it is now part of OpenAI and that it remains open source and MIT licensed. No date given. Its first listed price, in its own words: Community: free forever (plan price). Stated or listed on its pages: online evals. Read from the vendor's own pages on October 18, 2026.

Also Open source

Read from promptfoo.dev, raw.githubusercontent.com, github.com. Docs - Pricing - License - Compare every tool

What changed

Part of OpenAIPromptfoo
Promptfoo's README says it is now part of OpenAI and that it remains open source and MIT licensed. No date given.
“Promptfoo is now part of OpenAI.”
raw.githubusercontent.com
“Promptfoo remains open source and MIT licensed.”
raw.githubusercontent.com

Status, as we read it

Generally available (inferred)inferred

No page we read labels this product GA or beta. It is sold or downloadable without a beta label, so we read it as generally available.

Ownership, name or terms changes, as recorded: Promptfoo is now part of OpenAI (date not stated)
“Promptfoo is now part of OpenAI.”
raw.githubusercontent.com

How Promptfoo charges

Every price line Promptfoo states, in its own words and units, with the quote. Nothing is converted.

4 price lines for Promptfoo, in the vendor's own words and units
Price, in the vendor's words	Unit	Quote and source
Community: free forever	plan price	“Community Free Forever” promptfoo.dev
Community: red teaming up to 10k probes per monthinferredPlan from table layout (Community): the quote does not name it.	probes per month	“Red teaming (10k probes/month)” promptfoo.dev
Enterprise: custom	plan price	“Enterprise Custom For teams that need advanced features.” promptfoo.dev
On-Premise: custom	plan price	“On-Premise Custom” promptfoo.dev

Free tier and retention

Free tier: All LLM evaluation features in Communityinferred
Plan from table layout (Community): the quote does not name it.
“All LLM evaluation features”
promptfoo.dev
Data retention: Not stated
Not stated on the pricing page

Tracing and evaluation

OpenTelemetry / OTLP ingest: Not stated
Not stated on the pricing page or README
Tracing: Not stated
Not stated on the pricing page or README
Evals (LLM-as-a-judge): Not stated
Not stated on the pricing page or README
Online evals: Listed on pricing page
The pricing table marks plans with icons, so this is not a claim about which plan includes it.
“Continuous monitoring”
promptfoo.dev
Datasets: Not stated
Not stated on the pricing page or README

Prompts, playground and cost

Prompt management: Not stated
Not stated on the pricing page or README
Playground: Not stated
Not stated on the pricing page or README
Cost tracking: Not stated
Not stated on the pricing page or README

Self-hosting, license and compliance

Self-hosting: Run locally or self-host
“Run locally or self-host on your own infrastructure”
promptfoo.dev
License: MIT
“Promptfoo remains open source and MIT licensed.”
raw.githubusercontent.com
SOC 2 (as listed): Not stated
Not stated on the pricing page
HIPAA (as listed): Not stated
Not stated on the pricing page
Regions: Not stated
Not stated on the pricing page

SDKs and integrations

SDKs: Not stated
Not stated as a list on the pages reviewed
Integrations: Not stated
Not stated as a count

Vendor claims

Scale, speed and cost statements as Promptfoo makes them. Vendor-stated, not measured.

Vendor-stated: Vendor-stated: evals run locally
“LLM evals run 100% locally - your prompts never leave your machine”
raw.githubusercontent.com

FAQ

How much does Promptfoo cost?

In Promptfoo's own terms: Community: free forever (plan price); Community: red teaming up to 10k probes per month (probes per month); Enterprise: custom (plan price); On-Premise: custom (plan price). As of October 18, 2026; prices are not converted.

Does Promptfoo accept OpenTelemetry traces?

Not stated. Not stated on the pricing page or README.

Can I self-host Promptfoo?

Per the vendor: "Run locally or self-host on your own infrastructure" License: MIT.

What changed at Promptfoo?

Promptfoo's README says it is now part of OpenAI and that it remains open source and MIT licensed. No date given.

Sources

Source: promptfoo.dev/docs
Source: promptfoo.dev/pricing
Source: raw.githubusercontent.com/promptfoo/promptfoo/HEAD/README.md
Source: github.com/promptfoo/promptfoo/blob/main/LICENSE

Last verified 2026-10-18 from each vendor's own pricing pages, docs, READMEs and license files; every quote was string-matched against the page that day. This is not a ranking. Prices are shown in the vendor's own unit and change without notice, so check the linked source before you commit.

Newsletter

One email per week. New model releases, agent tooling changes and what they mean for what you build.