Online Evaluation

In depth

Scoring an LLM application's behavior on live production traffic as it happens, rather than on a curated test set before deployment. Because production traces have no reference outputs, online evaluators focus on reference-free checks such as safety, format, tone, and LLM-as-a-judge quality scores. Offline evaluation is the counterpart: it runs on datasets with expected answers and is used for pre-deployment and regression testing.

Primary source: docs.langchain.com