Glossary
LLM-as-Judge
Using one LLM to evaluate another LLM's outputs at scale.
Definition
LLM-as-judge is an evaluation technique where a language model is given a rubric and asked to score or classify the outputs of another AI system. It scales evaluation beyond what human review can cover — thousands of outputs graded per hour.
Context
LLM-as-judge is now core to enterprise AI evaluation pipelines. It works best on tasks with clear rubrics (faithfulness to source, adherence to format, tone) and less well on nuanced subjective judgments where models share the same biases they're evaluating. Best practice combines LLM-as-judge for scale with periodic human review for calibration.