LLM-as-a-judge means asking a language model to grade another model's output: score an answer against a rubric, pick the better of two responses, or rank several candidates. It scales evaluation far beyond what people can review by hand, which is why most eval suites for prompts and agents use it.
A judge is still a model, with its own biases. It tends to prefer the first or the longer answer, can favour text written in its own style, and gives different scores when the wording of the rubric changes. Clear rubrics, swapping the order of candidates, panels of several judges and spot checks by people keep it honest. These guides cover judges inside agent evals, generate-rank-respond with prompt ensembles, and using a jury of models instead of a single judge.
The library lists all guides with filters for topic, level, category and reading time.