LLM-as-judge
Using a model to score output against a written rubric, in place of a human.
By Lattice editorial · Entry last reviewed · method
What it implies
It works, with two caveats that are frequently ignored. Position bias: the same rubric graded with the answer and the reference in swapped order produces different scores, so randomise it. And the judge must be stronger than what it is judging — a weak judge grading a strong model measures the judge.