How do you evaluate the output of an LLM?

LLM output evaluation combines automated metrics, LLM-as-judge, and human review. Automated metrics like BLEU and ROUGE work for tasks with reference outputs. LLM-as-judge uses a strong model to score outputs against criteria. Human evaluation remains the gold standard for nuanced quality. Production systems typically use a mix, with LLM-as-judge for fast iteration and human spot-checks for ground truth.