What metrics should you monitor in production LLM systems?

Monitor latency including TTFT and inter-token latency, throughput in tokens per second, cost per request and per user, error rates, retrieval quality metrics for RAG systems, user feedback scores, hallucination rates from automated checks, and content filter trigger rates. Set up alerts for regressions on any of these. Most teams use observability tools like LangSmith, Arize, or LangFuse.