프로덕션 지원 LLM 에이전트: 오프라인 평가를 위한 포괄적인 프레임워크

Introduction & Context This scene plays out frequently across the industry. We’ve become remarkably good at building sophisticated agent systems, but we haven’t developed the same rigor around proving they work. When I ask teams how they validate their agents before deployment, I typically hear some combination of “we tested it manually,” “the demo went well,” and “we’ll monitor it in production.” None of these are wrong, but none of them constitute a quality gate that governance can sign off on or that engineering can automate. The Problem: Evaluating Non-deterministic Multi-Agent Systems The challenge isn’t that teams don’t care about quality — they do. The challenge is that evaluating LLM-based systems is genuinely hard, and multi-agent architectures make it harder. Traditional software testing assumes determinism. Given input X, we expect output Y, and we write an assertion to validate. But if we ask an LLM the same question twice and we’ll get different phrasings, different structures, sometimes different emphasis. Both responses might be correct. Or one might be subtly wrong in ways that aren’t obvious without domain expertise. The assertion-based mental model breaks down. Now multiply this complexity across a multi-agent system. A router agent decides which specialist handles the query. That specialist might retrieve documents from a knowledge base. The retrieved context shapes the generated response. A failure anywhere in this chain degrades the output, but diagnosing where things went wrong requires evaluating each component. I’ve observed that teams need answers to three distinct questions before they can confidently deploy: - Is the router doing its job? When a user asks a simple question, does it go to the fast, cheap agent? When they ask something complex, does it route to the agent with deeper capabilities? Getting this wrong has real consequences — either you’re wasting money and time on over-engineered responses, or you’re giving users shallow answers to questions that deserve depth. - Are the responses actually good? This sounds obvious, but “good” has multiple dimensions. Is the information accurate? If the agent is doing analysis, is the reasoning sound? If it’s generating a report, is it complete? Different query types need different quality criteria. - For agents using retrieval, is the RAG pipeline working? Did we pull the right documents? Did the agent actually use them, or did it hallucinate information that sounds plausible but isn’t grounded in the retrieved context? Offline vs Online: A Brief Distinction Before diving into the framework, I want to clarify what I mean by “offline evaluation” because the terminology can be confusing. Offline evaluation happens before deployment, against a curated dataset where you know the expected outcomes. You’re testing in a controlled environment with no user impact. This is your quality gate — the checkpoint that determines whether a model version is ready for production. Online evaluation happens after deployment, against live traffic. You’re monitoring real user interactions, sampling responses for quality checks, detecting drift. This is your safety net — the ongoing assurance that production behavior matches expectations. Both matter, but they serve different purposes. This article focuses on offline evaluation because that’s where I see the biggest gap in current practice. Teams often jump straight to “we’ll monitor it in production” without establishing what “good” looks like beforehand. That’s backwards. You need offline evaluation to define your quality baseline before online evaluation can tell you whether you’re maintaining it. Article Roadmap Here, I present a framework I’ve developed and refined across multiple agent deployments. I’ll walk through a reference architecture that illustrates common evaluation challenges, then introduce what I call the Three Pillars of offline evaluation — routing, LLM-as-judge, and RAG evaluation. For each pillar, I’ll explain not just what to measure but why it matters and how to interpret the results. Finally, I’ll cover how to operationalize with automation (CI/CD) and connect it to governance requirements. The System under Evaluation Reference Architecture To make this concrete, I’ll take an example that is becoming more common in the current environment. A financial services company is modernizing its tools and services supporting its advisors who serve end customers. One of the applications is a financial research assistant with capabilities to lookup financial instruments, do various analysis and conduct detailed research. This is architected as a multi agent system with different agents using different models based on task need and complexity. The router agent sits at the front, classifying incoming queries by complexity and directing them appropriately. Done well, this optimizes both cost and user experience. Done poorly, it creates frustrating mismatches — users waiting for simple answers, or