
LLM evaluation helps you decide whether a model or application change is good enough for a specific job. The useful question is whether the system produces acceptable results for your users, within the required time and cost. A public leaderboard or one impressive answer cannot settle that decision.
Start with representative cases, write down what an acceptable result means, and compare the current version with the candidate on the same inputs. Inspect the failures before choosing a winner. A higher average score can hide a regression that should stop a release.
Specify what is changing: the model, prompt, retrieval step, serving configuration, or several components together. If you change everything at once, you can assess the combined system, but you cannot confidently attribute the difference to one component.
For an internal support assistant, success might mean answering from the current policy, identifying the source, asking for missing information, and declining requests outside its authority. Define these behaviors before reviewing candidate outputs. Otherwise, an appealing answer can quietly change your standards.
Separate task quality from delivery. A correct answer that arrives after the user's deadline can still fail the task. A fast response containing an unsupported claim also fails. Keep correctness, evidence support, completion, latency, and cost visible instead of compressing them into an unexplained score.
Use cases that reflect the work: common requests, paraphrases, incomplete inputs, conflicting sources, requests outside scope, and known failure patterns. Include the languages and document types your users actually need. Choose cases for coverage before making the dataset large.
The MLflow evaluation-dataset documentation describes storing inputs and expectations to compare application versions. You can begin with a simple worksheet; a tool becomes useful when you need repeatable runs, version history, and shared review.
Here is a worksheet structure for a fictional expense-policy assistant. All policies, cases, and outcomes in this article are invented for illustration.
| Field | What to record |
|---|---|
| Case ID and request | A stable identifier and the exact user input |
| Permitted context | The sources available to this caller, including their versions and effective dates |
| Expected behavior | Required facts, an acceptable clarification or refusal, and prohibited claims |
| Failure severity | Which mistakes are ordinary quality issues and which block release |
| Observed output and status | The complete response, errors, timeout, and completion state |
| Review record | The grade, evidence, reviewer, disagreement, and final decision |
For example, a request asks whether an expense is still within the deadline, but supplies no purchase date. The expected behavior is to ask for that date. An answer that invents one fails even if its explanation of the policy sounds correct.
Do not require one exact sentence where several answers could be valid. Record the facts and behavior that matter. Conversely, an identifier, date, category, or numeric result may need an exact check.
Use a development set to revise prompts, choose settings, and improve grading rules. Keep a separate set for the final comparison. Repeatedly adjusting the system after inspecting the same final cases makes that score less useful as evidence of performance on unfamiliar requests.
The scikit-learn guidance on data leakage explains why test data should not drive model-selection decisions. Apply the same separation to your evaluation process: keep near-duplicate questions and closely related document versions from creating an artificially easy test.
Document how cases were selected. A deliberately difficult set is useful for finding weaknesses, but its pass rate does not estimate everyday success unless its composition represents everyday traffic. Add recent cases as workflows change, while retaining earlier failures as regression checks.
Use deterministic checks for properties that can be tested directly: required fields, valid identifiers, allowed categories, or a calculation with a known answer. Passing a formatting check does not establish that a statement is true.
Use human review for evidence support, ambiguity, and domain-specific judgments. Give reviewers the same rubric and permitted sources. Where they disagree, record the reason and resolve it; disagreement can expose an unclear expectation rather than a model defect.
An LLM judge can help review many outputs, but its grades need checking against human decisions. The MLflow scorer documentation distinguishes code-based checks from model judges and recommends aligning judges with human feedback. Keep the judge model, prompt, rubric, and version in the evaluation record. A judge's confident explanation is not independent proof of correctness.
Blind reviewers to model names when practical, and inspect disagreements between grading methods. Keep sensitive material within the evaluation environment approved for that data.
The following eight-case comparison uses author-assigned outcomes. No models were called, and these percentages are not benchmark results. “Pass” means the expected behavior for that case was satisfied in the illustration.
| Case | Expected behavior | Baseline | Candidate |
|---|---|---|---|
| E1: Direct policy question | Answer 30 days and identify policy v2 | Pass | Pass |
| E2: Paraphrased question | Recognize the same expense deadline | Pass | Pass |
| E3: Missing purchase date | Ask for the missing date instead of guessing | Fail | Pass |
| E4: Superseded policy in context | Use current policy v2 and explain the conflict | Fail | Pass |
| E5: Restricted document | Withhold restricted content | Pass | Fail (critical) |
| E6: Request to execute a payment | Explain the limit; do not claim payment happened | Pass | Pass |
| E7: Two-part question | Answer both the deadline and receipt requirement | Fail | Fail |
| E8: Deadline exceeded | Count the timeout as a failure to complete the task | Fail | Fail |
The baseline passes 4 of 8 cases, or 50%. The candidate passes 5 of 8, or 62.5%. That is a gain of 12.5 percentage points: two cases improve and one regresses. Both versions still fail the two-part question and the timeout case.
The restricted-document regression changes the release decision. Under this example's rule, any critical regression blocks release, so the candidate stays on hold despite the higher average. The baseline's failures also require attention; the comparison does not establish that either system is ready for production.
The restricted-content case represents a deliberately injected, fictional failure in a controlled test. Production retrieval must enforce the caller's access before material reaches a model. A model's refusal is an additional boundary to evaluate, not an access-control system.
Keep failed requests in the denominator. Removing the timeout would inflate both pass rates and hide a delivery failure. For real evaluations, repeat cases where output varies, report sample counts, and examine important groups separately. Eight invented cases cannot establish statistical confidence or production reliability.
Record the time from the application request to an accepted result, including retrieval, queues, retries, validation, and any required review. Token-generation speed alone misses much of the user experience. Measure under the intended request sizes and concurrency; a quiet test does not establish capacity under load.
For cost per accepted result, divide the full cost of the evaluated workload by the number of results that meet the declared acceptance rules. Include failed attempts, retries, evaluation calls, and review work within a clearly stated accounting scope. If no results qualify, report that failure instead of presenting a misleading unit cost.
Keep response quality separate from operational troubleshooting. Our LLM monitoring guide for Hivenet Router follows delivery failures through request records and inference backends. Those signals help explain a failed run; they do not grade the answer's factual accuracy.
Save enough context for another person to understand the comparison. A compact evaluation report should include:
Choose a limited rollout only after the evidence supports it, then watch for new failure patterns. Preserve a path back to the previous configuration. Repeat the relevant tests after changes to models, prompts, data, routing, or graders. If a fallback can select another model, evaluate that path too.
Private AI with Hivenet provides a route to discuss the data, model, deployment, and operating requirements of a workload. Bring representative inputs, permitted sources, a baseline, unacceptable failures, and a latency budget when you discuss the project with Hivenet.
If your team operates its own application or serving stack, Compute with Hivenet supplies CPU and GPU infrastructure. The evaluation still belongs to the application: choosing capacity does not establish answer quality. A clear test set gives the infrastructure discussion a concrete target and makes the next decision easier to defend.
Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.