
LLM structured output gives an application a predictable shape for a model response, such as a JSON object with defined fields. That makes integration easier, but a well-formed record can still contain invented evidence, misunderstand a request, or describe an action that never happened.
Start with the contract your application needs. Then check the serving system's support, validate each completed response, and decide what happens when a result cannot be accepted. A schema is one part of that process.
| Check | Question it answers | What it does not establish |
|---|---|---|
| JSON parsing | Can the response be read as JSON? | That required fields or allowed values are present |
| Schema validation | Does the record match the declared structure? | That its claims are supported by the input |
| Application validation | Do identifiers, evidence, and business rules support this result? | That the caller has permission to perform an action |
| Authorization and execution | May this operation run, and what actually happened? | That a model's proposed outcome already occurred |
Asking a model to return JSON, enabling a provider's JSON mode, and using schema-constrained generation are different choices. Read the exact endpoint's guarantees. Parseable JSON alone does not impose your field requirements; even constrained generation needs an application-side acceptance check.
Our OpenAI-compatible API buyer guide explains why a familiar API format is only a starting point for compatibility testing.
Consider a fictional assistant that prepares support-ticket records for review. Its source message, identified as msg-1, says: “I was charged twice for order A-104.” The assistant may summarize the customer's report and suggest a category. It cannot confirm the charge history or approve a refund.
An example record is:
{
"category": "billing",
"order_id": "A-104",
"source_ids": [
"msg-1"
],
"summary": "Customer reports two charges for order A-104."
}
The application owns the ticket identity and the allowed source messages. The model supplies only four fields: a category, an order reference if known, source references, and a short summary. Use an explicit unknown category when the input does not establish the request.
This example application schema uses JSON Schema Draft 2020-12:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"additionalProperties": false,
"required": [
"category",
"order_id",
"source_ids",
"summary"
],
"properties": {
"category": {
"type": "string",
"enum": [
"billing",
"technical",
"unknown"
]
},
"order_id": {
"type": [
"string",
"null"
],
"minLength": 1
},
"source_ids": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"uniqueItems": true
},
"summary": {
"type": "string",
"minLength": 1,
"maxLength": 240
}
}
}
JSON Schema's object documentation explains required properties and extra-property controls. Here, all four fields are required and additional fields are rejected. An invented approve_refund field therefore fails validation.
A required field can still allow null. In this contract, order_id: null means the order reference is unavailable; omitting the field is a different result. The JSON Schema null reference makes that distinction explicit. Do not force the model to invent a value just to fill a required string.
Keep meanings stable and version the contract. If you change the allowed categories or the interpretation of an empty evidence list, update consumers and evaluation cases together. This is a local validation schema, not a promise that every inference endpoint accepts every keyword it contains.
Schema support belongs to a specific engine, version, decoding backend, model configuration, and request path. The vLLM structured-output documentation describes schema-constrained generation and version-sensitive request options. Match examples to the version you operate rather than copying older fields without checking them.
Test the actual schema on the intended endpoint. Record which constraints are accepted, whether unsupported ones cause an explicit error, how incomplete output is reported, and where the final content appears. Streaming, reasoning output, and tools may add response handling that a basic text request does not exercise.
If multiple backends are eligible, test each one. A fallback that returns ordinary prose or interprets a field differently can break the contract even though the HTTP request succeeds. The Hivenet Router model-routing guide covers routing rules and capability checks; those do not replace validation of the returned record.
We checked the example schema locally with Python jsonschema 4.26.0 and its Draft 2020-12 validator. The validator documentation explains schema checking and instance validation. The 12 fixtures exercised JSON parsing and schema rules only; no live model, serving engine, or Router endpoint was called.
| Fixture group | Observed local result | Next application decision |
|---|---|---|
| Ordinary record; explicit unknown with a null order reference | Both pass the schema | Validate the ordinary record; route the unknown case to review |
| Missing field, wrong order type, unsupported category, extra action, null summary, empty summary, duplicate source reference | All seven fail the schema | Keep them out of downstream processing |
| Truncated JSON | Fails parsing | Handle as an incomplete response |
| Invented source ID; summary claiming a refund was approved and paid | Both pass the schema | Reject unsupported evidence or claims through separate application checks |
The last row is the important limitation. A string can satisfy the schema while pointing to no supplied message. A refund claim can fit within the summary length while contradicting the task. These examples demonstrate the boundary of schema validation, not model accuracy.
For this workflow, check that every source reference belongs to the authorized input set and actually supports the statement. Validate an order reference against the permitted context, and preserve the distinction between “the customer reports a charge” and “the charge is verified.” A reference that exists is not, by itself, supporting evidence.
Give unknown, refused, incomplete, and invalid responses distinct outcomes. An API-level refusal or error may arrive outside your business-record schema. Do not turn it into a fabricated record with default values merely to satisfy a downstream parser.
When receiving a stream, accumulate the expected completed content and check the documented completion state before accepting it. A partial record can look plausible before the response is finished. If the stream ends early or reaches a length limit, follow the incomplete-response path.
Set bounded retries with an overall time and cost budget. A malformed response may justify another attempt; an unsupported schema or a permission failure calls for a different remedy. Repeating the same incompatible request indefinitely increases cost without fixing the cause.
A generated field must never serve as proof that a refund, account change, or external write succeeded. The component executing that action must report its actual result. Treat instructions inside customer messages as data, not permission to bypass those controls.
Build a labeled evaluation set with straightforward requests, missing information, contradictory evidence, invalid source references, long messages, and unsupported requests. Keep final test cases separate from the examples used to tune prompts or acceptance rules.
Report schema pass rate separately from task correctness and evidence support. Also measure unknown and review rates, failures after retries, end-to-end latency, and total cost per accepted record. Include rejected generations, retries, validation, and review work in the cost. A high JSON success rate can coexist with an unusable application.
Repeat the evaluation after changes to models, prompts, schemas, decoding backends, or routing. The local fixtures above remain useful regression checks for the validator, but they do not establish how an inference deployment behaves under real traffic.
Compute with Hivenet provides infrastructure for customer-operated applications and serving stacks. Keep output validation in the application even when the chosen backend supports constrained generation. If several self-hosted backends need routing, verify the contract across every eligible path.
For a sensitive or custom workflow, Private AI with Hivenet provides a route to discuss the data, model, endpoint, and operating requirements. Bring the schema, representative inputs, unacceptable outputs, access rules, and acceptance tests when you discuss the workload with Hivenet. That makes the first evaluation about records the application can trust enough to use.
Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.