LLM Structured Output Explained: Getting Reliable JSON From AI Models

An app asks a model to read an invoice and return the number, date, total and currency. Most of the time it works. Then one response arrives wrapped in a Markdown code block, another adds “Let me know if you need more!”, and a third writes the date as 10/11/2026. The code that parses the reply crashes, or worse, stores the wrong date. Structured output techniques exist to make AI responses as dependable as an API.

We did not call a live model for this article. Instead we wrote six responses that imitate common real-world failures and validated them with Pydantic 2.14 in Python 3.13. The validation results below come from that run.

What is structured output?

Structured output is any approach that makes a large language model return data in a fixed, machine-readable shape instead of free text, so code can use it directly. JSON is the usual format, described by a JSON Schema: which fields exist, their types, which are required and which values are allowed. Typical uses include extraction from documents, classification, form filling, routing requests and calling tools or functions.

Levels of enforcement

The last row is not optional at any level. Provider features change, models are swapped, and no format guarantee covers whether the values are true.

Defining the schema

from typing import Literal
from pydantic import BaseModel, Field, field_validator

class Invoice(BaseModel):
    invoice_no: str
    date: str = Field(pattern=r"^\d{4}-\d{2}-\d{2}$")
    total: float = Field(ge=0)
    currency: Literal["INR", "USD", "EUR"]

    @field_validator("total")
    @classmethod
    def two_decimals(cls, v):
        if round(v, 2) != v:
            raise ValueError("total must have at most 2 decimals")
        return v

Pydantic generated a matching JSON Schema with Invoice.model_json_schema(): properties invoice_no, date, total and currency, all required. That schema can be sent to providers that support schema-constrained output, so one definition drives both generation and validation. In TypeScript, Zod plays the same role. If type hints and dataclasses are familiar, Pydantic models will feel natural.

How model output breaks

We validated six responses with Invoice.model_validate_json():

good            OK  1180.0
markdown fence  FAIL (1) ['root: json_invalid']
trailing text   FAIL (1) ['root: json_invalid']
wrong types     FAIL (4) ['invoice_no: string_type', 'date: string_pattern_mismatch',
                          'total: float_parsing', 'currency: literal_error']
missing field   FAIL (1) ['date: missing']
string number   OK  1180.0

Each failure mirrors something models really do when only prompted:

• Markdown fences: the JSON arrives inside ```json ... ```, because chat-tuned models format answers for humans.
• Chatty extras: a friendly sentence after the JSON makes it invalid.
• Wrong types and formats: the invoice number as a number, the date as “10/11/2026”, the total as “1,180” and the currency as “Rs”. Four errors in one response.
• Missing fields: a field the model could not find was simply left out.

Rescuing wrapped JSON

def extract_json(text):
    s, e = text.find("{"), text.rfind("}")
    return text[s:e + 1] if s != -1 and e > s else text

# markdown fence  after extract_json: OK INV-104 1180.0
# trailing text   after extract_json: OK INV-104 1180.0

Taking the text between the first { and the last } recovered both wrapped responses. It is a useful safety net, but a fragile one: it fails for arrays, for responses containing several objects, or when explanatory text itself contains braces. Treat it as a fallback, not a design.

Lax vs strict parsing

The “string number” response sent "total": "1180.00", a string rather than a number, and it passed. Pydantic’s default lax mode converts clean numeric strings to floats. That is convenient, but it means a type error in the model’s output can go unnoticed. If you want to detect when the model ignores the schema, for example to measure reliability, use strict mode, which would reject it. Choose deliberately: lax mode for robustness, strict mode for measurement and for fields where a type change would be suspicious.

Constrained decoding

Several providers and open-source inference tools offer schema-constrained output. Conceptually, the schema is turned into rules about which tokens may come next, and tokens that would break the format are blocked during generation. The model physically cannot produce a Markdown fence or a missing brace. This removes almost all syntax and shape errors. Some write-ups report that heavy constraints can slightly reduce answer quality on reasoning-heavy tasks, so measure on your own data. Supported schema features also vary between providers, so check what your provider accepts.

Valid is not the same as correct

Schema validation checks shape. It cannot tell whether 1080 or 1180 is the right total. We added a simple semantic check, confirming the extracted total appears in the source text:

semantic check (total in source): True
semantic check (wrong total):     False

A response with total 1080 was perfectly valid JSON with the right types, and it was still wrong. Business-rule checks catch these: totals that match line items, dates that are not in the future, IDs that exist in your database, values that appear in the source document. These are the checks that protect real users.

Retry and repair loops

1. Call the model with the schema and clear instructions.
2. Parse and validate the response.
3. If it fails, call again with the validation errors included, such as “date must match YYYY-MM-DD”.
4. Stop after a small number of attempts, then fall back: queue for human review, return a clear error, or use a default.
5. Log every failure with the raw response so you can improve the prompt or schema.

Pydantic’s error list, such as date: string_pattern_mismatch, is precise enough to feed back to the model. Keep retries bounded; an unbounded loop can multiply costs quickly.

Schema design tips

• Keep it flat and small. Deeply nested schemas fail more often.
• Use enums for categories instead of free text.
• Use explicit formats for dates and amounts, and say so in field descriptions.
• Allow null for information that may be missing.
• Add a confidence or notes field only if your code will actually use it.
• Version your schema, since changing it is a contract change, as discussed in API contract testing.

Testing structured output

Build a test set of real inputs with expected outputs, including messy documents and cases where fields are missing. Run it on every prompt, schema or model change, and track three numbers: the share of responses that parse, the share that pass validation, and the share whose values are correct. Unit-test your validators with deliberately broken outputs, like our six examples, so that parsing code never surprises you in production.

Putting it together: a minimal pipeline

1. One model definition in Pydantic or Zod, with field descriptions and allowed values.
2. Generate the JSON Schema from it and pass it to the provider’s structured-output or tool-calling feature.
3. Parse and validate the reply with the same model, falling back to JSON extraction only if needed.
4. Run semantic checks against the source data and business rules.
5. Retry once or twice with the error messages, then route failures to a human queue.
6. Log the raw input, raw output, errors and settings for every call, so failures can be reproduced and fixed.

Each layer catches problems the others miss. Together they turn a probabilistic model into a component the rest of your system can depend on.

Final thoughts

Getting JSON from a model is easy; getting reliable JSON takes layers. Define the schema once, in Pydantic or Zod, use your provider’s schema-constrained output where available, validate every response in your own code, add semantic checks for values that matter, and retry with error feedback a limited number of times. In our test, four of six prompt-style responses failed validation on first parse, and schema checks alone would have accepted a wrong total.

Frequently asked questions

Keep learning