Structured outputs and validation

01The problem

Paystream's triage prompt asks for {"category": "..."}. Most of the time that's what comes back. But across thousands of calls, some responses arrive wrapped in a sentence ("Sure! Here's the category: ..."), some use a category that isn't on the list ("Login problem"), and occasionally one is cut off halfway. If the code that routes tickets assumes every response is perfect, a single odd reply can crash the job or send a fraud report to the wrong team.

Any system built on an LLM needs a layer that checks every output before it's used. In software, that's validation, and it's one of the most important habits in AI engineering.

02The concept

Ask for structure

Ask for JSON with a fixed shape, and show the shape in the prompt. Many APIs also support tool use or structured output modes that make the model fill in a defined schema, which reduces (but doesn't eliminate) malformed responses.

Validate everything

A schema library such as pydantic describes what a valid output looks like, then checks each response:

• Is it valid JSON?
• Does it have the required fields?
• Is each value allowed (for example, one of the defined categories)?

Handle failures deliberately

Log every failure: a rising failure rate is an early sign that a prompt change or a model update has broken something.

03Example

Define the schema, then validate a batch of raw responses like the ones a real triage job produces:

import json
import re
from typing import Literal
from pydantic import BaseModel, ValidationError

Category = Literal["Failed or pending transfer", "Fees and charges", "Account access", "Verification and limits",
                   "Cards", "Fraud or scam", "Cash-out agent", "Savings", "Unclear"]

class Triage(BaseModel):
    category: Category

raw_responses = [
    '{"category": "Cards"}',
    'Sure! Here is the category: {"category": "Fraud or scam"}',
    '{"category": "Login problem"}',
    '{"category": "Savings"',
    '{"category": "Fees and charges"}',
    '{"Category": "Cards"}',
]

def parse(raw):
    """Return a valid Triage, or None if the response can't be trusted."""
    match = re.search(r"\{.*\}", raw, re.S)
    if not match:
        return None
    try:
        return Triage.model_validate(json.loads(match.group()))
    except (json.JSONDecodeError, ValidationError):
        return None

results = [parse(r) for r in raw_responses]
for raw, result in zip(raw_responses, results):
    print(f"{'OK    ' if result else 'REVIEW'} {raw}")
print("Valid:", sum(r is not None for r in results), "of", len(results))

OK     {"category": "Cards"}
OK     Sure! Here is the category: {"category": "Fraud or scam"}
REVIEW {"category": "Login problem"}
REVIEW {"category": "Savings"
OK     {"category": "Fees and charges"}
REVIEW {"Category": "Cards"}
Valid: 3 of 6

The extra sentence around the second response is stripped and the JSON inside is accepted. The invented category, the cut-off response and the wrongly capitalised field name are all rejected. In production, each rejected response would be retried once, then sent to a person.

04Walkthrough

1. Run the cell. Add another bad response of your own and check it's rejected.
2. Add a confidence field (a number from 0 to 1) to the schema, and check that a value of 1.5 fails validation.
3. Write a triage_with_retry function outline (in comments) for what happens after a failure.
4. Decide what "needs review" means at Paystream: who reviews, and how fast?

05Practice

06Check your understanding