Structured output is a shape guarantee with semantic work left
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Use a typed structured-output schema for extraction.
- Distinguish schema validity from factual support.
- Handle absent parsed output and evidence checks.
Choose a schema from the consumer backward
Structured output asks a supported model to produce data conforming to a specified schema. Begin with the code that will consume the result: which fields must exist, what their types mean, and how uncertainty is represented. A schema designed only to resemble a nice paragraph can remain difficult to validate or act on. Prefer a small number of fields with stable operational meaning.
For a support-ticket extractor, use an issue summary, a category from a fixed set, and a quoted evidence span. A category enum makes routing predictable. An unknown category provides a legitimate result when the source is insufficient. The evidence field gives the application something concrete to check, though a matching quote alone does not establish that the category follows logically from it.
Syntax, schema, and semantics are three checks
Plain JSON parsing checks whether bytes form a valid JSON value. A schema adds constraints such as required keys, field types, and allowed enum members. Semantic validation asks whether those values are supported by the source and permitted by the application. These checks solve different problems and should remain distinct in error reporting.
Suppose a model returns {"category":"hardware","evidence_quote":"battery is swollen"} for a ticket that only says "the app crashes at login." The object can be valid JSON and match the schema while inventing evidence. An exact-substring check rejects that quote. Conversely, a quote can be present but interpreted incorrectly, so production evaluation should also examine the relationship between the evidence and the conclusion.
Use documented schema tooling and explicit absence
The OpenAI Python SDK documents a Responses parse helper that accepts a Pydantic model through text_format and exposes parsed output through output_parsed. This can keep the Python type and the requested output schema aligned. Supported schema features and model capabilities must still be checked in the official guide; arbitrary schema constructs are not automatically supported by every model or interface.
A request can refuse, fail, or become incomplete rather than return the expected parsed object. Do not call methods on output_parsed without checking it exists, and do not turn a missing object into a fabricated default that looks like a successful extraction. Represent absence in the application lifecycle. The lesson's code requires completed status and a parsed value before applying additional evidence validation.
Design evolution and repair deliberately
Schemas become contracts between model adapters, stored runs, and downstream code. Changing a field from string to list or changing an enum member's meaning can break readers even if the new response is valid under a revised schema. Include a schema version in persistent application envelopes and add migration logic when necessary. Keep a small fixture suite for old and new consumers.
If semantic validation fails, a bounded repair request can explain the specific problem and ask for a corrected result. It should not repeatedly pressure the model to invent a value that the source does not contain. Sometimes the correct repair is an explicit unknown result or a request for missing input. Count repair attempts and evaluate how often they help; retries are part of the system's cost and latency.
Extracted data should carry its evidence
The code passes one fixed ticket to a typed extractor, checks completion and parsed output, then verifies that the evidence quote appears in the original text. It requires openai and pydantic packages plus backend credentials and an available model configured through OPENAI_MODEL. The response remains illustrative because no live request was made during content validation.
For a stronger application, store source identifiers and offsets rather than only text fragments, and preserve the original source revision. A later edit can otherwise make a once-valid quote appear unsupported. Evaluate both field accuracy and abstention behavior on ambiguous examples. The project uses deterministic fixtures to test these checks independently from the provider, so a schema or evidence regression can be reproduced without spending another model request.
Work through the code
Prerequisites: compatible openai and Pydantic packages, OPENAI_API_KEY on the backend, and OPENAI_MODEL set to an available model supporting structured output. The API call and parse helper were checked against official documentation and syntax-compiled; no live call was made. The substring check detects invented quotes but does not prove the category is correct.
import os
from typing import Literal
from openai import OpenAI
from pydantic import BaseModel, ConfigDict
class TicketExtraction(BaseModel):
model_config = ConfigDict(extra="forbid")
issue: str
category: Literal["hardware", "software", "unknown"]
evidence_quote: str
source = "Device R2 reboots after a firmware update."
client = OpenAI()
response = client.responses.parse(
model=os.environ["OPENAI_MODEL"],
input=[
{"role": "system", "content": "Extract the issue. Quote exact source text as evidence. Use unknown when the category is unclear."},
{"role": "user", "content": source},
],
text_format=TicketExtraction,
)
parsed = response.output_parsed
if response.status != "completed" or parsed is None:
raise RuntimeError("No completed structured extraction")
if not parsed.evidence_quote or parsed.evidence_quote not in source:
raise ValueError("Evidence quote is not supported by source")
print(parsed.model_dump_json(indent=2))
Illustrative output: depends on the configured model and account; no API request was made during course validation.
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
Your schema requires an ISO date string, but a source says only "next Thursday" without a reference date or timezone. What schema or workflow change avoids manufacturing certainty?
Check your understanding
A model returns a schema-valid object containing a fabricated quote. Which layer should reject it?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.