Prompt Engineering Patterns: Structured Outputs, JSON Mode, Function Calling, Schema Validation

Structured output means constraining the model to produce responses in a specific format like JSON, XML, or a predefined schema. Production systems usually need parseable outputs, not free-form text. Modern LLM APIs from OpenAI, Anthropic, and Google all support structured output through schemas, eliminating the need for fragile prompt-based JSON extraction.

Structured output is enforced at the decoding level by constraining the model's token choices to only those that produce valid output according to a schema. This guarantees the response will parse correctly, unlike prompt-based JSON requests which can produce malformed output. OpenAI's structured outputs use a JSON Schema; Anthropic's Claude supports tool use with strict schemas.

JSON mode is an older feature that guarantees the output is valid JSON, but does not enforce a specific schema. Structured outputs are newer and stricter: they enforce both valid JSON and conformance to a specific schema you define. Always prefer structured outputs over plain JSON mode when supported, because they eliminate parsing errors and schema mismatches.

Function calling lets you describe a set of available tools to the model with their names, descriptions, and parameter schemas. The model chooses which function to call and produces arguments that match the schema. The application then executes the function and returns the result. Function calling is the foundation of LLM agents and structured tool use.

Structured output produces a response matching a single schema you define. Function calling produces a choice among multiple tools, each with its own schema. Function calling is for picking and invoking actions; structured output is for producing one specific response shape. Many APIs implement structured output as a special case of single-tool function calling under the hood.

Design schemas to be specific, well-typed, and minimal. Avoid optional fields that the model can ambiguously include or omit. Use enums for fixed value sets instead of free strings. Add field descriptions explaining what each field means. Test the schema against real outputs to find edge cases where the model produces valid but unintended structures.

XML tagging is using tags like instructions, context, and examples to delimit different parts of a prompt. It helps the model distinguish instructions from data, system rules from user input, and different example types. Anthropic specifically recommends XML tags for Claude prompts because the model is trained to recognize them as semantic markers.

Even with structured output enforcement, occasional failures occur. Handle them by validating the output against your schema after receiving it, retrying with the validation error included as feedback, falling back to a simpler schema, or using a smaller model to fix malformed output. Production systems should always treat schema validation as a safety net, not a guarantee. ---