Structured outputs and tool calling

you need JSON your code can parse rather than prose, so you constrain the shape of the response instead of writing a regex over the answer.

Structured output

the response parsed cleanly and still had the wrong keys in it, because the flag only ever promised valid JSON.

JSON mode

you write out the types, required fields and enums once, and use the same thing for tool parameters and for the response shape.

JSON schema

you measure what fraction of a thousand runs actually validated against the schema, and that rate is the reliability number you report.

Schema conformance

at each step, delete every next-token option that would break the format, then sample from what remains.

Constrained decoding

you hand the local model a grammar and it cannot emit anything outside your format, not even a helpful preamble.

Grammar-based decoding

log every retry; a rising retry rate is an early warning that a prompt or model change degraded reliability.

Validation and retry loop

the model replies with the name of one of your functions and its arguments instead of an answer, and your code runs it and hands the result back.

Function calling / tool calling

the model kept picking the wrong tool until you rewrote the description like documentation for a junior engineer.

Tool definition / schema

you force one specific tool so the model has to answer in that schema instead of deciding to chat about it.

Tool choice

you send the output back referencing the call it answers, and when it failed you send the error as structured content rather than nothing.

Tool result

a while loop around an HTTP call, with a switch statement over tool names in the middle.

The tool loop

you stop capping it at one tool call and let the model keep calling tools until it decides it is finished.

Agentic loop

the model asked for three lookups in one turn and you ran them concurrently instead of one after another.

Parallel tool calls

the model called the same write tool twice, and it only mattered because you had not made the write idempotent.

Side effects and idempotency in tools

a USB standard for model tools; implement the server once, every compatible client can plug in.

Model Context Protocol (MCP)