Structured Outputs in OpenAI API: Complete Guide

Somewhere in my git history is a function I'm not proud of. It's called cleanModelJson(), it's forty lines long, and its job was to take whatever an LLM claimed was JSON and perform surgery: strip markdown code fences, delete the friendly "Here's your JSON:" preamble, hunt trailing commas, and pray. It ran before every JSON.parse in the project. It still failed about once a week, always at night.

If you've built anything on LLM APIs, you have your own version of that function. This guide is about deleting it — because OpenAI structured outputs make the model's response guaranteed to match your schema, and the difference between "usually JSON-shaped" and "guaranteed" turns out to be the difference between a demo and a product.

The three eras of getting JSON from a model

Era one: begging. You write "Respond ONLY with valid JSON, no other text" in the prompt, and the model complies — mostly. Prompt-begging fails in creative ways: fences around the JSON, a missing field when the model felt one was irrelevant, an apology paragraph when it got confused. My 2 a.m. failures all came from here.

Era two: JSON mode. Setting response_format: { type: "json_object" } guarantees the output parses as valid JSON. Real progress — no more fences or preambles. But valid is not correct: JSON mode makes no promise about structure. The model can still rename your field from category to type, skip one it deems optional, or nest things creatively. Your parse succeeds; your code breaks one line later.

Era three: structured outputs. You provide a JSON Schema with strict: true, and the API constrains the model's generation so the output cannot deviate from it. Not "is asked nicely to" — cannot. The tokens that would break the schema are never allowed to be produced. This is the era where cleanModelJson() gets deleted with prejudice.

The basic setup

Here's a real example — the support-ticket classifier that runs in my n8n triage workflow:

const response = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    { role: 'system', content: 'Classify the support message.' },
    { role: 'user', content: ticketText },
  ],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'ticket_classification',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          category: { type: 'string', enum: ['billing', 'technical', 'account', 'other'] },
          urgency: { type: 'string', enum: ['low', 'medium', 'high'] },
          summary: { type: 'string' },
          needs_human: { type: 'boolean' },
        },
        required: ['category', 'urgency', 'summary', 'needs_human'],
        additionalProperties: false,
      },
    },
  },
});

const result = JSON.parse(response.choices[0].message.content!);
// result.category is one of exactly four strings. Always. Every time.

Notice the enum fields. This is structured outputs' quiet superpower: the model can't invent a fifth category or write "Billing" with a capital B. Downstream switch statements become trustworthy.

The Zod shortcut (use this one)

Hand-writing JSON Schema gets old fast, and keeping it in sync with your TypeScript types is a bug farm. The official SDK ships a helper that derives the schema from Zod and gives you a parsed, typed result:

import { zodResponseFormat } from 'openai/helpers/zod';
import { z } from 'zod';

const Classification = z.object({
  category: z.enum(['billing', 'technical', 'account', 'other']),
  urgency: z.enum(['low', 'medium', 'high']),
  summary: z.string(),
  needs_human: z.boolean(),
});

const completion = await openai.chat.completions.parse({
  model: 'gpt-4o-mini',
  messages,
  response_format: zodResponseFormat(Classification, 'ticket_classification'),
});

const result = completion.choices[0].message.parsed; // fully typed, already validated

One schema, three jobs: it constrains the model, validates the response, and types your code. This is the version I use everywhere now.

The strict-mode rules that will trip you

Strict mode buys its guarantee by restricting what schemas it accepts, and the restrictions are exactly where everyone stumbles:

Every property must be in required. No optional fields — which sounds insane until you learn the idiom: optional-ness is expressed with a nullable union. z.string().nullable() (type ["string", "null"]) means "the model must always include this key, but may set it to null." Same semantics, schema-compatible.

additionalProperties: false is mandatory on every object. Zod's helper handles it; hand-writers forget it and get a rejection.

Not every JSON Schema feature is supported. Fancy keywords like minLength, numeric ranges, and format validators aren't part of the constrained decoding. The schema guarantees shape; semantic rules ("summary under 100 characters") still belong in the prompt, and truly critical constraints get a post-check in code.

First call with a new schema is slower. The API compiles the schema's grammar on first use, adding noticeable latency once, then caches it. Don't panic-debug the first request like I did.

Refusals: the field nobody reads about

If the model declines a request on safety grounds, it obviously can't answer and match your schema. Instead of garbage, you get a refusal field on the message. Handle it:

const msg = completion.choices[0].message;
if (msg.refusal) {
  // log it, route to a human, don't parse
}

It's rare in normal workloads — and precisely because it's rare, unhandled refusals produce the most confusing bug reports.

It works on tools too

The same strict: true applies to function definitions in tool calling, which quietly fixes the other half of the broken-JSON problem: tool arguments. Before strict tools, my validation layer regularly caught the model inventing argument fields. With strict schemas on the tool parameters, the arguments arrive shaped exactly right — I still validate (model-generated input stays untrusted input), but the validator has stopped finding anything.

What the guarantee doesn't cover

One honest boundary so this guide oversells nothing: structured outputs guarantee the container, not the contents. The category will absolutely be one of your four enums — but whether it's the right one of the four is still the model's judgment, which means prompt quality, good field naming, and evals still matter. I once watched a perfectly schema-valid classifier mark every message urgency: "high" because my system prompt described urgency badly. The JSON was flawless. The judgment needed work.

Schema for shape, prompt for meaning, and a deleted cleanModelJson() for morale. Of the three, I enjoy the last one most.