[AI-103 Hands-On] Part 3: How to Get Structured JSON Output

Hi, I’m Lena Nadir.

In Part 1, we built an Agent using the Web Search Tool. In Part 2, we covered RAG with File Search. Part 3 combines the two: search the web for a real job posting while converting the result into JSON using Structured Outputs — a topic also tested in AI-103’s Generative AI/Agent domain, and a pattern you’ll run into constantly in real-world implementations.

Converting unstructured text found via a web search into JSON via an LLM is standard practice in plenty of real implementations, but there are broadly two ways to do it.

1. Pattern A: Instructions only — tell the model in natural language, “output in this JSON format”
2. Pattern B: Instructions + JSON Schema — enforce the structure itself with a strict: true JSON Schema

Everyone “knows” JSON Schema is the safer choice, but how much does it actually matter in practice? Let’s measure it instead of guessing.

What We’re Testing

We run the same search-and-extract task multiple times under each of the two approaches above, and compare:

• Parse success rate: whether the output parses cleanly with json.loads
• Exact field-match rate: whether the expected fields are all present, with nothing missing or extra
• Type-valid rate: whether not just the field names, but the value types and vocabulary, are correct
• Output-shape variance: how many distinct key structures show up across runs, even under the same instructions
• Effective token cost including retries: the “cost per successful record,” counting any extra tokens spent on retries after a parse failure

That last point — effective cost including retries — is one of the two things at the heart of this post. There’s a common misconception that “using JSON Schema lowers your per-token price.” That’s not accurate: the price per token doesn’t change. What actually happens is that fewer parse-failure retries, and more concise output, tend to lower your real-world operating cost.

The Task

We search the web for a real job posting (a senior backend engineer role) and extract a fixed set of fields from it. As you’ll see below, the search and the extraction both happen inside a single call.

Fields to extract:

EXPECTED_FIELDS = {
    "role",
    "location",
    "primary_languages",
    "dev_environment",
    "required_skills",
    "salary_min",
    "work_style",
}

Creating Two Agents

Pattern A and Pattern B each get their own dedicated Foundry Agent, created via project.agents.create_version(...) and called through agent_reference. This is the same Instructions you’d otherwise configure by hand in Foundry Portal’s Playground back in Parts 1 and 2 — here we set it from code instead.

📝 Note: agent_reference() takes the created agent_a / agent_b objects directly, rather than a hardcoded name string like AGENT_NAME_A. If you hardcode the name, the call can end up silently pinned to a stale version once the agent gets recreated and its version number bumps. Reading agent.name and agent.version straight off the object instead guarantees each call targets the exact version this run just created.

Both agents share the exact same model, Instructions, and Tools (Web Search). The only difference is whether a JSON Schema is configured. We deliberately give the agent only the field names — no types (like integer or array of strings), and no enum values for work_style. If we spelled out type hints in the Instructions too, we’d effectively be embedding the same information JSON Schema provides, which would make the comparison unfair.

from azure.ai.projects.models import PromptAgentDefinition, WebSearchTool

AGENT_NAME_A = "job-posting-agent-instructions-only"
AGENT_NAME_B = "job-posting-agent-json-schema"

# NOTE: type/enum hints are deliberately omitted here (see the text below)
AGENT_INSTRUCTIONS = """You are an assistant that finds job postings and extracts structured data from them.

Given a role and location in the user message, find exactly one currently
open job posting matching those criteria, from a job board or a company's
own careers page.

From that posting, extract the following fields and return them as JSON:
- role
- location
- primary_languages
- dev_environment
- required_skills
- salary_min
- work_style

Return ONLY the JSON object, no other text.
"""

JSON_SCHEMA = {
    "type": "object",
    "properties": {
        "role": {"type": "string"},
        "location": {"type": "string"},
        "primary_languages": {"type": "array", "items": {"type": "string"}},
        "dev_environment": {"type": "array", "items": {"type": "string"}},
        "required_skills": {"type": "array", "items": {"type": "string"}},
        "salary_min": {"type": "integer"},
        "work_style": {"type": "string", "enum": ["remote", "hybrid", "onsite"]},
    },
    "required": [
        "role",
        "location",
        "primary_languages",
        "dev_environment",
        "required_skills",
        "salary_min",
        "work_style",
    ],
    "additionalProperties": False,
}

# For Pattern A: no schema
agent_a = project.agents.create_version(
    agent_name=AGENT_NAME_A,
    definition=PromptAgentDefinition(
        model=model,
        instructions=AGENT_INSTRUCTIONS,
        tools=[WebSearchTool()],
    ),
    description="Finds a job posting on the web and extracts structured fields from it as JSON (no schema).",
)

# For Pattern B: same Instructions/Tools, plus a JSON Schema (strict) via `text`
agent_b = project.agents.create_version(
    agent_name=AGENT_NAME_B,
    definition=PromptAgentDefinition(
        model=model,
        instructions=AGENT_INSTRUCTIONS,
        tools=[WebSearchTool()],
        text={
            "format": {
                "type": "json_schema",
                "name": "job_posting_extraction",
                "strict": True,
                "schema": JSON_SCHEMA,
            }
        },
    ),
    description="Finds a job posting on the web and extracts structured fields from it as JSON (schema-enforced).",
)

Shared Plumbing: agent_reference and the Search Query

agent_reference() and the search query (WEB_SEARCH_QUERY) are used identically by both Pattern A and Pattern B, so let’s define them once up front.

def agent_reference(agent):
    return {
        "agent_reference": {
            "name": agent.name,
            "version": agent.version,
            "type": "agent_reference",
        }
    }


WEB_SEARCH_QUERY_JA = (
    "日本語の求人サイトや企業の採用ページで、東京勤務のシニアバックエンド"
    "エンジニアの求人を探してください。"
)

WEB_SEARCH_QUERY_EN = (
    "Find a senior backend engineer job posting based in an English-speaking "
    "country (e.g. the US, UK, Canada, or Australia), from an English-language "
    "job board or a company's own careers page."
)

# Switch to "ja" to target a Japanese-language posting instead.
SEARCH_LANGUAGE = "en"  # "ja" | "en"
WEB_SEARCH_QUERY = WEB_SEARCH_QUERY_EN if SEARCH_LANGUAGE == "en" else WEB_SEARCH_QUERY_JA

WEB_SEARCH_QUERY is the one thing specific to a given call — “which posting to search for” — and gets passed as input for both Pattern A and Pattern B alike. The search brief and JSON-shaping rules already live in each agent’s Instructions, so the caller doesn’t need to repeat them.

📝 Note: The measured results later in this article were captured with SEARCH_LANGUAGE = "ja" (targeting Japanese-language postings), so the numbers match the Japanese edition of this article one-to-one. Switching to "en" does work, but expect different characteristics, not just a language change: English-language postings tend to omit salary figures far more often and carry less structured detail overall, which pushes both patterns’ parse success and retry counts in a worse direction — a good reminder that this kind of benchmark is sensitive to the underlying data source, not just the prompting method.

Pattern A: WebSearchTool + Instructions Only (One Call)

Calling agent_a (no schema) once is enough — search and answer generation both happen in that single call.

response = openai_client.responses.create(
    input=WEB_SEARCH_QUERY,
    extra_body=agent_reference(agent_a),
)

extra_body=agent_reference(agent_a) is how you pass agent_reference — an Azure-specific extension parameter — into responses.create(). It routes the request to the Agent named/versioned in agent_a, rather than to a raw model.

To more closely mirror real-world usage, the benchmark code also retries up to twice if parsing fails. Since a retry means starting the search over, one retry’s cost includes another full Web Search round — so we add these retry tokens into the “effective cost” tally later on.

Pattern B: WebSearchTool + Instructions + JSON Schema (One Call)

The call looks exactly like Pattern A’s — just swap in agent_b. No need to pass text at call time; it’s already configured on the agent.

response = openai_client.responses.create(
    input=WEB_SEARCH_QUERY,
    extra_body=agent_reference(agent_b),
)

agent_b’s schema explicitly sets additionalProperties: false and required, which strictly constrains the shape the model is allowed to output. No retry logic is needed for parse failures, either — as long as the model actually completes a normal response.

⚠️ Caveat: JSON Schema (strict) only guarantees the output shape for a normal completion. If the model’s safety/content-policy layer decides to refuse instead — for example, returning "I'm sorry, but I cannot assist with that request." instead of JSON — that refusal is plain free-form text and bypasses the schema entirely, so try_parse_json() still fails on it. Testing with SEARCH_LANGUAGE = "en" surfaced exactly this: certain job postings pulled from the web apparently trip a refusal, and Pattern B’s parse success rate dropped below 100% as a result. In other words, “Pattern B never needs retries” only holds as long as no refusal occurs — for production use, you’d want retry/fallback handling on Pattern B too, not just Pattern A. Worth flagging: these runs were executed from a Japan-based Azure environment while searching for postings in the US/UK/Canada/Australia — the Web Search Tool’s results can vary by the resource’s region, so some of the added instability when targeting English-speaking-country postings from Japan may come from that mismatch rather than the language switch alone. This benchmark doesn’t isolate that variable, so treat it as a hypothesis, not a confirmed cause.

Measured Results

Results from running stability_benchmark.py (model: gpt-4.1-mini, 20 runs per pattern, each run searching independently and live).

The benchmark’s internal validation logic also computes a “Type-valid rate” (Pattern A 0% vs. Pattern B 100% — see the raw log below), and on its own that number makes Pattern A look badly broken. But agent_a was never given any type or enum hints in the first place, so of course it won’t match a schema it was never told about — scoring that 0% as “Pattern A’s failure” in isolation isn’t quite a fair read. That’s why the table above instead reports a fairer comparison: how consistently Pattern A expresses the same underlying information, measured as the number of distinct phrasings it produces for the same field. The work_style breakdown below shows exactly what that wording drift looks like in practice.

The type of salary_min’s value came back as int in all 20 runs for both patterns. The only field where a difference showed up was work_style.

Breakdown of work_style output values (Pattern A: WebSearch + Instructions only, 20 runs, top 3 shown)

(Translated from Japanese — the source postings were in Japanese, so the model’s original output was too. These are close, natural-English renderings of each distinct phrasing, not literal word-for-word translations; the point isn’t the exact wording but that the model expressed the same underlying information differently almost every time.)

Every one of these is just the posting’s original wording copied verbatim — 17 distinct phrasings across 20 runs (the full list is in the raw log at the end of this section). Before even asking whether any of these match the enum values (remote / hybrid / onsite), the more basic problem is that the same underlying information comes back worded differently almost every single time.

Breakdown of work_style output values (Pattern B: WebSearch + Instructions + JSON Schema, 20 runs)

Both are valid enum values, so types_valid is True for all 20 runs (which one shows up — hybrid or remote — just reflects the actual posting each run found, not a structural failure).

--- Pattern A: WebSearch + Instructions only (with retry) ---
  Parse success rate               : 100%
  Field-set match rate (keys only) : 90%
  Type-valid rate (keys + types)   : 0%
  Distinct field-set count (lower = more consistent): 2
  work_style value distribution: {'一部リモート可': 4, '月給制、フレックスタイム制、コアタイムなし、育児・介護による時短勤務制度あり、転勤なし': 1, 'フルリモート可, フレックスタイム制度, 副業制度あり': 1, 'フルリモート可、フレックスタイム、服装自由、副業制度あり': 1, 'フルリモート可、フレックスタイム制度、副業制度あり': 1, 'フレックスタイム制、リモートワーク可、コアタイムなし': 1, 'フルリモート可、フレックスタイム制、正社員': 1, 'リモートワーク可、フレックスタイム制度、副業制度あり、私服勤務OK': 1, '一部リモート可、フルフレックス制度あり': 1, '一部リモート可, フルフレックス': 1, '正社員、フレックスタイム制、リモートワーク可、コアタイムなし、転勤なし': 1, '正社員、一部リモート可、フルフレックス勤務可能': 1, 'リモートワーク可（フルリモート可能）、フレックスタイム制度あり、正社員': 1, 'リモートワーク可、フレックスタイム制度、完全週休二日制（土日）、土日祝日休み、有給休暇、年末年始休暇、慶弔休暇': 1, '一部リモート可, 正社員, フルフレックス': 1, 'フレックスタイム制度あり、リモートワーク可（一部リモート可）': 1, 'フレックスタイム制、リモートワーク制度あり、コアタイムなし、週2日程度の出社あり': 1}
  salary_min value type distribution: {'int': 20}
  Total retries needed across all runs: 1
  Avg input tokens per successful record : 21031.3
  Avg output tokens per successful record: 306.1
  Avg total tokens per successful record : 21337.4

--- Pattern B: WebSearch + Instructions + JSON Schema ---
  Parse success rate               : 100%
  Field-set match rate (keys only) : 100%
  Type-valid rate (keys + types)   : 100%
  Distinct field-set count (lower = more consistent): 1
  work_style value distribution: {'hybrid': 19, 'remote': 1}
  salary_min value type distribution: {'int': 20}
  Total retries needed across all runs: 0
  Avg input tokens per successful record : 20572.5
  Avg output tokens per successful record: 225.4
  Avg total tokens per successful record : 20797.9

What We Learned

• Stability: Pattern B’s biggest advantage is that JSON Schema (strict) enforces the output shape itself, so no post-processing or validation is needed after the fact. Pattern A, in contrast, produced 17 distinct phrasings of work_style across 20 runs (wording drift) — output that needs a normalization step before it’s safe to use downstream. Pattern B only ever returns values within the enum (hybrid / remote), so it can be trusted as-is
• Cost: Pattern A triggered one retry out of 20 runs, while Pattern B completed every run in a single call with no retries. Total token counts were close either way — 21337.4 for Pattern A versus 20797.9 for Pattern B (about 2.5% less) — and the per-token price is identical, so cost wasn’t a major differentiator here

Why This Happens

• JSON Schema (strict) masks invalid tokens at generation time, so the model literally cannot generate a value outside an enum’s allowed set or of the wrong type
• With Instructions-only, the model can only infer “which vocabulary to use” from the natural-language description — without type/enum hints, a categorical field like work_style just gets the posting’s raw wording copied straight through
• JSON Schema has to be configured on the agent definition (the text parameter of PromptAgentDefinition), not at call time through agent_reference (there’s no response_format parameter)

Summary

• Structured output with Instructions-only is prone to wording drift like work_style’s 17 phrasings across 20 runs, which means a normalization step is needed before the output is safe to use downstream
• JSON Schema (strict: true) guarantees value types and enums structurally, so no post-processing is needed — but it has to be configured on the agent definition (the text parameter of PromptAgentDefinition), not at call time through agent_reference
• Cost came out about the same for both patterns (token counts were close, and Pattern B needed no retries). For production tasks where JSON structure matters — especially categorical fields like enums — it’s worth pairing Instructions with JSON Schema

Next time (Part 4), we’ll cover setting up Responsible AI guardrails.

Confirmation Quiz (AI-103 Style)

Three questions styled after the actual AI-103 exam format (scenario setup + choose the best option). Check your understanding of this episode’s content.

Q1. When you instruct JSON output using Instructions alone, which of the following problems is most likely to occur?

• A The per-token price increases
• B Parsing may succeed, yet a categorical field's value drifts outside the expected vocabulary (enum)
• C The model becomes unable to call tools
• D Authentication errors become more likely

Q2. What is the main effect of specifying strict: true in text.format.json_schema?

• A Response speed is always faster
• B The model is constrained so it cannot generate tokens that violate the schema during generation
• C It increases the number of models you can use
• D Input token billing is waived

Q3. Which is the most accurate explanation of the claim, “using JSON Schema lowers cost”?

• A JSON Schema lowers the per-token price itself
• B JSON Schema is free to use, so no cost is incurred in the first place
• C The per-token price doesn't change, but fewer parse-failure retries and more concise output tend to lower the real-world operating cost
• D Using JSON Schema always costs more than Instructions-only

Full Code

The code used in this article is published in the GitHub repository under azure/ai-103/episode-03-structured-output/.

👉 tech-hands-on-labs (GitHub)

💡 Want to Check Your Readiness Before Exam Day?

Once you’ve worked through the hands-on material, if you want to check whether your understanding is actually at exam level, I’ve published an AI-103 practice test collection on Udemy.

• 👉 AI-103 Practice Tests 2026 | Python, Foundry & AI Agents

🎟 Launch coupon: RELEASE-CAMPAIGN (33% off, through 2026/09/24) → Apply the coupon here

© 2026 Lena Nadir. All rights reserved.