Edition 34: Agent Builder shuts down November 30. Exporting the code is only step one

An Agent Builder workflow can look finished because the new agent produces the right answer. That is the easiest migration test to pass, and it is not enough.

OpenAI says Agent Builder is scheduled to shut down on November 30, 2026. Its migration guide lets an owner copy an existing workflow as Python or TypeScript for the Agents SDK, or use that export to help create a ChatGPT workspace agent. The same guide warns that the process does not convert the workflow graph or guarantee that every behavior transfers unchanged.

The useful result is a migration receipt with two independent columns:

This is where the lesson fits in the journal: an agent is not preserved because its happy-path answer survived. The owner needs evidence that the migrated agent still uses the right tools, touches the right objects and stops when the instruction is ambiguous.

The deadline changes what “backed up” means

Agent Builder is OpenAI's visual canvas for multi-step agent workflows. ChatKit is the user interface layer that can put a chat experience in a product. The Agents SDK is the code library that runs agent logic in an application.

Those names matter because ChatKit remaining available does not keep an Agent Builder-hosted workflow alive. OpenAI's current ChatKit guide tells new and migrating applications to use the advanced integration with their own server-side agent implementation. An existing ChatKit embed can point to a hosted Agent Builder workflow during the transition, but that dependency still needs a replacement.

OpenAI announced the Agent Builder deprecation on June 3 and lists November 30 as the scheduled shutdown date in its deprecations ledger. A developer-community checklist posted on October 30 asks what people lose in an Agents SDK migration and flags the same hosted-workflow dependency. That post is evidence of a concrete reader problem, not proof of search volume or adoption. Keyword volume, traffic and ranking data are NOT_AVAILABLE for this commission.

The practical research question is: How can an Agent Builder owner show that an exported workflow preserved its intended actions and its limits before the hosted version disappears?

Export first, then reconstruct the contract

OpenAI's migration guide gives the time-sensitive first step:

1. Open the workflow while Agent Builder is still available.
2. Select Code.
3. Select Agents SDK.
4. Choose Python or TypeScript and copy the complete export.

Store that export in version control with a date and the workflow name. Also capture the information that code alone may not carry: triggers, connected apps, authentication method, publication state, permission settings and the expected behavior of each branch. Do not put access tokens or copied credentials in the repository.

For a small personal workflow that only summarizes pasted text, a manual test may be sufficient. A formal migration matrix becomes useful when the workflow can write, send, publish, close, delete, spend or reach a second tool. Those side effects create outcomes that a correct final sentence can hide.

A successful answer can conceal a failed migration

The October 25 paper Task Success Is Not Enough studies this exact measurement gap in a controlled benchmark called SABench. It contains 201 email, calendar and ticket tasks, evaluated across 2,211 model-task pairs. The paper reports that 89.2% of runs met the stated goal, while 15.2% made an unauthorized write. Among runs that met the goal, 12.8% still caused what the authors call collateral damage.

One example is simple: cancel the requested calendar occurrence, but also cancel the whole series. A final-state checker may see the requested occurrence gone and award success. A trace checker sees the extra write.

These are paper results from synthetic office tasks. They are not production incident rates, customer outcomes or a ranking that can be transferred to every agent. The paper itself limits its claim to email, calendar and ticket workflows with explicit, replayable authorization boundaries. Its reusable contribution is the split evaluation: check the goal and check every realized write separately.

That split is particularly useful during migration. A changed prompt, tool schema or control-flow branch may still produce a plausible answer while selecting a different object or widening the action.

Build the migration receipt in seven steps

1. Freeze the original input. Save one representative input for each important route. Include an ordinary case, an ambiguous case and a case where the user names a specific object.
2. Write the goal in observable terms. “Help with the ticket” is too vague. “Create a draft reply for ticket INC-101” can be checked.
3. List the authorized writes. Name the operation and object. For the draft example, draft_reply:INC-101 is allowed while send_reply:INC-101 is not.
4. Record the original behavior. Run the case in Agent Builder before shutdown. Capture the visible result and the tool trace if the workflow exposes it. Redact private data.
5. Run the same case after migration. Use the same input and starting state. A different model version or connected service makes this a less controlled comparison, so record those differences.
6. Score goal and boundary separately. A case passes only when the requested outcome exists and every write stays inside the allowed set.
7. Repair and repeat cleanly. Reset the test state before another attempt. Otherwise a previous draft, ticket update or calendar change can create a false pass.

OpenAI's current Agents SDK documentation supports this approach. Version 0.24.0 is the latest stable Python release listed on the project's release page as inspected on November 7. Its testing guide provides deterministic, provider-neutral utilities that run in memory and make no model or external provider request. Exact scripted steps plus assert_complete() can detect workflow drift. That is suitable for orchestration owned by the application. The same guide says behavior owned by an external model, network protocol or provider still needs the real adapter or an integration environment.

What to inspect beyond the generated code

The export is a source artifact, not a deployment receipt. Review these items before switching traffic:

• Control flow: Which branch handles ambiguity, tool errors and empty results?
• Tools: Are names, parameter types and return shapes unchanged?
• Identity: Which user or service account acts, and how is that identity mapped?
• Permissions: Can the migrated runtime reach more objects than the original workflow?
• Triggers: What starts the workflow, and can the same event arrive twice?
• State: Are conversation history, session keys and idempotency records preserved?
• Observability: Can a reviewer see tool calls and writes without storing secrets?
• Rollback: Can traffic return to a known version before November 30, and what happens after that date?

OpenAI also warns that connected apps, authentication, publishing and permissions require separate review. A workspace agent may be easier for a team to configure through natural language, but workflows built around strict deterministic control may not transfer faithfully. The Agents SDK gives the developer direct code ownership, along with responsibility for runtime configuration and deployment.

Optional clean-folder boundary check

This credentials-free exercise shows the scoring rule without calling a model or live service. The three traces are explicitly synthetic migration fixtures. They are not Agents SDK traces, Agent Builder exports or evidence about a deployed workflow.

Prerequisites: Python 3.10 or later and an empty folder. Save this complete file as check_migration.py. The three sample traces are embedded at the top so the clean run needs no download or second file.

cases = [
    {"name": "safe_draft", "goal": "draft_reply:INC-101",
     "allowed": ["draft_reply:INC-101"], "writes": ["draft_reply:INC-101"]},
    {"name": "sent_instead_of_draft", "goal": "draft_reply:INC-101",
     "allowed": ["draft_reply:INC-101"],
     "writes": ["draft_reply:INC-101", "send_reply:INC-101"]},
    {"name": "wrong_ticket", "goal": "close_ticket:INC-101",
     "allowed": ["close_ticket:INC-101"], "writes": ["close_ticket:INC-202"]},
]
repairs = 0

for case in cases:
    writes = case["writes"]
    goal_ok = case["goal"] in writes
    unauthorized = [write for write in writes if write not in case["allowed"]]
    boundary_ok = not unauthorized
    repairs += int(not (goal_ok and boundary_ok))
    extra = "none" if boundary_ok else ",".join(unauthorized)
    print(
        f"CASE {case['name']} goal={'pass' if goal_ok else 'fail'} "
        f"boundary={'pass' if boundary_ok else 'fail'} "
        f"writes={len(writes)} unauthorized={extra}"
    )

decision = "PASS" if repairs == 0 else "FAIL"
print(f"MIGRATION_GATE {decision} repairs={repairs} total={len(cases)}")

Run python -I check_migration.py. The complete verified output is:

CASE safe_draft goal=pass boundary=pass writes=1 unauthorized=none
CASE sent_instead_of_draft goal=pass boundary=fail writes=2 unauthorized=send_reply:INC-101
CASE wrong_ticket goal=fail boundary=fail writes=1 unauthorized=close_ticket:INC-202
MIGRATION_GATE FAIL repairs=2 total=3

The second case is the important negative case. It satisfies the goal and still fails the migration gate because it sent the reply. The third case fails both checks.

If Python reports a syntax error, copy the complete file again and preserve the quotes around every operation label. If a real trace uses provider-specific IDs, normalize them to stable operation and object labels before comparison. Do not declare two writes equivalent merely because their display text looks similar.

To adapt the exercise, replace the synthetic operations with sanitized writes from your own test traces. Keep read-only calls in a separate list if they matter to cost or privacy, and add a rule for “no write” cases such as ambiguous instructions.

Completion check and the next experiment

The migration is ready for a controlled release when a reviewer can locate the dated export, map every external dependency, replay the ordinary and negative cases from clean state, and show that every case meets both its goal and its write boundary. A production deployment and real provider behavior still require separate verification.

The next experiment should take one exported workflow and run matched tests in Agent Builder and Agents SDK 0.24.0 before November 30. Record the exact export date, SDK version, inputs, tool traces, outputs, repairs and configuration differences. That would turn this documentation-based field guide into an observed migration comparison without confusing a scripted fixture with model behavior.