DEVOURED
Building Self-Repairing Agent Loops
OpenAI demonstrates agent loops that automatically fix stale code examples through structured review, repair, and validation cycles until tests pass.
What: A cookbook tutorial showing a three-phase agent workflow for repairing outdated Jupyter notebooks: review (inspect and return structured findings), repair (apply focused edits using findings and validation feedback), validate (run checks and report remaining issues). Uses OpenAI's Codex CLI in headless mode with JSON schemas for each phase. Three example notebooks iterate 1-3 times with models gpt-5.4-mini and gpt-5.5 until validation passes. Includes business rules defining 'good' (modern API patterns like client.responses.create, qdrant.query_points, current Evals workflows).
Why it matters: This pattern makes agent outputs reliable by closing the loop: validation failures become structured input for the next repair iteration rather than a dead end. The notebook repair is the example, but the architecture applies wherever output can be measured with trustworthy feedback (unit tests, schema validation, policy checks, simulations, human approval).
Takeaway: The full working notebook with companion data is at developers.openai.com/cookbook/examples/codex/build_iterative_repair_loops_with_codex. The three-phase pattern (structured review → focused repair → measured validation → repeat) is reusable for any domain where you can define correctness and measure progress toward it.
Deep dive
- Three-phase loop architecture: review inspects without editing and returns structured findings; repair applies focused changes using findings plus latest validation feedback; validate runs checks and reports what still needs work
- Uses Codex CLI (@openai/codex npm package version 0.130.0) in headless mode so repair steps run from Python cells instead of interactive chat UI
- Structured outputs with JSON schemas at each phase: findings schema (artifact, issue_type, severity, description, suggested_fix_direction), fix schema (changes_made, unresolved_items, updated_artifact_path), validation schema (overall_passed, cases array, remaining_delta)
- Business rules define what good looks like: current API patterns (client.responses.create not chat.completions.create, modern tools schema not legacy function-calling, qdrant.query_points not qdrant.search, current Evals API not oaieval CLI), runnable local setup, preserved teaching goals
- Three example notebooks with increasing repair depth: one-pass (modernize Qdrant query path), two-pass (modernize Evals flow then remove result-log brittleness using validation feedback), three-pass (modernize model/API shape, tighten runnable setup, restore full retrieval teaching flow)
- Validation executes each repaired notebook end-to-end; failures become structured feedback for next iteration rather than terminal errors
- Models: defaults to gpt-5.4-mini for speed (configurable via REPAIR_MODEL env var), uses gpt-5.5 for COOKBOOK_CHAT_MODEL, supports REPAIR_REASONING_EFFORT setting
- Separation of concerns keeps each phase focused: review doesn't edit files, repair doesn't validate, validate doesn't prescribe fixes
- Working Python code loads sample notebooks, extracts metadata (target_iteration, repair_depth), runs review/repair/validate cycle tracking iterations until validation passes or limit reached
- Setup: npm install -g @openai/codex, set OPENAI_API_KEY, download companion data/docs folder with pre-repair sample notebooks
- Pattern is domain-agnostic: substitute notebook execution with your validation method (unit test suite, schema validator, policy checker, simulation harness, approval workflow)
- Structured handoffs between phases make the loop debuggable, rerunnable, and adaptable to other artifact types beyond notebooks
Decoder
- Codex CLI: OpenAI's agent orchestration tool (npm package @openai/codex), distinct from the deprecated Codex code-generation model. Runs agent workflows programmatically with structured JSON schemas for input/output rather than free-form chat, enabling headless automation.
Original article
Full article content is not available for inline reading.
Explore all 50 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi