Skip to content

paper_flow() fails on its own README example: dict-typed nodes field breaks strict structured output, UUID id fields reject the model's natural output #91

Description

@isorobo

Summary

The single-paper example in the README does not run against the currently resolved dependency versions (openai-agents>=0.14, no ceiling, resolves to 0.18.1):

import asyncio
from quantmind.configs import PaperFlowCfg
from quantmind.configs.paper import ArxivIdentifier
from quantmind.flows import paper_flow

async def main() -> None:
    paper = await paper_flow(
        ArxivIdentifier(id="2401.12345"),
        cfg=PaperFlowCfg(model="gpt-4o-mini"),
    )

asyncio.run(main())

Two separate bugs stack on top of each other.

Bug 1 — Agent(output_type=Paper) raises before any LLM call

TreeKnowledge.nodes is dict[UUID, TreeNode] (quantmind/knowledge/_tree.py). OpenAI's strict-mode structured output — the default in current openai-agents releases — cannot represent dict-typed fields at all (arbitrary keys are incompatible with a fixed JSON Schema). paper_flow (quantmind/flows/paper.py) passes the bare Paper type straight into Agent(output_type=out_type), with no strict_json_schema=False escape hatch, so the run fails immediately:

agents.exceptions.UserError: Strict JSON schema is enabled, but the output type is not valid.
Either make the output type strict, or wrap your type with AgentOutputSchema(YourType, strict_json_schema=False)

Bug 2 — once strict mode is off, UUID-typed id fields reject the model's output

Wrapping the output type in AgentOutputSchema(Paper, strict_json_schema=False) gets past bug 1, but a non-strict schema drops the UUID format constraint. The extraction agent then does exactly what you'd expect an LLM to do when free-form: it fills TreeNode.node_id / parent_id / children_ids with readable slugs ("root", "introduction", "methodology") and BaseKnowledge.id with the paper's arXiv id ("2404.11584"), rather than leaving these at their UUID4 defaults. Validation then fails with ~80+ uuid_parsing errors.

Repro

Ran the unmodified README example against a real paper (arXiv 2404.11584, "The landscape of emerging AI agent architectures for reasoning, planning, and tool calling: A survey", model gpt-4o-mini) on the current main (8e21888). Fails both ways described above. Full traceback available on request.

Fix

Opened # — changes TreeNode.node_id/parent_id/children_ids, TreeKnowledge.root_node_id/nodes, BaseKnowledge.id, and PaperKnowledgeCard.paper_id from UUID to str (accepts both UUIDs and readable slugs), and has paper_flow pass AgentOutputSchema(out_type, strict_json_schema=False) instead of a bare type. Verified: the unmodified README example now completes end-to-end with zero validation errors against the same real paper, and the full test suite passes (232/233 — the one failure is pre-existing and unrelated, a Windows path-separator assertion in a local-file test).

Environment

  • openai-agents==0.18.1 (resolved from the unpinned >=0.14 floor)
  • Windows 11, Python 3.12.12

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: flowsPublic operation implementations under quantmind/flows/staletype: bugExisting behavior violates its contract or documented expectation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions