Skip to content

fix: stop overwriting multi-message INPUT_VALUE in OpenAI instrumentor#184

Open
nigamgauri wants to merge 2 commits into
future-agi:mainfrom
nigamgauri:fix/openai-multimessage-input-overwrite
Open

fix: stop overwriting multi-message INPUT_VALUE in OpenAI instrumentor#184
nigamgauri wants to merge 2 commits into
future-agi:mainfrom
nigamgauri:fix/openai-multimessage-input-overwrite

Conversation

@nigamgauri

Copy link
Copy Markdown

What does this PR do?

Fixes a bug in the OpenAI instrumentor where multi-turn conversations lost every message after the first one in the trace.

Why?

_process_input_data in _span_io_handler.py set INPUT_VALUE on the span three times. The first write was correct, a JSON array of every message in the conversation. The next two writes overwrote it: first with a joined string of all message text, then with just eval_input[0], the text of the first message only. Any conversation with more than one message lost everything past turn one in the trace. There's no EVAL_INPUT attribute defined in fi_types.py, so the eval_input accumulation had no other consumer, it was just clobbering good data. Fixes #151.

How was it tested?

  • Unit tests added (pytest)
  • Integration tests pass (N/A, no gateway behavior changed)
  • ruff check passes on the files I changed
  • mypy was not run cleanly on this branch (pre-existing errors in unrelated files, unrelated to this fix)
  • Public types, env vars, or SDK behavior documented in README

Added tests/test_span_io_handler.py with two cases: a plain multi-turn conversation, and a multimodal message list with text and an image. Both fail on main (the multi-turn case raises a JSONDecodeError since INPUT_VALUE ends up as raw text instead of JSON) and pass with the fix.

Checklist

  • Branch is off main
  • Commit messages follow Conventional Commits
  • No TODOs or commented-out code left in
  • No real API keys or secrets in the diff
  • N/A, no prose changes

Notes for reviewers

I didn't touch the multimodal branch's image handling, only removed the three-way overwrite on INPUT_VALUE. Happy to add a changelog entry if the project wants one for this package.

@Phoenix1454

Copy link
Copy Markdown

Comparing this with #152 they fix the same bug but land in very different places.

#152 removes the JSON write and keeps eval_input joined as a flat string. This PR does the opposite drops eval_input entirely and keeps the structured JSON. For an observability tool the JSON is clearly right: role metadata survives, the trace can actually be replayed or filtered by role downstream. The flat string in #152 throws that away permanently.

One edge case I don't see tested: a message where content is a list containing only image_url items with no text. That hits continue at line 41 and the message is silently absent from input_content so INPUT_VALUE gets set but that turn just isn't in it. The URL does end up in INPUT_IMAGES but there's no indication in the span that a message was dropped. Worth either documenting that this is intentional, or adding the message dict (without the image bytes) to input_content as a placeholder so the conversation structure stays intact.

Otherwise the fix is solid and the InMemorySpanExporter tests are the right way to catch this class of bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

INPUT_VALUE overwritten in _process_input_data leading to loss of multi-message context

2 participants