Skip to content

OpenAI Responses API Instrumentation - #1813

Open
TimPansino wants to merge 5 commits into
mainfrom
feat-openai-responses-apis
Open

TimPansino wants to merge 5 commits into
mainfrom
feat-openai-responses-apis

Conversation

@TimPansino

@TimPansino TimPansino commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Overview

  • Instrument the newer openai.responses APIs.
  • Add a new set of tests that mirrors the original chat_completions tests.

@TimPansino
TimPansino requested a review from a team as a code owner August 6, 2026 17:27
@github-actions

github-actions Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

✅MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Max errors Warnings Elapsed time
✅ ACTION actionlint 11 0 0 1.05s
✅ MARKDOWN markdownlint 8 0 0 0 1.2s
✅ PYTHON ruff 1141 0 0 0 1.15s
✅ PYTHON ruff-format 1141 0 0 0 0.26s
✅ YAML prettier 26 0 0 0 1.23s
✅ YAML v8r 26 0 0 6.12s
✅ YAML yamllint 26 0 0 0.72s

See detailed reports in MegaLinter artifacts

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 6, 2026
@codecov-commenter

codecov-commenter commented Aug 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.93103% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.70%. Comparing base (e3e51a1) to head (740fa7d).

Files with missing lines Patch % Lines
newrelic/hooks/mlmodel_openai.py 87.82% 2 Missing and 12 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1813      +/-   ##
==========================================
- Coverage   81.75%   81.70%   -0.05%     
==========================================
  Files         223      223              
  Lines       28013    28116     +103     
  Branches     4433     4456      +23     
==========================================
+ Hits        22901    22972      +71     
- Misses       3560     3575      +15     
- Partials     1552     1569      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TimPansino
TimPansino force-pushed the feat-openai-responses-apis branch from 5f17359 to 12dfc71 Compare August 7, 2026 21:22
@mergify mergify Bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Aug 7, 2026
@mergify mergify Bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Aug 12, 2026
if isinstance(item, dict) and item.get("role") is not None:
content = item.get("content")
# Content is a string, or a list of typed parts each carrying `text`.
if isinstance(content, (list, tuple)):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For chat completions, we deliberately suppress content-less tool call messages (on line 595 by checking "tool_call" in finish_reason). Since we don't want to report every tool call as an LlmChatCompletionMessage with content: "", do we want to mirror that suppression behavior here if possible?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is that why "role" is being checked for explicitly?

)
stream_proxy._nr_openai_attrs["content"] = stream_proxy._nr_openai_attrs.get("content", "") + delta
stream_proxy._nr_openai_attrs["role"] = stream_proxy._nr_openai_attrs.get("role") or "assistant"
elif event_type in ("response.completed", "response.incomplete", "response.failed"):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we capturing errors surfaced via response.failed somewhere? From my scan of the codebase, it seems like APIErrors are raised when the raw payload has a top-level error key. response.failed events nest errors inside response.error and the stream just ends normally afterwards. I just want to make sure we are capturing errors in this case also.

@mergify mergify Bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Aug 17, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 26, 2026
@mergify mergify Bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Sep 14, 2026
@mergify mergify Bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Sep 21, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Sep 30, 2026
@mergify mergify Bot removed the tests-failing Tests failing in CI. label Oct 7, 2026
@TimPansino
TimPansino force-pushed the feat-openai-responses-apis branch from 64543e9 to 740fa7d Compare October 9, 2026 19:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants