Repository navigation
Conversation
Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com> (cherry picked from commit 2f413f0)
Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com> (cherry picked from commit 1d18295)
…aphs_completed_to_end (ai-dynamo#1238) Signed-off-by: Elias Bermudez <dbermudez@nvidia.com> (cherry picked from commit f8c8e36)
Signed-off-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com> Signed-off-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 093038a)
…ai-dynamo#1239) Signed-off-by: Alex <jihui.huang@daocloud.io> (cherry picked from commit aaaa72e)
…i-dynamo#1192) (cherry picked from commit 9be59a9)
…bing (ai-dynamo#1000) (cherry picked from commit 59cd43e)
…ntent serialization (ai-dynamo#1259) Signed-off-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 37467dc)
Signed-off-by: lkomali <lkomali@nvidia.com> (cherry picked from commit 00ba1c5)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> (cherry picked from commit 6480e54)
Signed-off-by: Elias Bermudez <dbermudez@nvidia.com> (cherry picked from commit 23ed221)
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com> Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com> Co-authored-by: Harshini Komali <157742537+lkomali@users.noreply.github.com> (cherry picked from commit 1e32a51)
ai-dynamo#1269) Signed-off-by: lkomali <lkomali@nvidia.com> (cherry picked from commit 215be05)
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com> (cherry picked from commit 516faa1)
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com> (cherry picked from commit ce45358)
Signed-off-by: Elias Bermudez <dbermudez@nvidia.com> (cherry picked from commit 082a518)
ai-dynamo#1185) Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 21f8ad7)
Signed-off-by: Anant Sharma <anants@nvidia.com> (cherry picked from commit e659d2a)
Signed-off-by: Anant Sharma <anants@nvidia.com> (cherry picked from commit 9e96b49)
… unit (ai-dynamo#1240) Signed-off-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com> (cherry picked from commit 260d00f)
…undled first streamed chunk (ai-dynamo#1279) Signed-off-by: lkomali <lkomali@nvidia.com> (cherry picked from commit 324bb05)
…format (ai-dynamo#1282) (cherry picked from commit 810fd8b)
…i-dynamo#1314) Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com> (cherry picked from commit e5ebe91)
…ed (ai-dynamo#1287) Signed-off-by: Shimi Bandiel <shimib@google.com> (cherry picked from commit bfe3315)
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com> (cherry picked from commit dd3f09b)
… warmup prefix (ai-dynamo#1256) Signed-off-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 1b34de0)
…i-dynamo#1400) Signed-off-by: Navjot Singh <navjot.singh@shopify.com> Signed-off-by: navjot10 <navjot.singh@shopify.com> (cherry picked from commit ab7c8ee)
The config-envelope cases build specs through aiperf.kubernetes, and the warmup-per-lane flag is routed by the fork's agentic replay converter.
Reverts the pick of upstream ea28b2e; the fork does not track uv.lock.
Reverts the pick of upstream 0968f7e; the fork does not track uv.lock, so 'uv sync --locked' would fail its unit-test workflow.
Reverts the pick of upstream 56945bb; on the fork's default branch it would start closing the fork's own PRs and issues.
Upstream ai-dynamo#1277 raised the floor to 3.14.3; aiohttp is the client that times streamed chunks, so 1.0.x keeps 1.0.9's 3.13 line.
# Conflicts: # src/aiperf/timing/conversation_source.py
…i-dynamo#1560) Signed-off-by: heihei-push <17313890385@vip.163.com> Signed-off-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com> Co-authored-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit b0276e1)
…ynamo#1584) Signed-off-by: Apoorva Garg <271577+agarg21@users.noreply.github.com> Co-authored-by: Apoorva Garg <271577+agarg21@users.noreply.github.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit b1746a8)
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com> Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit be66ff3)
…e_turn datasets (ai-dynamo#1481) Signed-off-by: meka-giridhar_hpeprod <meka.giridhar@hpe.com> Signed-off-by: Maze999 <162086986+Maze999@users.noreply.github.com> Co-authored-by: meka-giridhar_hpeprod <meka.giridhar@hpe.com> Co-authored-by: Harshini Komali <157742537+lkomali@users.noreply.github.com> (cherry picked from commit 8bf9af3)
…-dynamo#1473) Signed-off-by: manunicholasjacob <manunicholasjacob@gmail.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit ed548a4)
…oning items (ai-dynamo#1478) Signed-off-by: Srividhya R <195692899+ra-srivid@users.noreply.github.com> Co-authored-by: Srividhya R <195692899+ra-srivid@users.noreply.github.com> (cherry picked from commit 0c4d9db)
Signed-off-by: Dmitry Voropaev <dy.voropaev@gmail.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit b227ebc)
…ai-dynamo#1585) Signed-off-by: SongMingzhe <65708305+SongMingzhe@users.noreply.github.com> Co-authored-by: SongMingzhe <65708305+SongMingzhe@users.noreply.github.com> Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 98d8e53)
…rker (ai-dynamo#1623) Signed-off-by: lkomali <lkomali@nvidia.com> (cherry picked from commit 6e096ab)
ai-dynamo#1450) Co-authored-by: Frank <3429989+FrankD412@users.noreply.github.com> (cherry picked from commit 2602253)
…ice snapshot Root cause: export_results runs in a worker thread while ingestion can continue. _build_slice_ragged_index indexed record_bin, sized at the snapshot, with every current record index, so a ragged value appended for a later record raised IndexError and aborted the export. Fix: map record indices beyond the snapshot to no slice, as get_values_for_mask does, keeping value positions unchanged. Drop the redundant _compute_timeslice_bins docstring. Validation: new test appends ragged values for a record past the snapshot while the index is built; the timeslices match the export without them. Signed-off-by: weireweire <20922698+weireweire@users.noreply.github.com> (cherry picked from commit 0b166c3)
…ream Root cause: #49 fixed the agentic warmup-start scrape in two places: it set phase_kind="warmup" on the synthesized agentic warmup, and it added a PhaseBaselineRequestMessage.phase fallback for boundaries without a kind. Upstream aiperf (ai-dynamo#1325) fixes it with phase_kind="warmup" alone. The fallback cannot trigger: validated PhaseConfigs always carry a kind, and the agentic warmup now sets one. Fix: keep phase_kind="warmup" and its test assertion; drop the phase fallback, the message field, the publisher wiring, their tests, and the added comments, so this code matches upstream. Validation: tests/unit/server_metrics and tests/unit/timing. Signed-off-by: weireweire <20922698+weireweire@users.noreply.github.com> (cherry picked from commit 70cfe62)
…d of shared state Root cause: the slice index and the active slice were stored on the accumulator while the export's slice loop ran. export_results runs in a worker thread, and the realtime summarize runs on the event loop with no lock, so a summary that overlapped the slice loop gathered ragged values from the export's current slice instead of its own mask. Fix: build the index as a local in _compute_timeslices and pass each slice's value positions to _compute_results as an argument; the accumulator holds no per-computation state. Also trim the index: int32 slice ids and value positions, and slice bounds from bincount + cumsum instead of a sorted copy and two searchsorted passes. Validation: new test runs a phase-scoped summary in the middle of the slice loop; both the export and the summary match their standalone results. Existing index tests still match the per-slice mask path. Signed-off-by: weireweire <20922698+weireweire@users.noreply.github.com> (cherry picked from commit 8847786)
|
This PR is too large for Bugbot to review. It changes 56,515 lines and 4,000,234 characters. Split the change into smaller pull requests to get a review. |
Try out this PRQuick install: pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@d4b1563e3a1483aa58905551fc06a707a25b311aRecommended with virtual environment (using uv): uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@d4b1563e3a1483aa58905551fc06a707a25b311aLast updated for commit: |
|
Fern Docs Preview: generation failed — see the Actions log for details. This does not block merge; ask a maintainer to retry if needed. |
Allow the "unparseable" spelling in codespell, as upstream does, and apply ruff format to the inter-token latency test.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings upstream aiperf fixes into the 1.0 line without changing benchmark behavior, so 1.0.x results stay comparable with earlier 1.0 releases.
Cherry-Picked
!=the negation of==on case-insensitive enums--config(without the control-hook routing 1.0 lacks)reasoning_textfrom non-streaming Responses API replies--input-file--search-spacekeywordsaudio_duration_secondsfor single_turn datasetsprevious_response_id--system-promptand--system-prompt-file--per-chunk-usage--export-level rawimplies outputs.jsonsystem/agentxpresetagentxpresetLeft out because they change benchmark behavior