Avoid counting typed image payloads as context text - #46
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A valid 6.31 MB PNG was counted as roughly 2.1 million text tokens, so protected current input could fail before any provider call. Estimate textual content without counting typed image transport (base64/URLs), mark the image portion as unmeasured, and use the same estimate for compaction deltas. Original messages and images are unchanged; quoted data and tool arguments still count as text. Native provider counts and real text-budget failures remain authoritative.
Class review refreshed microsoft/amplifier docs/MODULES.md at 33b75749 and checked known contexts, loops, providers, and bundled contexts. context-persistent inherits this fix; loop-streaming and bundled context-managed need companion patches. context-handoff remains unavailable (repository not found). Existing OpenAI PR103 already handles provider-layer typed-media budgeting.
Validation: 227 context-simple tests passed (1 expected failure); 21 context-persistent tests passed; 449 loop-streaming tests passed with both patches, including streaming/nonstreaming, estimate/actual meters, absent/unavailable/exact/oversized/failed counters. The real Unified/loop-live fixture now dispatches the original PNG exactly once and completes; zero provider network calls. Live ChatGPT/Anthropic/OpenAI acceptance remains pending. New files pass Ruff; main-file lint findings are unchanged from the base. No new runtime dependency.
Companion draft PRs: #46, microsoft/amplifier-module-loop-streaming#64, microsoft/amplifier-bundle-context-managed#11.
Text estimates are partial, not a complete-request fit guarantee. Provider-native counts, request-size validation, and actual provider errors retain authority.