Skip to content

perf(text): consume classified output with a buffer cursor - #184

Open
Afloat16 wants to merge 1 commit into
Edge0-AI:mainfrom
Afloat16:afloat/20261111-next-linear-output-parser
Open

Afloat16 wants to merge 1 commit into
Edge0-AI:mainfrom
Afloat16:afloat/20261111-next-linear-output-parser

Conversation

@Afloat16

Copy link
Copy Markdown

What & why

Classified output erased its buffer front for each character, while trimming created a suffix string per name character. Large chunks and long names repeatedly copied content.

Track a consumed offset, compact before the next feed and scan/trim with string views. Incomplete-call output includes only unconsumed bytes.

Area

Engine, CLI or server (edge0-infer/).

How was it tested?

moe_test_edge0_35b_output

The target and existing split-invariance/streaming checks pass. A 65536-character function-name case preserves the name and completed empty arguments; allocations of at least 32 KiB fall from 32780 on main to 9. The baseline fails this bound. This is parser allocation evidence, not whole-model throughput.

All 14 L0 checks passed in a local integration with the other focused fixes. git diff --check passed.

Local C++ environment: Linux x86_64, GCC 14.2, CPU preset. Targeted builds downgrade pre-existing ignored-attributes, dangling-reference, range-loop-construct and misleading-indentation warnings (plus maybe-uninitialized in the existing diagnostics source); these accommodations are not source changes. The all-target build is blocked by existing assets_test.cpp use of std::count_if without . A clean full L0/L1 result, GPU tests and model-asset execution are not claimed.

Checklist

  • Full ctest --preset cpu -L "L0|L1" passes locally
  • Added/updated focused regressions and ran the affected target
  • No public interface/configuration change requiring additional documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant