feat: estimate cycles

[not-matthias]

No description provided.

[codspeed-hq]

Merging this PR will improve performance by 11.8%

⚡ 1 improved benchmark
❌ 3 regressed benchmarks
✅ 36 untouched benchmarks
⏩ 80 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_valgrind[valgrind-3.25.1, python3 testdata/test.py, full-with-inline]	6.9 s	234.3 s	-97.05%
❌	test_valgrind[valgrind-3.25.1, testdata/take_strings-aarch64 varbinview_non_null, full-with-inline]	7.7 s	94.9 s	-91.86%
❌	test_valgrind[valgrind-3.26.0, python3 testdata/test.py, full-no-inline]	6.8 s	8.7 s	-22.18%
⚡	test_valgrind[valgrind-3.25.1, echo Hello, World!, full-with-inline]	612,627.1 ms	733.7 ms	×830

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing feat/cycle-estimation (d2d09c3) with master (fa9ee2e)}

Footnotes

1. 80 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Greptile Summary

This PR adds --cycle-estimation=yes to Callgrind: at instrumentation time each guest instruction is decoded by Capstone and looked up in a generated cost table (x86_caps_lut.inc / arm64_caps_lut.inc) keyed by a packed (insn-id, operand-sig) value defined in sigkey.h. The resulting centi-cycle throughput cost is stored per-InstrInfo, accumulated into a running cy_incl sum for O(1) call-graph inclusive updates, and surfaced as a new Cy event in Callgrind output.

• Capstone is statically linked from a hardening-free, x86+arm64-only build; configure.ac makes it mandatory for this fork. CI workflows build and cache the library before the Valgrind configure step.
• The cycledecode.c translation unit is deliberately isolated from Valgrind tool headers and provides all required libc symbols as freestanding shims backed by a 1 MiB bump arena.
• A Python/C lockstep test (lut-gen/test_lut.py) validates that sigkey.py and sigkey.h pack keys identically, and golden anchor rows guard against cost regressions in the LUT.

Confidence Score: 5/5

Safe to merge. The cycle-estimation path is entirely opt-in and off by default; all new code paths are guarded by CLG_(clo).cycle_estimation. Capstone is statically linked with no runtime dependency.

The change introduces a substantial new subsystem but it is well-isolated: cycledecode.c is a self-contained translation unit with its own freestanding shims, the LUT key contract is validated by a lockstep test, and all integration points in main.c/bbcc.c/sim.c are guarded behind the new CLI flag. The only technical gap is in the realloc shim, which is latent code that Capstone's cs_disasm_iter flow does not exercise in practice.

callgrind/cycledecode.c — specifically the realloc shim's memcpy source-length bound.

Important Files Changed

Filename	Overview
callgrind/cycledecode.c	New file: Capstone-backed cycle-cost decoder with freestanding libc shims and binary-search LUT. One over-read hazard in the realloc shim (uses new size for memcpy source length).
callgrind/cycledecode.h	Clean public C ABI header for the cycle decoder; plain types only, no tool-header pollution.
callgrind/sigkey.h	Packed 64-bit LUT key contract shared between the C runtime and the Python generator; bit layout is clearly documented and tested by test_lut.py.
callgrind/main.c	Adds cycle-cost decode at IMark time (first translation only) and the cy_incl running-sum pass; guarded by CLG_(clo).cycle_estimation throughout. Graceful fallback when Capstone init fails.
callgrind/sim.c	Adds EG_CYCLES event group (conditionally registered, unconditionally added to full set per existing codebase pattern) and per-instruction cycle cost accumulation in cachesim_add_icost.
callgrind/bbcc.c	Uses cy_incl (pre-computed running sum) for O(1) inclusive cycle-cost update at each side exit; correctly handles both normal and skipped-function paths.
configure.ac	Makes Capstone mandatory for this fork with clear error messages; fortify-off CFLAGS match the -nodefaultlibs tool link constraint.
.github/workflows/ci.yml	Adds Capstone static build step before configure; switches to --enable-only64bit, removes now-unneeded 32-bit multilib deps.
.github/workflows/codspeed.yml	Capstone build correctly gated on cache-hit != 'true' && matrix.valgrind == 'local', matching the Valgrind build gate so CAPSTONE_DIR is always set when needed.
.github/workflows/release.yml	Capstone built and CAPSTONE_DIR forwarded via debuild -e; additional-deps matrix entries removed correctly.
debian/rules	Adds --enable-only64bit universally and forwards CAPSTONE_DIR via --with-capstone when set; consistent with the release workflow.
callgrind/lut-gen/lib/sigkey.py	Python mirror of sigkey.h; correctness enforced by test_lut.py's lockstep test against the C header defines.
callgrind/lut-gen/test_lut.py	Comprehensive LUT validation: lockstep sigkey.h vs sigkey.py bit layout, cost non-negativity, flag/width constraints, and golden measured/grafted anchor rows.

Sequence Diagram

sequenceDiagram
    participant VG as Valgrind Core
    participant INST as CLG_(instrument)
    participant CD as cycledecode.c
    participant LUT as Cost LUT (sorted array)
    participant BBCC as setup_bbcc
    participant SIM as cachesim_add_icost

    VG->>INST: instrument(BB, first_translation)
    loop Each IMark (first translation only)
        INST->>CD: "clg_cycle_cost(cia, len, &cy)"
        CD->>CD: cs_disasm_iter → compute_sig
        CD->>LUT: binary search CLG_CSKEY(insn_id, sig)
        LUT-->>CD: "Row{cy, cl} or miss"
        CD-->>INST: "cy (centi-cycles) or fallback=100"
        INST->>INST: "ii->cy_cost = cy"
    end
    INST->>INST: compute cy_incl running sums over BB

    VG->>BBCC: setup_bbcc (side exit / call)
    BBCC->>BBCC: "cost[EG_CYCLES] += last_ii->cy_incl"

    VG->>SIM: cachesim_add_icost(cost, bbcc, ii, exe_count)
    SIM->>SIM: "cost[EG_CYCLES] += exe_count * ii->cy_cost"

Loading

_{Reviews (3): Last reviewed commit: "ci(callgrind): build 64-bit only so cycl..." | Re-trigger Greptile}

[greptile-apps]

Diagnostic variable clg_cd_init_stage not declared in the public header

clg_cd_init_stage is accessed here via an extern declaration inside a function body, which is an unusual pattern that bypasses the header contract. Since cycledecode.h already defines the public ABI for this translation unit, adding extern int clg_cd_init_stage; to that header would keep all callers in sync and make the diagnostic visible to any future caller without copy-pasting the extern declaration.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

realloc copies n (new size) bytes from old, potentially reading past the arena

The comment says "stays within the arena array, so no fault", but that holds only if (old - arena) + n ≤ sizeof(arena). If old sits late in the 1 MiB arena and n is larger than the remaining arena space, memcpy(p, old, n) over-reads past arena[1u << 20] into adjacent memory. A simple clamp to the remaining arena space preserves the copy-what-you-can semantics without needing to track the old block's actual size.

Suggested change

	void* realloc(void* old, size_t n)
	{
	/* Old size is unknown; copy n bytes (stays within the arena array, so no
	* fault) and leak the old block. Rarely hit under cs_disasm_iter. */
	void* p = malloc(n);
	if (p && old)
	memcpy(p, old, n);
	return p;
	}
	void* realloc(void* old, size_t n)
	{
	/* Old size is unknown; copy min(n, remaining arena from old) bytes and
	* leak the old block. Rarely hit under cs_disasm_iter. */
	void* p = malloc(n);
	if (p && old) {
	size_t arena_end = (size_t)(arena + sizeof(arena));
	size_t old_pos = (size_t)old;
	size_t avail = (old_pos < arena_end) ? (arena_end - old_pos) : 0;
	memcpy(p, old, avail < n ? avail : n);
	}
	return p;
	}