Skip to content

Commit 507f075

Browse files
r41k0uclaude
andcommitted
Tests: Record what the first selftest porting spike found
The four ports were the experiment; this is the result. Covers whether LLM-assisted porting is worth continuing (yes, in feature-sized increments), what a passing port actually proves (a compiler assertion, not a behavioural one), and the four distinct global-variable shapes the spike turned up. Includes the finding that libbpf implements globals as single-element BPF_MAP_TYPE_ARRAY maps, so a one-element ArrayMap -- not a HashMap -- is the structurally faithful stand-in, and that the ELF half of global support already works today via the machinery behind @bpfglobal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 0459b3a commit 507f075

1 file changed

Lines changed: 111 additions & 0 deletions

File tree

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Porting kernel selftests: what the first spike found
2+
3+
Four programs from `tools/testing/selftests/bpf/progs/` were ported as an experiment,
4+
to answer two questions before anyone commits to doing this at scale:
5+
6+
1. Is LLM-assisted porting of kernel selftests viable?
7+
2. What must real global-variable support actually handle?
8+
9+
Short answers: **viable, with a caveat about what a passing port proves**; and
10+
**four distinct global shapes showed up in four programs**, which is the more
11+
actionable finding.
12+
13+
## The spike
14+
15+
| Port | Upstream | Section | Outcome |
16+
|---|---|---|---|
17+
| `tracing/tracepoint_sched_switch.py` | `test_tracepoint.c` | `tracepoint/sched/sched_switch` | passes |
18+
| `tracing/get_cgroup_id.py` | `get_cgroup_id_kern.c` | `tracepoint/syscalls/sys_enter_nanosleep` | passes |
19+
| `tracing/autoattach.py` | `test_autoattach.c` | `raw_tp/sys_enter`, `raw_tp/sys_exit` | passes |
20+
| `vmlinux/perf_skip.py` | `test_perf_skip.c` | `perf_event` | strict xfail — nested ctx access |
21+
22+
## 1. Is it viable?
23+
24+
**Yes, for programs inside the envelope — three of four compiled and passed `llc` on the
25+
first attempt.** The mechanical part of a port (decorators, ctypes annotations, map
26+
declarations, helper names) is regular enough to be reliable.
27+
28+
The failure was not a translation error. `perf_skip` needs `ctx.regs.ip`, which PythonBPF
29+
genuinely cannot express, and no amount of care in the port changes that. That is the
30+
useful kind of failure: it converts into a roadmap test that documents the gap.
31+
32+
Two caveats that matter more than the pass rate:
33+
34+
**A passing port proves less than the test it came from.** A kernel selftest is two
35+
halves — the BPF program, and a `prog_tests/` driver that loads it through a skeleton,
36+
triggers it, and asserts on the result. Only the BPF half is portable here, because this
37+
framework compiles and verifies but never runs. Everything ported becomes a compiler
38+
assertion: *PythonBPF emits a loadable, verifiable object for this program type and
39+
feature mix*. That is worth having — it is how the `raw_tp` and `perf_event` program types
40+
came under test at all — but it is not what "we ported the kernel's selftests" sounds
41+
like. Closing that gap needs a runtime test tier, which is a much larger piece of work.
42+
43+
**Selection is the expensive step, not translation.** Of 820 real programs, 28 are
44+
portable today. Picking those out required scoring the whole corpus against the compiler's
45+
actual envelope; guessing from filenames does not work. The classifier that did it is
46+
worth keeping around and re-running after each feature lands.
47+
48+
**Recommendation: viable and worth continuing, in small increments tied to features.**
49+
Port a handful, let them reveal the next gap, fix the gap, port more. Bulk porting ahead of
50+
the features would just produce a large pile of xfails.
51+
52+
## 2. What real globals must support
53+
54+
Every port that touches a global currently substitutes a one-entry `HashMap`, tagged
55+
`WORKAROUND(globals)`. Four programs produced four distinct shapes:
56+
57+
| Shape | Example | What globals must support |
58+
|---|---|---|
59+
| none | `tracepoint_sched_switch` | — (control case) |
60+
| scalar in + scalar out | `get_cgroup_id` | read a global, write a different one |
61+
| flags across programs | `autoattach` | two programs in one object sharing global state |
62+
| scalar in, compared against ctx | `perf_skip` | read-only input set by userspace before attach |
63+
| array + cursor *(next increment)* | `cgroup_preorder` | indexed writes and read-modify-write on a global |
64+
65+
The last row is not in this spike but is the recommended next port precisely because it is
66+
the most demanding shape: `result[idx++] = N` needs an array global *and* a read-modify-write
67+
cursor, which together constrain the design more than anything here does.
68+
69+
### A design note worth acting on
70+
71+
**libbpf implements global variables as single-element `BPF_MAP_TYPE_ARRAY` maps.**
72+
`.bss`, `.data` and `.rodata` become internal array maps at load time. Two consequences:
73+
74+
- A one-element **`ArrayMap`** is the structurally faithful stand-in for a global, not a
75+
`HashMap`. `HashMap` is used here only because `ArrayMap` is still a placeholder that
76+
raises `NotImplementedError`. Landing `ArrayMap` first would make the eventual migration
77+
to real globals close to mechanical.
78+
- **Most of the ELF work is already done.** `@bpfglobal` is vestigial — a metadata carrier
79+
for `LICENSE` — but the machinery behind it already emits globals that LLVM places into
80+
`.bss` and `.data` correctly, and that libbpf already recognises:
81+
82+
```
83+
libbpf: map 'g.bss' (global data): at sec_idx 5, offset 0, flags 0.
84+
libbpf: map 'g.data' (global data): at sec_idx 6, offset 0, flags 0.
85+
```
86+
87+
What is missing is narrower than "implement global variables": name resolution in
88+
`expr_pass.get_operand_value` (which resolves against `local_sym_tab`, then vmlinux
89+
enums, then gives up), a Python-level surface for declaring one, and userspace access
90+
through `pylibbpf`.
91+
92+
## 3. Incidental findings
93+
94+
- **Nested struct field access fails with a misleading error.** `ctx.regs.ip` reports
95+
`SyntaxError: Undefined variable actual` — naming the assignment target rather than the
96+
nested access that caused it. `_allocate_for_attribute` declines to allocate when the
97+
attribute's base is not a plain `Name`, logging at debug level, and the expression pass
98+
then trips over the missing symbol. The diagnostic should name the real cause.
99+
- **One level of nested-context access already works.** `ctx.sample_period` on
100+
`struct_bpf_perf_event_data` compiles and `llc`s cleanly, so `perf_event` contexts are
101+
usable today for anything that does not need `regs`.
102+
- **`@section` really does accept anything.** `tc`, `socket`, `fentry/…`, `lsm/…`,
103+
`cgroup_skb/egress`, `netfilter` and `tp_btf/…` all compile and land in the ELF verbatim.
104+
Program type is not a constraint; the context type is.
105+
106+
## Re-running the corpus scoring
107+
108+
The audit behind this spike scored all 976 programs against the compiler's envelope. It is
109+
worth re-running after each feature lands, to see what the change unlocked. The blocker
110+
histogram at the time of writing, over 820 real programs: globals 52%, verifier-test
111+
annotations 27%, typed program macros 26%, kfuncs 20%, inline asm 16%, loops 12%.

0 commit comments

Comments
 (0)