Skip to content

Commit caf87e2

Browse files
committed
ownership-lab: semantic-coverage results (census, freeze inputs, OFF/ON scans, witness passes, triage, textual recount) and final scripts
Research corpus only; no production change. Census and scan outputs are gzipped; witness verdicts, rankings, metrics, triage and answers are plain JSON. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Am9eQwzNfbugH72eVKetC2
1 parent 11a23fd commit caf87e2

55 files changed

Lines changed: 28776 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
{
2+
"answers": {
3+
"1 Is a simple IL-body hash a sufficient proof identity?": "NO for compositional rows. a1-v1: with Caller.Make's IL unchanged and only Helper swapped (v1 fresh / v2 cached), five of six variants (one-hop, two-hop, virtual/interface, static cached singleton, generic helper) keep the caller's body hash while the witness flips fresh -> cached. Only the direct-syntactic class (A) is safe under a body hash. KILLED as an identity for anything that reaches through a callee.",
4+
"2 What is the cheapest dependency-sensitive identity that survives the falsifier?": "A TYPED proof-slice Merkle hash: H(own IL hash, IL hashes of the transitive callee closure restricted to callees returning disposable / type-parameter / object types, plus the .cctor of types whose static fields are read); a bodiless member in the slice (interface/abstract/virtual target) makes the row binary-pinned (MVID) instead of hash-portable. a2-v1: catches B/C/E/F, refuses D as binary-pinned; mean slice size 4.6 members; TFM builds differ in 2/264 slices; patch release 0 changes; major jump 138/293. KEEP_AS_SEMANTIC_IDENTITY. The E2 census then showed the practical cost: binary-pinned rows do not transfer between the per-TFM builds of one package version (net8.0 vs net10.0 assets), so the identity needs a re-pin per resolved asset, which for source-derived rows is a re-derivation on the same source (rederived_same_source class) and for witnessed rows a witness on that asset (TFM-targeted witnesses).",
5+
"3 How much of the USED surface do whole-source rows cover?": "13% weighted by public call-site lines over the frozen eight-library surface (stage-b-v1: 64 rows / 507 surface callables; 43 of 360 used callables carry a row; 317 used callables are non-provable); 15.4% when the production ADO.NET convention (CreateCommand / BeginTransaction / ExecuteReader tracked without rows) is counted as covered (stage-cd addendum, 597-row census). Whole-source derivation is also NOT monotone: SSH.NET's heavily used wrapper factories (SftpClient.Open/OpenRead/OpenWrite/Create) and Npgsql's CreateCommand are proved on the 25-file subset and LOST on the whole tree (H-22: an interface resolving as IDisposable turns the adopting constructor argument into an escape).",
6+
"4 Which high-frequency surfaces remain non-provable?": "stage-c universe (205 callables after the preregistered filter: 73 getters routed to ALIAS_OR_SHARED, 126 below the use threshold, 34 bodiless routed to NEEDS_IMPLEMENTATION_EVIDENCE, 4 internal namespaces): SkiaSharp 147 (Copy, Encode, Resize, Snapshot, PeekPixels, Subset, Create* families through native handles), Npgsql 34 (data-source / connection / reader / COPY families), SSH.NET 12 (the H-22 wrappers), MailKit 7, StackExchange.Redis 3, RabbitMQ.Client 1, LibGit2Sharp 1. The reasons are native handles (P/Invoke returns), pooled/reused instances, interface-typed adoption and lock/ambient-token bodies, not missing source.",
7+
"5 Can relational candidates be independently validated?": "YES, cheaply: 166 of the 205 candidates were executed by the H-10 runtime witness (39 unwitnessable: no receiver / no arguments / no environment); labels VALIDATED_FRESH 122, VALIDATED_SHARED_OR_CACHED 12, INCONCLUSIVE 30 (14 null results, 10 reused-instance in sequential mode, 5 environment, 1 mixed), UNWITNESSABLE 2 absent in the tested build. Every label is a runtime observation on the exact binary; the LLM never labels; no label came from the ranking mechanisms.",
8+
"6 Do relational rules beat name+LLM per validator budget?": "By a few rows only. Validated fresh per 25 executions: D (logistic regression over RelDump relations) 18, U (usage share) 18, C (rules) 17, B (LLM) 15, A (names) 13; per 10: U 9, B 8, D 7, A 6, C 4. Contradicted (proposed fresh, witnessed cached) at 25: A 1, B 2, C 2, D 1, U 0. The ranking question is moot at this scale because the whole filtered universe was witnessed for about nine minutes of compute: the binding cost was environment engineering (receivers, arguments, servers, protocol state), not validator executions.",
9+
"7 Does a simple learned model beat the rules?": "Marginally (18 vs 17 at 25; 7 vs 4 at 10) with library holdout (trained on the other libraries' CT positives). Not a decision-relevant margin, and it is ranking only: it never labels.",
10+
"8 Has CodeTrek-proper earned a demanding instance?": "NO. Floors met (205 candidates, 134 validated labels, 122 useful positives) but condition 5 fails: there is no validator-budget problem to solve when exhaustive witnessing of the filtered universe is this cheap, and the relational arms lead the baselines by a few rows. Learned walks stay outside the trusted base and outside the plan (stage-cd codetrek_proper_gate).",
11+
"9 Do binary-only surfaces have unique value?": "MEASURED, SMALL: all 122 fresh rows are binary-only-style rows (RelDump -> witness -> row pinned to the tested asset, no source). In the E2 census they applied as witnessed_exact_binary on every consumer that resolved a tested asset (NpgsqlRest 15 rows, RepoDB 754 row-resolutions, wolverine 2955, quartznet 195, DatabaseManager 150, FeatherQR 651 plus 216 by TYPED-slice identity transfer across TFM builds), but they produced exactly 3 trusted opportunities (RepoDB BulkOperations NpgsqlConnection.BeginBinaryImport, one source span on three TFMs) and 0 ON-only findings. Their unique value is real but thin: the heavily used non-provable callables are called in using-declarations (FeatherQR: 74 declaration-shaped sites, all `using`), in assignment shapes (H-23), or under reuse protocols (Npgsql readers/transactions). The witness path stays KEEP_AS_VALIDATOR; the rows it yields are cheap to keep current per asset (a witness per package version and TFM) but buy few findings on this pool.",
12+
"10 Is dense-consumer exposure reachable?": "NOT with a textual pool: 17 consumers censused (ranks 1-18 of the frozen top-30 minus one unclonable), 43 trusted semantic opportunities (13 in production projects, 30 in test/sample/doc projects; 26 TFM-collapsed spans, 8 production), 7 consumers with at least one (5 production), 2 dependency families (Npgsql, SharpZipLib). The floor (>=10 consumers AND >=30 opportunities AND >=5 with >=1 AND >=3 families) is met on the consumer count and, only when test projects are counted, on the opportunity count; it fails on families either way and on production opportunities. Decision INCONCLUSIVE_EXPOSURE. The census explains the gap mechanically: NpgsqlRest (rank 1 by text, 475 matches) writes its 59 production CreateConnection calls as assignments (H-23); marten/weasel/wolverine/WebVella resolve Npgsql 9.0.x/10.0.0 (version drift, cured by three two-minute re-derivations, which produced 30 of the 43 opportunities); FeatherQR's SkiaSharp calls are using-declarations; aspire, PerformanceMonitor and efcore.pg cannot be restored in this container; two consumers pin SkiaSharp 1.60/3.119.",
13+
"11 Production findings?": "UNDEREXPOSED MEASUREMENT: OFF vs ON with the frozen rows on the 7 consumers with opportunities gave 7 unique ON-only sites and 0 lost findings, engines in agreement on every project/TFM unit: 1 TRUE_PRODUCTION (victor-wiki/DatabaseManager DbConnector.cs:50: a new NpgsqlDataSourceBuilder(...).Build() data source, hence a new connection pool, per connection request, never disposed; unbounded pool growth), 5 TRUE_TEST_SAMPLE (JasperFx/weasel DocSamples: per-call NpgsqlDataSource never disposed in documentation samples), 1 BENIGN (NpgsqlRest Builder.cs:3250: the multi-host source behind a cached wrapper is never disposed but is built once at startup). All seven come from the source-derived NpgsqlDataSource family (Build / BuildMultiHost / Create); no witnessed-only row produced a finding.",
14+
"12 Findings unique to Own.NET versus a manual model elsewhere?": "The one production finding and the benign one are OWN_NET_ONLY: IDisposableAnalyzers 4.0.8 (built into the projects) flags neither DbConnector.cs:50 nor Builder.cs:3250 (it flags a neighbouring site in each), CA2000 flags neither; the five documentation-sample findings are EXISTING_TOOL_WITHOUT_MODEL (IDISP001 at the same lines from the return type alone, CA2000 on the static Create). CodeQL could not be run in the container (UNKNOWN for CodeQL). Note the inverse case: Builder.cs:3088, the same pattern as 3250, is flagged by IDisposableAnalyzers and not by Own.NET (an indexer store of a derived value is treated as an escape).",
15+
"13 Cheapest Pareto-optimal pipeline?": "(1) whole-source derivation with the frozen rules per (package, version) actually resolved by the consumers, re-run on version drift (minutes; it produced every finding); (2) the TYPED proof-slice identity to re-pin rows across TFM builds and patch releases and to refuse binary-pinned transfers; (3) exhaustive runtime witnessing of the filtered non-provable universe with sequential mode, the receiver probe and per-asset TFM targeting, as a validator, not as a ranking problem; (4) no relational ranking beyond a cheap filter, no LLM in the trusted base, no learned walks; (5) a semantic (hit-log) census, never a textual one, to pick consumers, with version re-derivation before the census; (6) the assignment-site application gap (H-23) fixed in the engine BEFORE the next production experiment, because it, not row breadth, now bounds exposure on real consumers. What actually bought a bug we could not otherwise find: whole-source E1 rows on a heavily used factory family (NpgsqlDataSource) applied on the consumer's exact package version, plus Own.NET's declaration-site tracking where IDisposableAnalyzers stays silent."
16+
},
17+
"review_minutes": 35,
18+
"model_only_competitor_findings": "none measured: the LLM arm produced candidates only (never labels, never findings); no model-only tool was run on the consumers",
19+
"decision_binary_only": "KEEP_AS_VALIDATOR (witness path); the rows it yields are exact-asset rows with thin exposure on this pool",
20+
"decision_dense_consumers": "NARROW to the semantic census with per-version re-derivation; INCONCLUSIVE_EXPOSURE on the frozen pool",
21+
"decision_e3": "MEASURED, UNDEREXPOSED: 1 TRUE_PRODUCTION (OWN_NET_ONLY), 5 TRUE_TEST_SAMPLE, 1 BENIGN, 0 FALSE, 0 lost",
22+
"final_question": "Own.NET can obtain trustworthy resource/lifetime semantics broadly enough to matter only where three things line up: a heavily used factory family, source-derived rows on the exact package version the consumer resolves, and a call-site shape the engine applies effects to. Deterministic whole-source proofs supplied every row that produced a finding; the dependency-sensitive identity kept those rows valid across TFM builds and patch releases and refused the transfers that are not valid; the runtime witness validated 122 further rows cheaply but they reached almost no verdict-changing site; relational candidate discovery ordered a universe that was cheaper to witness whole; learned models stayed outside the trusted base at no loss. The piece that actually bought a bug we could not otherwise find is the combination of whole-source E1 rows re-derived per resolved version with Own.NET's declaration-site tracking (DatabaseManager's per-request NpgsqlDataSource pool leak, silent in IDisposableAnalyzers and CA2000). The piece that now bounds everything is not semantics but application: assignment-site factory calls (H-23) and using-declarations make most real call sites invisible or irrelevant to the oracle.",
23+
"unresolved": [
24+
"H-23: how many of the 59 NpgsqlRest and 18-33 assignment-shaped sites per consumer would become verdict-changing opportunities if the minted-return lowering applied at assignment sites (not measured: frozen rules)",
25+
"whether the reused-instance factories (Npgsql readers/transactions, 10 sequential-cached rows) can be validated by a pooled-ownership witness rather than identity",
26+
"the exposure floor on a semantically ranked pool: the census here started from a textual pool; a pool ranked by hit-log opportunities on restored consumers was not built",
27+
"Builder.cs:3088 versus 3250: Own.NET's escape rule for indexer stores of derived values (one identical pattern, disjoint tool verdicts)",
28+
"MailKit and RabbitMQ.Client remain UNWITNESSABLE_ENV (no server in the container)"
29+
],
30+
"rejected": [
31+
"simple IL-body hash as a row identity (A1)",
32+
"learned walks / CodeTrek-proper in this track (no validator-budget problem)",
33+
"new inference patterns or API names during Stage B/E (frozen rules)",
34+
"MVID as a semantic identity (it is only a pin)"
35+
]
36+
}
Lines changed: 299 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,299 @@
1+
{
2+
"per_consumer": [
3+
{
4+
"repo": "DuendeSoftware/products",
5+
"sha": "100e14c801",
6+
"production_projects": 111,
7+
"restored_ok": 111,
8+
"trusted_units": 0,
9+
"unverified_units": 1,
10+
"units_by_class": {
11+
"version_drift": 1
12+
},
13+
"families_hit": [],
14+
"extractor_failures": 0,
15+
"textual_assignment_sites": 0,
16+
"textual_declaration_sites": 0,
17+
"trusted_units_production": 0
18+
},
19+
{
20+
"repo": "JasperFx/marten",
21+
"sha": "7a6c9c3bb1",
22+
"production_projects": 55,
23+
"restored_ok": 55,
24+
"trusted_units": 6,
25+
"unverified_units": 0,
26+
"units_by_class": {
27+
"rederived_same_source": 6
28+
},
29+
"families_hit": [
30+
"Npgsql"
31+
],
32+
"extractor_failures": 0,
33+
"textual_assignment_sites": 58,
34+
"textual_declaration_sites": 238,
35+
"trusted_units_production": 0
36+
},
37+
{
38+
"repo": "JasperFx/weasel",
39+
"sha": "73b158bf5a",
40+
"production_projects": 22,
41+
"restored_ok": 23,
42+
"trusted_units": 20,
43+
"unverified_units": 0,
44+
"units_by_class": {
45+
"rederived_same_source": 20
46+
},
47+
"families_hit": [
48+
"Npgsql"
49+
],
50+
"extractor_failures": 0,
51+
"textual_assignment_sites": 36,
52+
"textual_declaration_sites": 336,
53+
"trusted_units_production": 4
54+
},
55+
{
56+
"repo": "JasperFx/wolverine",
57+
"sha": "7ee3df905e",
58+
"production_projects": 128,
59+
"restored_ok": 130,
60+
"trusted_units": 4,
61+
"unverified_units": 2,
62+
"units_by_class": {
63+
"rederived_same_source": 4,
64+
"version_drift": 2
65+
},
66+
"families_hit": [
67+
"Npgsql"
68+
],
69+
"extractor_failures": 0,
70+
"textual_assignment_sites": 25,
71+
"textual_declaration_sites": 131,
72+
"trusted_units_production": 2
73+
},
74+
{
75+
"repo": "NpgsqlRest/NpgsqlRest",
76+
"sha": "6cc6f55fa6",
77+
"production_projects": 11,
78+
"restored_ok": 12,
79+
"trusted_units": 7,
80+
"unverified_units": 1,
81+
"units_by_class": {
82+
"version_drift": 1,
83+
"rederived_same_source": 7
84+
},
85+
"families_hit": [
86+
"Npgsql"
87+
],
88+
"extractor_failures": 0,
89+
"textual_assignment_sites": 57,
90+
"textual_declaration_sites": 58,
91+
"trusted_units_production": 2
92+
},
93+
{
94+
"repo": "WebVella/WebVella-ERP",
95+
"sha": "389e4ff7de",
96+
"production_projects": 19,
97+
"restored_ok": 20,
98+
"trusted_units": 0,
99+
"unverified_units": 0,
100+
"units_by_class": {},
101+
"families_hit": [],
102+
"extractor_failures": 0,
103+
"textual_assignment_sites": 39,
104+
"textual_declaration_sites": 116,
105+
"trusted_units_production": 0
106+
},
107+
{
108+
"repo": "dotnet-ad/SkiaSharp.Components",
109+
"sha": "4d5d3fcbf0",
110+
"production_projects": 11,
111+
"restored_ok": 12,
112+
"trusted_units": 0,
113+
"unverified_units": 1,
114+
"units_by_class": {
115+
"version_drift": 1
116+
},
117+
"families_hit": [],
118+
"extractor_failures": 0,
119+
"textual_assignment_sites": 0,
120+
"textual_declaration_sites": 0,
121+
"trusted_units_production": 0
122+
},
123+
{
124+
"repo": "erikdarlingdata/PerformanceMonitor",
125+
"sha": "b342229e3c",
126+
"production_projects": 22,
127+
"restored_ok": 0,
128+
"trusted_units": 0,
129+
"unverified_units": 0,
130+
"units_by_class": {},
131+
"families_hit": [],
132+
"extractor_failures": 0,
133+
"textual_assignment_sites": 0,
134+
"textual_declaration_sites": 0,
135+
"trusted_units_production": 0
136+
},
137+
{
138+
"repo": "guitarrapc/FeatherQR",
139+
"sha": "02e5112561",
140+
"production_projects": 7,
141+
"restored_ok": 6,
142+
"trusted_units": 0,
143+
"unverified_units": 0,
144+
"units_by_class": {},
145+
"families_hit": [],
146+
"extractor_failures": 0,
147+
"textual_assignment_sites": 11,
148+
"textual_declaration_sites": 74,
149+
"trusted_units_production": 0
150+
},
151+
{
152+
"repo": "microsoft/aspire",
153+
"sha": "4f442cdb5a",
154+
"production_projects": 379,
155+
"restored_ok": 0,
156+
"trusted_units": 0,
157+
"unverified_units": 0,
158+
"units_by_class": {},
159+
"families_hit": [],
160+
"extractor_failures": 0,
161+
"textual_assignment_sites": 0,
162+
"textual_declaration_sites": 0,
163+
"trusted_units_production": 0
164+
},
165+
{
166+
"repo": "mikependon/RepoDB",
167+
"sha": "edabfb2044",
168+
"production_projects": 104,
169+
"restored_ok": 104,
170+
"trusted_units": 3,
171+
"unverified_units": 0,
172+
"units_by_class": {
173+
"witnessed_exact_binary": 3
174+
},
175+
"families_hit": [
176+
"Npgsql"
177+
],
178+
"extractor_failures": 0,
179+
"textual_assignment_sites": 18,
180+
"textual_declaration_sites": 366,
181+
"trusted_units_production": 3
182+
},
183+
{
184+
"repo": "npgsql/efcore.pg",
185+
"sha": "930bd291f7",
186+
"production_projects": 3,
187+
"restored_ok": 0,
188+
"trusted_units": 0,
189+
"unverified_units": 0,
190+
"units_by_class": {},
191+
"families_hit": [],
192+
"extractor_failures": 0,
193+
"textual_assignment_sites": 0,
194+
"textual_declaration_sites": 0,
195+
"trusted_units_production": 0
196+
},
197+
{
198+
"repo": "potatobeanradio/circuitRF",
199+
"sha": "9761afe90b",
200+
"production_projects": 22,
201+
"restored_ok": 23,
202+
"trusted_units": 0,
203+
"unverified_units": 0,
204+
"units_by_class": {},
205+
"families_hit": [],
206+
"extractor_failures": 0,
207+
"textual_assignment_sites": 0,
208+
"textual_declaration_sites": 0,
209+
"trusted_units_production": 0
210+
},
211+
{
212+
"repo": "quartznet/quartznet",
213+
"sha": "15d90a9c26",
214+
"production_projects": 42,
215+
"restored_ok": 43,
216+
"trusted_units": 1,
217+
"unverified_units": 0,
218+
"units_by_class": {
219+
"rederived_same_source": 1
220+
},
221+
"families_hit": [
222+
"Npgsql"
223+
],
224+
"extractor_failures": 0,
225+
"textual_assignment_sites": 88,
226+
"textual_declaration_sites": 938,
227+
"trusted_units_production": 0
228+
},
229+
{
230+
"repo": "seiggy/lucia-dotnet",
231+
"sha": "8b27632738",
232+
"production_projects": 17,
233+
"restored_ok": 18,
234+
"trusted_units": 0,
235+
"unverified_units": 0,
236+
"units_by_class": {},
237+
"families_hit": [],
238+
"extractor_failures": 0,
239+
"textual_assignment_sites": 19,
240+
"textual_declaration_sites": 59,
241+
"trusted_units_production": 0
242+
},
243+
{
244+
"repo": "victor-wiki/DatabaseManager",
245+
"sha": "032506d906",
246+
"production_projects": 15,
247+
"restored_ok": 14,
248+
"trusted_units": 2,
249+
"unverified_units": 0,
250+
"units_by_class": {
251+
"rederived_same_source": 2
252+
},
253+
"families_hit": [
254+
"ICSharpCode.SharpZipLib",
255+
"Npgsql"
256+
],
257+
"extractor_failures": 0,
258+
"textual_assignment_sites": 3,
259+
"textual_declaration_sites": 29,
260+
"trusted_units_production": 2
261+
},
262+
{
263+
"repo": "wieslawsoltes/ProGPU",
264+
"sha": "7941cdec3e",
265+
"production_projects": 129,
266+
"restored_ok": 110,
267+
"trusted_units": 0,
268+
"unverified_units": 0,
269+
"units_by_class": {},
270+
"families_hit": [],
271+
"extractor_failures": 0,
272+
"textual_assignment_sites": 0,
273+
"textual_declaration_sites": 0,
274+
"trusted_units_production": 0
275+
}
276+
],
277+
"floor": {
278+
"consumers": 17,
279+
"unique_trusted_opportunities": 43,
280+
"unique_trusted_opportunities_production": 13,
281+
"unique_trusted_opportunities_test_projects": 30,
282+
"consumers_with_at_least_one": 7,
283+
"consumers_with_at_least_one_production": 5,
284+
"dependency_families": [
285+
"ICSharpCode.SharpZipLib",
286+
"Npgsql"
287+
],
288+
"dependency_families_production": [
289+
"ICSharpCode.SharpZipLib",
290+
"Npgsql"
291+
],
292+
"unverified_units_total": 5,
293+
"unique_source_spans_tfm_collapsed": 26,
294+
"unique_source_spans_tfm_collapsed_production": 8,
295+
"met_all_projects": false,
296+
"met_production_only": false,
297+
"rule": ">=10 consumers AND >=30 unique semantic opportunities AND >=5 consumers with >=1 AND >=3 dependency families; decided on the stricter production-only count (test-project opportunities reported separately)"
298+
}
299+
}
Binary file not shown.
28.2 KB
Binary file not shown.
Binary file not shown.
8.33 KB
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

0 commit comments

Comments
 (0)