Skip to content

[feat] add read-only Starlark evaluation (LET-35) - #186

Closed
vt128 wants to merge 1 commit into
masterfrom
codex/starlark-eval-let35
Closed

vt128 wants to merge 1 commit into
masterfrom
codex/starlark-eval-let35

Conversation

@vt128

@vt128 vt128 commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Superseded by #187, which implements the approved go_idiomatic builtin design. This earlier separate-module candidate is closed without merging; its original description and validation history remain below.


Starlet can execute whole scripts, but has no expression module with an explicit read-only data boundary. Add starlark.eval(source, vars={}) and starlark.try_eval(...) for Starlark syntax. The expr name remains available for a separate Expr-language adapter; no Expr dependency or syntax fallback is introduced. Implements LET-35.

The module copies and freezes native data inputs, preserves nested types and aliases, rejects cycles/host objects/callables, and copies, validates and freezes returned data. Native read methods and private temporary computation remain available. Unapproved Universe builtins, including print, are blocked without changing global dialect flags or Universe. Ordinary failures can return bounded structured error data; cancellation, accumulated caller steps and caller panic behavior remain in force. There is no implicit caller-local capture or Machine re-entry.

Source/data admission limits are conservative and documented. They do not repair the pinned parser generally or provide hard process memory/CPU isolation. Direct Go callers own Thread cancellation and source-graph synchronization. Register and classify the module, and update module-list tests and documentation together.

Validation completed:

  • Behavioral red tests retained for missing registry/source guards; defensive tests cover typed nils, hostile host values, cycles, alias-path depth, scalar/edge quotas, output capabilities, bounded diagnostics and caller Cancel/Uncancel/OnMaxSteps behavior.
  • Full host Go 1.27.3 and Docker Go 1.19.13 build, race tests repeated twice, atomic coverage and vet pass. Final candidate tests were rerun after a test-only cancellation supplement; earlier results are retained.
  • New module covers 334/335 statements. The remaining statement is the unreachable compile-error panic for a trusted integer-literal VM probe. All existing 53 host / 50 floor file pairs retain ordered statement weights and hit flags. Total statement coverage increases from 80.862% to 81.957% on host (including existing vendor code), and 95.102% to 95.413% at the floor.
  • Independent static security review passes. No dependency, Go floor, interpreter pin, or existing Machine layout/execution method changes.

Framework validation also completed:

  • Four consumer modules (Starbox with the test-only BOX-14 companion, CLI main/examples, and Base), host/floor and baseline/candidate: 16 fresh legs pass. All eight coverage pairs, 3,328 blocks and 26 package pairs retain coverage; six package pairs have no unit tests, and 32 skips are classified. Examples were built, not demo-run. Independent terminal review passes.
  • Sixteen actual CLI invocations validate registration, rich values, read-only parent data, None, ordinary errors, rejected output capabilities/print, and fatal shared-budget exhaustion. The first acceptance script incorrectly expected exit 1 for budget exhaustion; the frozen CLI contract specifies exit 5. Both original failures are retained and accepted by a separate source-backed audit, without rerunning or changing the product.
  • The added builtin name is an expected registry difference. Starbox preserves its explicit Safe/Network allowlists and tests Full/explicit opt-in. CLI follows its existing CapPure policy.
  • Exact-head 23 required / 24 reported checks and both Codecov contexts pass; checks-watch exits 0 and head/base remain unchanged. Five minor Markdown findings were reviewed: four line-length findings conflict with the repository's explicit MD013=false rule; one required-heading finding conflicts with established module headings. No Go finding or service configuration change.

The sole predeclared performance comparison is complete: six alternating baseline/candidate pairs per compiler at 500ms/CPU1, plus three absolute-cost samples per new-feature case. Existing B/op and allocs/op are unchanged. Paired timing medians and bootstrap 95% intervals:

Compiler / benchmark Change Interval
Host / RunNoCache -1.13% -3.50% to +0.11%
Host / RunMemoryCache +0.10% -0.95% to +0.83%
Floor / RunNoCache -17.08% -17.77% to -0.72%
Floor / RunMemoryCache +0.55% -0.68% to +1.11%

The large apparent floor NoCache gain is not causally attributed to this change; existing execution methods are unchanged. Intervals crossing zero do not establish equivalence. No extra samples were added. New Scalar/Nested/TryError medians are 3.577/8.204/4.182 microseconds on host and 7.282/14.780/8.504 at the floor; these are absolute costs, not comparisons with a previous implementation.

Keep this PR in draft pending full-ecosystem and final performance acceptance. This directed cohort does not replace the full 84-leg ecosystem cohort or prove live-service, native Windows/WASM, or hard process-isolation behavior. No tag, release, or auto-merge.

@codacy-production

codacy-production Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 5 minor

Results:
5 new issues

Category Results
BestPractice 1 minor
CodeStyle 4 minor

View in Codacy

🟢 Metrics 396 complexity · 13 duplication

Metric Results
Complexity 396
Duplication 13

View in Codacy

🟢 Coverage 99.78% diff coverage · +0.27% coverage variation

Metric Results
Coverage variation ✅ +0.27% coverage variation (-1.00%)
Diff coverage ✅ 99.78% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (bf94a75) 7915 7501 94.77%
Head commit (179266c) 8367 (+452) 7952 (+451) 95.04% (+0.27%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#186) 452 451 99.78%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Nov 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.46950% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.90%. Comparing base (bf94a75) to head (179266c).

Files with missing lines Patch % Lines
lib/starlark/starlark.go 99.04% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #186      +/-   ##
==========================================
+ Coverage   93.57%   93.90%   +0.33%     
==========================================
  Files          50       53       +3     
  Lines        6316     6692     +376     
==========================================
+ Hits         5910     6284     +374     
- Misses        258      259       +1     
- Partials      148      149       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant