Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Inspired by the [*Starlight*](https://github.com/starlight-go/starlight) and [*S
| [`base64`](/lib/base64) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/base64.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/base64) | Base64 encoding & decoding functions |
| [`csv`](/lib/csv) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/csv.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/csv) | Parses and writes comma-separated values (csv) contents |
| [`file`](/lib/file) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/file.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/file) | Functions to interact with the file system |
| [`goidiomatic`](/lib/goidiomatic) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/goidiomatic.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/goidiomatic) | Go idiomatic functions and values for Starlark |
| [`goidiomatic`](/lib/goidiomatic) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/goidiomatic.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/goidiomatic) | Go-style helpers, values and constructors; read-only expression evaluation |
| [`hashlib`](/lib/hashlib) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/hashlib.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/hashlib) | Hash primitives for Starlark |
| [`http`](/lib/http) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/http.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/http) | HTTP client and server handler implementation for Starlark |
| [`json`](/lib/json) | [![godoc](https://pkg.go.dev/badge/github.com/1set/starlet/lib/json.svg)](https://pkg.go.dev/github.com/1set/starlet/lib/json) | Utilities for converting Starlark values to/from JSON strings |
Expand Down
69 changes: 69 additions & 0 deletions lib/goidiomatic/BENCHMARKS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Expression evaluation benchmark notes

These local measurements compare direct compiled Starlark with the public
`eval` and `try_eval` builtins. See [usage and qualitative guidance](README.md#performance).
They describe the measured workloads and systems; run the benchmarks on your
deployment target before making latency or throughput assumptions.

Local measurements on an Apple M4 Pro, Go 1.27.3, darwin/arm64, `-cpu=1`:
medians of six 500 ms runs, in **microseconds per call** (1000 µs = 1 ms).
Both cases use `qty=3`; the scalar case uses `price=12`, and the nested case
evaluates `{"values": [x * qty for x in items]}` with `items=list(range(32))`.

| Path | `price * qty` | 32-item nested result |
| --- | ---: | ---: |
| Normal compiled Starlark function | 0.062 µs | 1.821 µs |
| `eval`, reused `vars` dict | 4.504 µs (72.4×) | 19.709 µs (10.8×) |
| `eval`, new `vars` dict per call | 4.657 µs (75.2×) | 20.022 µs (11.0×) |
| `try_eval`, reused `vars`, success | 4.630 µs (74.6×) | 19.896 µs (11.0×) |

The normal/reused-`eval` allocation costs were 80/6976 B and 1/111 allocations
for the scalar case, and 1760/25025 B and 11/682 allocations for the nested case.
Every path calls a compiled Starlark function with identical inputs. Outer
compilation, input preparation and warmup are excluded; per-call `eval` parsing,
compilation, copying, validation and freezing are included. Normal execution
does not provide this isolation. Ratios are medians of the six run-wise ratios;
these are local examples, not application-wide latency or throughput guarantees.

Use direct script code for fixed calculations and hot loops. `eval` is intended
for dynamic expressions: it does not cache compiled expressions, even when
`vars` is reused. Run the same workloads with:

```sh
go test ./lib/goidiomatic -run '^$' -bench '^BenchmarkEvalComparison$' \
-benchmem -benchtime=500ms -cpu=1 -count=6
```

## Increasing the work per call

`eval` reuses the caller's Thread. Each call builds a private variable environment
and compiles an expression; it does not fork the whole runtime. To measure how
this preparation cost compares with longer computation, keep the source fixed:
`len([(i * 3 + 7) % 97 for i in range(n)])`. Only integer `n` is passed in and
returned, so larger input/output graphs do not add copying work.

On the same host, six alternating normal/eval pairs at 300 ms per workload gave:

| Iterations | Normal (µs) | eval (µs) | Paired extra time (µs) | Paired extra (%) |
| --- | ---: | ---: | ---: | ---: |
| 0 | 0.238 | 7.394 | 7.155 | 3013.1% |
| 10 | 1.336 | 8.603 | 7.255 | 544.4% |
| 100 | 9.514 | 17.097 | 7.619 | 80.9% |
| 1,000 | 91.241 | 99.590 | 8.878 | 9.7% |
| 10,000 | 984.503 | 1035.482 | 54.321 | 5.5% |
| 100,000 | 13076.609 | 12915.692 | -156.653* | -1.3%* |

Times are per-path medians; extra time/percentages are medians of paired
differences/ratios, so subtracting the displayed medians need not give the delta.
At zero iterations, about **7.2 µs** estimates the extra cost for this source and
small data graph. The extra allocation was roughly 9.4 KB per call. Relative
overhead becomes smaller as computation grows, but the time gap is not constant.
*At 100,000 iterations, individual paired differences ranged from -1.21 to
+1.38 ms: the negative median does not establish a speedup or absence of cost.*

These timings depend on CPU, OS, architecture, Go/compiler, GC and system load.
Larger source or input/output graphs also increase preparation work. Re-run
`BenchmarkEvalWorkload` with `-benchtime=300ms -cpu=1 -count=6` on your target
system; these results are estimates for the measured workloads, not a universal
per-call constant.

186 changes: 184 additions & 2 deletions lib/goidiomatic/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# go_idiomatic

`go_idiomatic` provides Go-flavored helpers, constants, and constructors for Starlark scripts — base conversions, `length`/`sum`/`distinct` utilities, struct and shared-dict factories, and `print`-style output. It is **not** a mirror of any single Python module; it borrows the spirit of several Python builtins (`hex`/`oct`/`bin`/`sum`/`bytes.hex`) while adding Star\* idioms.
`go_idiomatic` provides Go-flavored helpers, values and constructors for Starlark, including `true`/`false`/`nil`, structs and shared dictionaries. It also adds collection, output and read-only expression-evaluation helpers.

Capability profile: **Log + Process**. `eprint`/`pprint` write to stderr or the thread's `Print` handler (Log); `sleep` blocks the thread and `exit`/`quit` halt the program (Process). Everything else is pure.

Expand All @@ -10,6 +10,8 @@ Load name for `load()`: `go_idiomatic`.

| function | description |
|----------|-------------|
| `eval(source, vars={}) -> value` | Evaluate a Starlark expression. [Parameters and examples](#expression-evaluation). |
| `try_eval(source, vars={}) -> (value, error_info)` | Go-style value/error results. [Return values and errors](#returns-and-errors). |
| `length(obj) -> int` | Length of a string (in Unicode code points), bytes, or any sequence. |
| `sum(iterable, start=0) -> number` | Sum of `start` plus the items of `iterable`, left to right. |
| `distinct(iterable) -> iterable` | Iterable with duplicates removed; shape follows the input type. |
Expand Down Expand Up @@ -43,6 +45,187 @@ print(true, false, nil)
# Output: True False None
```

## Expression evaluation

Evaluate one **Starlark expression** with `eval`. Use `try_eval` for Go-style
`(value, error)` results: `error_info == None` means success.

```python
load("go_idiomatic", "eval", "try_eval")

total = eval("price * qty", vars={"price": 12, "qty": 3})
print(total) # 36

value, error = try_eval("price") # No implicit caller variables.
print(value, error["phase"]) # None resolve
```

### Parameters

```text
eval(source, vars={}) -> value
try_eval(source, vars={}) -> (value, error_info)
```

Both parameters accept positional or keyword arguments.

| Parameter | Type | Required / default | Meaning |
| --- | --- | --- | --- |
| `source` | Native `string` | Required | A single expression, such as `"price * qty"`. Statements are not accepted. |
| `vars` | Native `dict` with string keys | Optional; `{}` | Variables visible to the expression. Keys must be valid Starlark identifiers, at most 512 bytes each. `None` is not accepted. |

`vars` supports nested native `None`, bool, int, float, string, bytes, list,
tuple, dict and set values. Functions, methods, modules, iterators, cycles and
Go-backed/custom values (including `shared_dict`) are rejected.

Input snapshots and returned containers are frozen; originals are untouched.
Native reads such as `.upper()`, `.get()` and `.index()` work; input writes fail.
New temporary containers inside an expression may be mutated.

```python
items = [1, 2]
value, error = try_eval("items.append(9)", vars={"items": items})
print(error["phase"]) # runtime
items.append(3) # Original list remains writable: [1, 2, 3].
```

Expressions see only `vars` and these standard computation names:

```text
None True False abs any all bool bytes chr dict dir enumerate fail float
getattr hasattr hash int len list max min ord range repr reversed set
sorted str tuple type zip
```

Caller locals, other helpers (`sum`, `sleep`, `exit`) and modules such as
`runtime` are not inherited. `print()` is disabled.

### Returns and errors

| Outcome | `eval` | `try_eval` |
| --- | --- | --- |
| Success | Frozen native value | `(value, None)` |
| Successful expression `None` | `None` | `(None, None)` |
| Argument, input or evaluation failure | Raises an error | `(None, error_info)` |
| Caller cancellation or total step limit reached | Raises an execution error | Raises an execution error |

Test `error != None` for failure; valid results include `None`, `False`, `0` and
empty containers. Cancellation and cumulative steps remain in force.
`try_eval` does not catch caller panics or errors computing its arguments:
`try_eval("1 // 0")` catches division by zero; `try_eval(1 // 0)` does not.

### `error_info` structure

`error_info` is a **frozen dict** with six fields. Use `error["phase"]` to read
fields; nested dicts and frames are frozen too.

| Field | Type | Meaning |
| --- | --- | --- |
| `phase` | `string` | Failing stage, listed below. |
| `kind` | `string` | Same as `phase`, except `policy` uses `"capability"`. |
| `message` | `string` | Human-readable diagnostic, at most 4096 bytes. Do not match its exact wording in program logic. |
| `position` | `None` or `dict` | Best available source location: `{"file": string, "line": int, "column": int}`. Lines and Unicode character (rune) columns are 1-based; 0 means unknown. |
| `frames` | `tuple` of `dict` | Frames from outermost to innermost, retaining the most recent 32. Each has the location fields above plus `name: string`. Builtin frames can use `"<builtin>"`, 0, 0. |
| `truncated` | `bool` | Whether message/location/frame text or the frame count was truncated. File names are capped at 512 bytes, frame names at 256. |

| `phase` | Typical cause |
| --- | --- |
| `arguments` | Missing/extra arguments, duplicate or unknown keywords. |
| `input` | Wrong parameter type, invalid variable name or unsupported/oversized input data. |
| `source` | Source length or AST complexity limit exceeded. |
| `parse` | Invalid expression syntax. |
| `resolve` | Undefined name, such as an omitted variable or caller-only helper. |
| `policy` | Denied builtin, such as `print`; `kind` is `"capability"`. |
| `runtime` | Division by zero, invalid operation or attempted input mutation. |
| `output` | Result contains a callable/unsupported value or exceeds data limits. |

For `try_eval("price")` with no variables:

```python
{
"phase": "resolve",
"kind": "resolve",
"message": "eval: resolve: <eval>:1:1: undefined: price",
"position": {"file": "<eval>", "line": 1, "column": 1},
"frames": (),
"truncated": False,
}
```

Argument/input/source errors usually have `position=None` and `frames=()`.
Runtime/policy errors can include caller frames; `position` uses the innermost
available non-builtin location. Cancellation/budget errors are raised, never
returned with `phase="execution"`.

### Loading and limits

Preload `go_idiomatic` for bare calls, or enable lazy loading for `load(...)`.
`NewDefault()` enables neither. Name precedence is host globals < preloads <
per-run extras < explicit `load`. Avoid clashes with an alias:
`load("go_idiomatic", evaluate="eval")`.

| Limit | Value |
| --- | --- |
| Source | 4096 bytes; 512 non-whitespace bytes, including literals/comments |
| AST | Depth 64; 512 nodes |
| Each copied input/output graph | Depth 64; 4096 visited edges, including repeated references |
| Each graph's scalar/name payload | 1 MiB |
| Integer magnitude | 4096 bits |

Only ASCII space, tab, CR and LF are excluded from the source count. Put large
data in `vars`. These quotas do not cap native intermediate allocations, total
CPU or process memory; configure caller execution limits for your workload.

### Go callers

The Go helper returns `(starlark.Value, error)`, not an `error_info` dict:

```go
import (
"github.com/1set/starlet/lib/goidiomatic"
"go.starlark.net/starlark"
)

func evaluateQuantity(qty int) (starlark.Value, error) {
thread := &starlark.Thread{Name: "expression"}
thread.SetMaxExecutionSteps(10000)
return goidiomatic.Eval(thread, "qty * 2", starlark.StringDict{
"qty": starlark.MakeInt(qty),
})
}
```

Own the Thread and own/synchronize input data, or deeply freeze it first.
Bridge direct Go cancellation to `Thread.Cancel`; Machine does this already.
Use `errors.As` with `*goidiomatic.Error` to inspect `Phase` and `Kind`; error
causes remain available through `errors.As`/`errors.Is`. Results are not converted
to Go/JSON. Keep upstream Universe standard at startup and never mutate it
concurrently. See [`Eval`](eval.go) for the host contract.

The module remains Log + Process and is already in Starbox Safe/Network.
Use host module policy for opt-in; hard CPU/memory isolation requires a separately
validated process boundary.

### Performance

`eval` reuses the caller's Thread and builds a private environment from explicit
variables. Every call parses and compiles the expression, copies and validates
input/output data, and checks the caller's execution limits. Reusing `vars`
does not cache the compiled expression or remove this preparation work.

Benchmarks against the same expression in a compiled Starlark function found
that this preparation has a noticeable cost for small calculations. With fixed
source and small input/output graphs, more computation can reduce the relative
cost. The total time difference is not a single constant: longer source, larger
data graphs, allocation and GC also affect it. Large-workload variation does not
establish that dynamic evaluation is faster or has no overhead.

Use direct script code for fixed calculations and hot loops; use `eval` when the
expression itself is supplied at runtime. Timings depend on CPU, OS,
architecture, Go/compiler and system load. Measure on your target system with
`BenchmarkEvalComparison` and `BenchmarkEvalWorkload`; [benchmark notes](BENCHMARKS.md)
provide the workloads, sample results and reproduction commands.

## Types

### `shared_dict`
Expand Down Expand Up @@ -277,7 +460,6 @@ pprint({"key": "value", "list": [1, 2, 3]})

## Notes / boundaries

- **Not a Python module clone.** Names echo Python builtins, but this is a Star\* utility grab-bag, not a 1:1 API mirror — there is no module-level namespace; members load individually.
- **`length` vs `len`.** `length` counts Unicode code points for strings; the builtin `len` counts UTF-8 bytes. They agree for sequences.
- **`distinct` ordering.** First-seen order is preserved for list/tuple/custom inputs; dict-key order from `.keys()` is **not** guaranteed (sort it if you need stability). Sets are returned unchanged.
- **Process effects.** `sleep` blocks (and respects context cancellation); `exit`/`quit` do not stop the goroutine themselves — they return `ErrSystemExit` for the host runner to act on, and set the thread-local `exit_code`.
Expand Down
Loading
Loading