Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Unknown builtin names make `NewWithNames` panic. For names supplied by callers,
- `Reset()` discards runtime bindings and loaded-module state, while retaining configuration and the optional compiled-program cache. It does not undo host-side effects.
- `RunScript` executes supplied bytes without the compiled-program cache. For reusable named scripts, use `SetScript` + `Run`, or `RunFile`; enable caching with `SetScriptCacheEnabled(true)` when needed.
- Route script `print(...)` with `SetPrintFunc`; `nil` uses stderr and `NoopPrintFunc` discards output. See [file execution, output, and errors](USAGE.md).
- `SetMaxPrintBytes(n)` optionally limits messages sent through that handler; `0` is unlimited. See [print byte budgets](USAGE.md#print-byte-budget).
- After loading function definitions, use `Call`, `CallWithContext`, or `CallWithTimeout` to invoke them from Go. Context cancellation is cooperative; it cannot preempt arbitrary Go callbacks.
- `SetMaxExecutionSteps` bounds VM steps; `0` means unlimited. The counter resets for each Run/Call, and file-loaded module initialization receives a separate thread allowance. This is not a wall-clock or memory limit.

Expand Down
9 changes: 9 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,15 @@ actual selection with `go list -m go.starlark.net` and review any change. An
application's `replace` directive is local to that application and does not
provide a backport to consumers of this library.

## Print output limits

`Machine.SetMaxPrintBytes` limits message bytes delivered through the Machine's
print handler, with one allowance per Run/Call (or REPL session), shared with
Machine-loaded source modules. It does not bound temporary message allocations,
loggers, direct stderr output, returned values, or trusted loader helpers that
execute on their own threads. It does not supply the external process resource
limits described above.

## Reporting

Report vulnerabilities privately through
Expand Down
18 changes: 17 additions & 1 deletion USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,14 @@ func main() {
// Output: script: hello
```

### Print byte budget

`m.SetMaxPrintBytes(64 * 1024)` limits bytes delivered through the Machine's print handler. The default `0` is unlimited. Each Run or Call gets a fresh allowance; a REPL session shares one allowance across its inputs. Machine-loaded source modules consume the caller's allowance. `Reset` preserves the configured limit.

The limit counts UTF-8 message bytes before host formatting or added newlines. A message that would exceed it is withheld in full, and Run/Call stop with the value error `PrintLimitExceededError`; use `errors.As` as shown below. An exceeded REPL allowance ends the session and reports the error.

The limit does not bound loggers, direct stderr writes such as `eprint()`, returned values, message construction, or trusted source-loader helpers using separate threads. Those channels need separate host limits. A nonzero budget adds allocation and synchronized accounting work; configuring it can allocate before the first execution. Costs depend on the compiler, machine and workload.

## Errors and cancellation

Machine execution returns Go errors. `ExecError` unwraps its cause; use `errors.As` to classify the exported **value** error types, rather than matching message text:
Expand All @@ -126,13 +134,16 @@ func describe(err error) {
var missing starlet.ModuleNotFoundError
var withheld starlet.ModuleWithheldError
var steps starlet.MaxStepsExceededError
var printLimit starlet.PrintLimitExceededError
switch {
case errors.As(err, &missing):
fmt.Println("missing:", missing.Name)
case errors.As(err, &withheld):
fmt.Println("withheld:", withheld.Name)
case errors.As(err, &steps):
fmt.Println("step limit:", steps.Limit)
case errors.As(err, &printLimit):
fmt.Println("print byte limit:", printLimit.Limit)
default:
fmt.Println(err)
}
Expand All @@ -147,15 +158,20 @@ func main() {
m.SetMaxExecutionSteps(1)
_, err = m.RunScript([]byte(`answer = [i * i for i in range(10)]`), nil)
describe(err)
m.SetMaxExecutionSteps(0)
m.SetMaxPrintBytes(2)
_, err = m.RunScript([]byte(`print("too long")`), nil)
describe(err)
}

// Output:
// withheld: json
// missing: missing_module
// step limit: 1
// print byte limit: 2
```

Use `errors.As(err, &evalErr)` with `var evalErr *starlark.EvalError` to obtain a Starlark backtrace. `SetMaxExecutionSteps(0)` removes the step limit. The budget is per Run/Call; file-loaded module initialization gets a separate allowance.
Use `errors.As(err, &evalErr)` with `var evalErr *starlark.EvalError` to obtain a Starlark backtrace. `SetMaxExecutionSteps(0)` removes the step limit. The step budget is per Run/Call; file-loaded module initialization gets a separate allowance.

`RunWithContext` uses source previously set with `SetScript`; `CallWithContext` invokes an already-defined function. An already-cancelled context fails before main-script execution; module preparation happens first:

Expand Down
30 changes: 16 additions & 14 deletions cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ import (
// with a BSD 3-clause license (see the LICENSE file in that repo).
// Original source code: https://github.com/google/starlark-go/blob/master/starlark/example_test.go#L211

// machineState keeps optional print configuration with the runtime load state.
// Runtime initialization reuses state created by SetMaxPrintBytes, keeping
// optional configuration out of the Machine value.
type machineState struct {
loads cache
maxPrintBytes uint64
}

// cache is a concurrency-safe, duplicate-suppressing,
// non-blocking cache of the doLoad function.
// See Section 9.7 of gopl.io for an explanation of this structure.
Expand All @@ -29,16 +37,10 @@ type cache struct {
execOpts *syntax.FileOptions
loadMod func(s string) (starlark.StringDict, error) // load from built-in module first
readFile func(s string) ([]byte, error) // and then from file system
// newThread builds the thread that executes a loaded module, carrying the
// Machine's execution context (print func, step budget, run context). When
// nil the load path falls back to a bare thread — a module loaded that way
// would inherit none of the Machine's guards.
newThread func(load func(*starlark.Thread, string) (starlark.StringDict, error)) *starlark.Thread
// watchCancel, when set, arms cancellation of the load thread from the run's
// context and returns a stop func to disarm it once loading finishes, so a
// module executed by load() honours the run timeout instead of running past
// it on its own thread. When nil the load thread is not cancelled by context.
watchCancel func(thread *starlark.Thread) (stop func())
// Both load-thread construction and cancellation belong to the same Machine.
// Keep one owner instead of allocating two bound method values. A standalone
// cache with no owner retains its bare-thread, uncancelled fallback.
machine *Machine
}

type entry struct {
Expand Down Expand Up @@ -144,11 +146,11 @@ func (c *cache) doLoad(cc *cycleChecker, module string, loadThread **starlark.Th
return c.get(cc, module)
}
var thread *starlark.Thread
if c.newThread != nil {
if c.machine != nil {
// inherit the Machine's print func, step budget, and run context so a
// loaded module cannot bypass them (e.g. a top-level loop escaping the
// DoS guard, or print() going to stdout instead of the print func)
thread = c.newThread(loadFn)
thread = c.machine.newLoadThread(loadFn)
} else {
thread = &starlark.Thread{
Print: func(_ *starlark.Thread, msg string) { fmt.Println(msg) },
Expand All @@ -160,8 +162,8 @@ func (c *cache) doLoad(cc *cycleChecker, module string, loadThread **starlark.Th
// Cancel this load thread when the run's context fires, so a long
// computation inside a loaded module honours the run timeout rather than
// running to completion on its own thread. Disarmed once loading finishes.
if c.watchCancel != nil {
defer c.watchCancel(thread)()
if c.machine != nil {
defer c.machine.watchLoadThread(thread)()
}

// 1: load from built-in module, the first field returns nil if not found
Expand Down
12 changes: 9 additions & 3 deletions call.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,12 @@ func (m *Machine) callInternal(ctx context.Context, name string, args []interfac

defer func() {
if r := recover(); r != nil {
if me, ok := r.(MaxStepsExceededError); ok {
err = errorStarlarkError("call", me)
} else {
switch e := r.(type) {
case MaxStepsExceededError:
err = errorStarlarkError("call", e)
case PrintLimitExceededError:
err = errorStarlarkError("call", e)
default:
err = errorStarlarkPanic("call", r)
}
}
Expand Down Expand Up @@ -77,6 +80,9 @@ func (m *Machine) callInternal(ctx context.Context, name string, args []interfac
// reset the thread, arm the step budget, and wire the context
m.thread.Uncancel()
m.applyStepBudget()
if limit := m.state.maxPrintBytes; limit != 0 {
m.installPrintBudget(limit)
}
if ctx == nil {
// plain Call: no cancellation channel, as before
m.thread.SetLocal("context", context.TODO())
Expand Down
147 changes: 147 additions & 0 deletions call_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,162 @@ import (
"context"
"errors"
"fmt"
"os"
"reflect"
"strings"
"testing"
"testing/fstest"
"time"

"github.com/1set/starlet"
"go.starlark.net/starlark"
)

// Print budget sections: per-call reset, overflow termination, and updating the
// handler/limit on a reused thread.
func TestMachine_CallPrintByteBudget(t *testing.T) {
m := starlet.NewDefault()
if _, err := m.RunScript([]byte("def emit(s):\n print(s)\n return 7"), nil); err != nil {
t.Fatal(err)
}
m.SetMaxPrintBytes(2)
delivered := 0
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { delivered += len(msg) })
for i := 0; i < 2; i++ {
if _, err := m.CallWithTimeout(time.Second, "emit", "ok"); err != nil {
t.Fatalf("call %d did not get a fresh allowance: %v", i, err)
}
}
var limitErr starlet.PrintLimitExceededError
if out, err := m.Call("emit", "too long"); !errors.As(err, &limitErr) || out != nil {
t.Fatalf("want withheld result and typed limit: out=%v err=%v", out, err)
}
if delivered != 4 {
t.Fatalf("delivered %d bytes, want 4", delivered)
}
m.SetMaxPrintBytes(0)
m.SetPrintFunc(starlet.NoopPrintFunc)
if _, err := m.CallWithContext(context.Background(), "emit", "unlimited again"); err != nil {
t.Fatalf("clearing the limit after a rejected call: %v", err)
}
}

// Call print configuration changes take effect on the next execution, including
// replacing handlers on an existing thread and clearing an exhausted allowance.
func TestMachine_CallPrintConfigurationChanges(t *testing.T) {
const script = "def emit(s):\n print(s)\n return 7"
t.Run("replace unlimited handler after Run", func(t *testing.T) {
m := starlet.NewDefault()
var original, replacement strings.Builder
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { original.WriteString(msg) })
if _, err := m.RunScript([]byte(script), nil); err != nil {
t.Fatal(err)
}
if _, err := m.Call("emit", "first"); err != nil {
t.Fatal(err)
}
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { replacement.WriteString(msg) })
if out, err := m.Call("emit", "second"); err != nil || out != int64(7) {
t.Fatalf("replacement call: out=%v err=%v", out, err)
}
if original.String() != "first" || replacement.String() != "second" {
t.Fatalf("handlers: original=%q replacement=%q", original.String(), replacement.String())
}
})
t.Run("disable limit after replacing handler", func(t *testing.T) {
m := starlet.NewDefault()
var original, replacement strings.Builder
m.SetMaxPrintBytes(2)
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { original.WriteString(msg) })
if _, err := m.RunScript([]byte(script), nil); err != nil {
t.Fatal(err)
}
if _, err := m.Call("emit", "ok"); err != nil {
t.Fatal(err)
}
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { replacement.WriteString(msg) })
if _, err := m.Call("emit", "hi"); err != nil {
t.Fatalf("limited replacement handler: %v", err)
}
var limitErr starlet.PrintLimitExceededError
if _, err := m.Call("emit", "rejected"); !errors.As(err, &limitErr) {
t.Fatalf("want rejected message before clearing allowance: %v", err)
}
m.SetMaxPrintBytes(0)
if _, err := m.Call("emit", "unlimited"); err != nil {
t.Fatalf("clearing limit: %v", err)
}
if original.String() != "ok" || replacement.String() != "hiunlimited" {
t.Fatalf("handlers: original=%q replacement=%q", original.String(), replacement.String())
}
})
t.Run("nil handler restores default stderr", func(t *testing.T) {
f, err := os.CreateTemp(t.TempDir(), "call-stderr-")
if err != nil {
t.Fatal(err)
}
previous := os.Stderr
os.Stderr = f
defer func() { os.Stderr = previous; _ = f.Close() }()
for _, limit := range []uint64{0, 2} {
m := starlet.NewDefault()
var original strings.Builder
m.SetPrintFunc(func(_ *starlark.Thread, msg string) { original.WriteString(msg) })
m.SetMaxPrintBytes(limit)
if _, err := m.RunScript([]byte(script), nil); err != nil {
t.Fatal(err)
}
if _, err := m.Call("emit", "ok"); err != nil {
t.Fatal(err)
}
m.SetPrintFunc(nil)
if limit != 0 {
m.SetMaxPrintBytes(0)
}
if _, err := m.Call("emit", "default handler"); err != nil {
t.Fatalf("limit=%d default handler: %v", limit, err)
}
if original.String() != "ok" {
t.Fatalf("limit=%d retained previous handler: %q", limit, original.String())
}
}
os.Stderr = previous
if err := f.Close(); err != nil {
t.Fatal(err)
}
got, err := os.ReadFile(f.Name())
if err != nil || string(got) != "default handler\ndefault handler\n" {
t.Fatalf("default stderr: err=%v output=%q", err, got)
}
})
}

func TestMachine_CallLoadedFunctionPrintByteBudget(t *testing.T) {
for _, cached := range []bool{false, true} {
t.Run(fmt.Sprintf("compiled cache=%v", cached), func(t *testing.T) {
m := starlet.NewDefault()
m.SetScriptCacheEnabled(cached)
m.SetPrintFunc(starlet.NoopPrintFunc)
m.SetMaxPrintBytes(3)
m.SetScript("main.star", []byte("load(\"emit.star\", \"emit\")\nexported = emit"), fstest.MapFS{
"emit.star": &fstest.MapFile{Data: []byte("print(\"abc\")\ndef emit(s):\n print(s)")},
})
for run := 0; run < 2; run++ {
if _, err := m.Run(); err != nil {
t.Fatalf("run %d: %v", run, err)
}
if _, err := m.Call("exported", "abc"); err != nil {
t.Fatalf("loaded function retained the previous allowance: %v", err)
}
var limitErr starlet.PrintLimitExceededError
if _, err := m.CallWithContext(context.Background(), "exported", "abcd"); !errors.As(err, &limitErr) {
t.Fatalf("loaded function bypassed the current allowance: %v", err)
}
}
})
}
}

func TestMachine_Call_Preconditions(t *testing.T) {
m := starlet.NewDefault()

Expand Down
11 changes: 11 additions & 0 deletions error.go
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,14 @@ type MaxStepsExceededError struct {
func (e MaxStepsExceededError) Error() string {
return fmt.Sprintf("execution exceeded the step limit (%d)", e.Limit)
}

// PrintLimitExceededError marks an execution stopped by SetMaxPrintBytes.
// Detect it with errors.As through the execution error chain.
type PrintLimitExceededError struct {
Limit uint64
}

// Error returns the error message.
func (e PrintLimitExceededError) Error() string {
return fmt.Sprintf("execution exceeded the print byte limit (%d)", e.Limit)
}
Loading
Loading