Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 42 additions & 3 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,48 @@ print("new" in original, original["items"])
# Output: False [1, 2]
```

Own or synchronize mutable nested values separately. Default Starlark
formatting/printing and the Go `String`/`ToJSON` methods still require external
synchronization with mapping writers.
Prefer one owner goroutine for a shared value, with other goroutines sending it requests. Serialize both updates and output at that owner. Default Starlark formatting/printing and the Go `String`/`ToJSON` methods do not lock the mapping themselves.

If multiple goroutines need access, use one application mutex for **every access to the shared mutable graph**, including output and reads/writes through nested aliases. Hold it for an entire Machine `Run`/`Call` that can read or change the same value. The dictionary's internal locks do not make other accesses participate in this application mutex. This minimal Go example keeps the dictionary private to the synchronized operations:

```go
package main

import (
"fmt"
"log"
"sync"

"github.com/1set/starlet/dataconv"
"go.starlark.net/starlark"
)

func main() {
var gate sync.Mutex
shared := dataconv.NewSharedDict()
store := func(n int) error {
gate.Lock()
defer gate.Unlock()
return shared.SetKey(starlark.String("count"), starlark.MakeInt(n))
}
encode := func() (string, error) {
gate.Lock()
defer gate.Unlock()
return shared.ToJSON()
}
if err := store(1); err != nil {
log.Fatal(err)
}
text, err := encode()
if err != nil {
log.Fatal(err)
}
fmt.Println(text)
// Output: {"count":1}
}
```

The example contains only a native integer. Formatting and JSON encoding of Go-backed values can invoke host callbacks: while holding the application mutex, do not reenter it or wait for work that needs it. A callback at the owner goroutine must likewise not wait for that owner to handle another request. For complex host values, have the owner construct independent plain data, then encode that data after leaving the ownership or lock boundary. `to_dict()`/`CloneDict()` are shallow copies; freezing the original shared value changes that original rather than creating an isolated copy. Neither substitutes for ownership of nested mutable values.

From Go, use `text := shared.SnapshotString()` to format a snapshot of the
mapping's entries without racing with mapping updates. Formatting callbacks run
Expand Down
11 changes: 5 additions & 6 deletions dataconv/share.go
Original file line number Diff line number Diff line change
Expand Up @@ -373,14 +373,13 @@ func (s *SharedDict) CloneDict() (*starlark.Dict, error) {
return CloneDict(s.dict)
}

// ToJSON serializes the SharedDict instance into a JSON string representation.
// ToJSON serializes the SharedDict with the Starlark JSON encoder.
// Direct Go callers must synchronize it with mapping writes; the Starlark
// to_json method acquires the mapping lock through its attribute wrapper.
// This method facilitates the conversion of complex, nested data structures stored within a SharedDict into a universally recognizable format (JSON),
// making it easier to export or log the data contained within the SharedDict.
//
// It is important to note that the serialization process adheres to JSON's limitations, such as not supporting circular references. If the SharedDict contains
// circular references or types not supported by JSON (e.g., functions), `ToJSON` will return an error.
// Mutable nested values require their own ownership or synchronization.
// The encoder rejects unsupported values and cycles in native Starlark containers.
// This does not guarantee cycle detection or termination inside arbitrary host
// MarshalJSON methods, attribute callbacks or Go-backed values.
func (s *SharedDict) ToJSON() (string, error) {
return EncodeStarlarkJSON(s.dict)
}
Expand Down
74 changes: 73 additions & 1 deletion dataconv/share_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package dataconv

// SharedDict tests cover constructors/names, script methods, frozen/nil values,
// concurrent access, formatting snapshots, and JSON conversion.
// concurrent access, caller-synchronized output, snapshots, and JSON conversion.

import (
"fmt"
Expand Down Expand Up @@ -280,6 +280,78 @@ func TestSharedDict_SnapshotStringConcurrent(t *testing.T) {
workers.Wait()
}

// This verifies the caller's synchronization discipline; String and ToJSON do
// not acquire the application mutex or become safe for unsynchronized callers.
func TestSharedDict_ExternalSynchronization(t *testing.T) {
sd := NewSharedDict()
key := starlark.String("generation")
items := starlark.NewList([]starlark.Value{starlark.MakeInt(0)})
if err := sd.SetKey(key, starlark.MakeInt(0)); err != nil {
t.Fatal(err)
}
if err := sd.SetKey(starlark.String("items"), items); err != nil {
t.Fatal(err)
}

var gate sync.Mutex
generation := 0
update := func() error {
gate.Lock()
defer gate.Unlock()
generation++
value := starlark.MakeInt(generation)
if err := sd.SetKey(key, value); err != nil {
return err
}
// The alias mutation belongs to the same compound update as SetKey.
return items.SetIndex(0, value)
}
check := func() error {
gate.Lock()
defer gate.Unlock()
wantText := fmt.Sprintf("shared_dict({\"generation\": %d, \"items\": [%d]})", generation, generation)
if got := sd.String(); got != wantText {
return fmt.Errorf("String() = %q, want %q", got, wantText)
}
got, err := sd.ToJSON()
if err != nil {
return err
}
wantJSON := fmt.Sprintf("{\"generation\":%d,\"items\":[%d]}", generation, generation)
if got != wantJSON {
return fmt.Errorf("ToJSON() = %q, want %q", got, wantJSON)
}
return nil
}

const iterations = 128
start := make(chan struct{})
var workers sync.WaitGroup
workers.Add(4)
run := func(name string, operation func() error) {
defer workers.Done()
<-start
for i := 0; i < iterations; i++ {
if err := operation(); err != nil {
t.Errorf("%s iteration %d: %v", name, i, err)
return
}
}
}
for i := 0; i < 2; i++ {
go run("writer", update)
go run("reader", check)
}
close(start)
workers.Wait()
if generation != 2*iterations {
t.Errorf("generation = %d, want %d", generation, 2*iterations)
}
if err := check(); err != nil {
t.Error(err)
}
}

type sharedDictFormatValue struct {
stringFunc func() string
hashFunc func() (uint32, error)
Expand Down
2 changes: 2 additions & 0 deletions lib/goidiomatic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,8 @@ print(sd.to_json())
# Output: {"new_key":"new_value"}
```

For Go `String`/`ToJSON` and shared nested values, use one owner or a common application mutex for every access, including all writers. See the [shared-value synchronization example and callback limits](../../USAGE.md#shared-values).

## Details & examples

### `length(obj) -> int`
Expand Down
Loading