Skip to content

Fix data races and duplicate initialization of App session objects - #656

Open
HvonP wants to merge 4 commits into
masterfrom
fix-app-sessionobjects-race
Open

HvonP wants to merge 4 commits into
masterfrom
fix-app-sessionobjects-race

Conversation

@HvonP

@HvonP HvonP commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A load test crashed after ~27 minutes with:

fatal error: concurrent map read and map write
github.com/qlik-oss/gopherciser/senseobjects.(*App).GetDimension
    senseobjects/app.go:548
github.com/qlik-oss/gopherciser/session.SetHyperCube
    session/objecthandling.go:379
...
github.com/qlik-oss/gopherciser/pending.(*Handler).QueueRequest.func1

GetDimension read app.dimensions without holding app.mutex, while addDimension wrote to it under the mutex. Object layouts are handled in parallel via the pending handler, and SetHyperCube calls GetDimension for every cyclic (Grouping == "C") dimension. So two objects with uncached cyclic dimensions could hit the map at the same time.

The cached session objects (GetSheetList, GetBookmarkList, GetVariableList, GetAppsPropsList, GetStoryList, GetLoadModelList, GetDimensionList, GetFieldList, GetCurrentSelections) had two related problems. The cached locale info (GetLocaleInfo) had the first one:

  • The cache check read the field without the lock, while the setter wrote it under the lock. That's a data race.
  • Setup wasn't coordinated (pointed out in review). Two callers could both create the object. The replaced one could register its change event after the setter had already run deregistration, which leaked the engine session object and its event handler. The object was also published before its layout and properties were fetched, so a concurrent caller could get one that wasn't fully set up.

Fix

  • GetDimension reads the cache through getDimensionFromMap, which holds the mutex (same pattern as getBookmarkFromMap). Duplicate fetches of the same dimension are harmless, so no further coordination is needed there.
  • New lazySessionObject[T] (senseobjects/lazysessionobject.go) runs the whole create → fetch layout/properties → RegisterEvent sequence under a lock per object. The locale info uses it too. It publishes the object (via atomic.Pointer) only after that sequence succeeds. Concurrent callers wait and get the same object. A failed setup isn't cached, so the next caller retries.
  • If creating the engine session object succeeds but fetching its layout or properties fails, the session object is left in the engine and the next caller creates a new one. This matches the client, which never destroys these objects, and they're cleaned up when the session times out after disconnect.
  • Since a published object is never replaced, the set* methods and their deregistration paths are removed.
  • GetCurrentSelections with requestData refreshes an existing object through its registered event function outside the setup lock, as before.
  • GetAggregatedSelectionFields reads current selections without waiting, and returns "" if they haven't been set up yet, instead of dereferencing nil.
  • CurrentSelections.Layout() and Properties() now read under the object's mutex, and GetAggregatedSelectionFields works from one layout snapshot. Before, it read layout directly while UpdateLayout replaced it under the mutex. That race predates this PR.

The setup lock is held across SendRequest. This is safe: change events triggered by SendRequest are dispatched in their own goroutines and not waited on, and no setup sequence calls the same getter again.

Tests

senseobjects/app_test.go adds a minimal fake engine socket:

  • TestAppConcurrentGetDimension: reproduces the reported race at app.go:548 under -race against the original code.
  • TestAppConcurrentGetDimensionList: 20 concurrent callers must cause exactly one CreateSessionObject, all get the same object, and every object has its layout and properties. Fails against the first commit of this PR (duplicate setup).
  • TestAppGetDimensionListRetryAfterError: injects a failure in CreateSessionObject, GetLayout or GetEffectiveProperties. In each case the failed setup isn't cached, the next call succeeds, and no DestroySessionObject request is sent.
  • TestAppConcurrentAggregatedSelectionFields: runs current selections layout updates against GetAggregatedSelectionFields. It fails under -race without the CurrentSelections locking.

go test -race ./... passes, go test -race -count=10 ./senseobjects/ passes, and go vet and golangci-lint are clean for the changed packages.

Follow-ups (pre-existing, not changed here)

  • Three other senseobjects types read layout without the mutex that setLayout uses: SheetList (Layout(), GetSheetEntry), OdagToolbarNavpoint (Layout(), GetHypercubeData) and DynamicAppViewList (Layout()). The Layout() accessors are marked //TODO DECISION: wait for write lock?. GetSheetEntry is reachable from object handling in session/objecthandling.go, concurrently with sheet list change events.
  • AppPropsList.UpdateLayout iterates items without the lock, while queued addItem calls write to it.

🤖 Generated with Claude Code

GetDimension read the dimensions map without holding the app mutex while
addDimension wrote to it under the mutex. Concurrent layout handling of
objects with cyclic dimensions could then crash with "fatal error:
concurrent map read and map write".

The session object getters (GetSheetList, GetFieldList, GetDimensionList,
GetCurrentSelections, etc.) had the same pattern for their cached
pointers. Read these under the mutex as well, and bind event handlers to
the object created in the call instead of re-reading the field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HvonP
HvonP requested a review from DnlLrssn October 27, 2026 16:07
HvonP and others added 2 commits October 27, 2026 17:52
Review feedback: locking each field access separately did not coordinate
initialization. Two callers could both create a session object, and the
replaced one could register its change event after the setter had
already run deregistration, leaking the object and its event handler.
The object was also published before its layout and properties were
fetched, so a concurrent caller could get a partially initialized one.

Add lazySessionObject, which runs the whole create, fetch and register
sequence under a per-object lock and publishes the object only when it
succeeds. Concurrent callers wait and share the same object; a failed
initialization is not cached and is retried by the next caller. With no
object ever replaced, the set* methods and their deregistration paths
are removed.

GetAggregatedSelectionFields returns an empty string when current
selections have not been initialized instead of dereferencing nil.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review feedback:

When creating a session object succeeded but fetching its layout or
properties failed, the object was not cached and the next caller created
a new one, leaving the first in the engine. initSessionObject runs the
steps after creation and destroys the session object if any fails.

GetAggregatedSelectionFields read currentSelections.layout directly
while UpdateLayout replaced it under the CurrentSelections mutex.
CurrentSelections.Layout and Properties now read under the mutex, and
aggregation uses a single layout snapshot. This race predates the PR.

Tests inject failures in CreateSessionObject, GetLayout and
GetEffectiveProperties and check destroy requests, and race selection
layout updates against GetAggregatedSelectionFields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HvonP

HvonP commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

For future reference:

The same unlocked layout read exists in other senseobjects types: sheet list, dimension and field list iteration, bookmark list, the ODAG nav point and the dynamic app view list. Most carry the same //TODO DECISION comment. AppPropsList.UpdateLayout also reads its items without the lock while queued addItem calls write to it. All of these predate the PR and none were changed. I listed them under "Follow-ups" in the PR description rather than widening this PR; they'd fit better as a separate change.

@HvonP HvonP changed the title Fix data races on App session objects Fix data races and duplicate initialization of App session objects Oct 27, 2026
@atluq
atluq self-requested a review October 28, 2026 08:23
Comment thread senseobjects/app.go Resolved
Comment thread senseobjects/app.go Resolved
Review feedback: the client never destroys session objects that fail
initialization, so the simulation should not either. Sending
DestroySessionObject only on server-side failures would make the load
differ from real clients exactly when the server is struggling. The
objects are cleaned up when the session times out after disconnect.

initSessionObject is removed and the getters run their requests
directly again. The CurrentSelections layout locking is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants