Skip to content

Stop releasing WP11_Lock from a thread that did not take it - #253

Closed
LinuxJedi wants to merge 1 commit into
wolfSSL:masterfrom
LinuxJedi:fix-wp11-lock-non-owner-unlock
Closed

LinuxJedi wants to merge 1 commit into
wolfSSL:masterfrom
LinuxJedi:fix-wp11-lock-non-owner-unlock

Conversation

@LinuxJedi

@LinuxJedi LinuxJedi commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

WP11_Lock is the reader/writer lock used for the global, slot and token state. Its read path locked the write mutex for the first reader and unlocked it for the last reader, which is often a different thread. Releasing a pthread mutex, or leaving a critical section, from a thread that does not own it is undefined behaviour, and it shows:

  • Under Wine, pkcs11mtt hangs. With the old lock it hung until a timeout in 4 of 5 runs (master; master with the test harness fixed; and current code with only the lock switched back). With this change it passed 8 of 8.
  • ThreadSanitizer cannot track the lock, so a TSAN make check reports hundreds of false races and lock-order inversions, even in single-threaded tests. This is part of why TSAN is disabled in sanitizer-tests.yml; Fix WP11_Lock ownership and remaining TSAN races, enable TSAN in CI #254 re-enables it on top of this one.

The lock is now one mutex plus a condition variable, with a reader count and a writer flag. The mutex only guards the counters and is never held while the lock is held. The semantics are unchanged: a waiting writer does not stop new readers, so a thread can take the read lock again while holding it (the old lock allowed that too, so no code path that nests read locks can deadlock).

  • pthreads (WOLFSSL_PTHREADS): pthread_mutex_t and pthread_cond_t.
  • Windows (USE_WINDOWS_API, Vista and later, not CE): CRITICAL_SECTION and CONDITION_VARIABLE.
  • Other threading APIs keep the two-mutex lock as a fallback, with a comment that it is only correct where a mutex may be released by another thread (for example a binary semaphore). SINGLE_THREADED is unchanged.

Testing:

  • macOS: pre-commit hook make test passes (debug and default configs); ASAN and TSAN make check pass.
  • Interleaved A/B timing of concurrent_destroy_object_test and pkcs11mtt (5 runs each, old vs new library, same build): no measurable difference.
  • Windows: MinGW-w64 (ucrt64) static cross-build, confirmed to use InitializeConditionVariable / SleepConditionVariableCS / WakeAllConditionVariable, run under Wine: 58 pass, 4 skip, 0 fail, pkcs11mtt 64/64. MSVC was not built locally; the win-test.yml jobs on this PR cover it.

WP11_Lock_LockRO locked the write mutex for the first reader and
WP11_Lock_UnlockRO unlocked it for the last reader, which is often a
different thread. Unlocking a pthread mutex, or leaving a critical
section, from a thread that does not own it is undefined behaviour.
Under Wine pkcs11mtt hangs because of it, and ThreadSanitizer cannot
track the lock at all.

Rebuild the lock from one mutex and a condition variable, with a
reader count and a writer flag. The mutex only guards the counters and
is not held while the lock is held. The semantics are unchanged: a
waiting writer does not block new readers, so a thread can take the
read lock again while holding it. pthreads use pthread_mutex_t and
pthread_cond_t; Windows (Vista and later) uses CRITICAL_SECTION and
CONDITION_VARIABLE. The two-mutex lock remains as a fallback for other
threading APIs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Canceling a blocked pthread waiter can leave the internal mutex locked and deadlock subsequent operations.

1 open finding
What changed in this PR

Reworks WP11_Lock to avoid cross-thread mutex releases when protecting global, slot, and token state.

Changes:

  • Adds mutex-and-condition-variable locking for pthreads and supported Windows targets.
  • Preserves nested read locking and documents the existing fallback’s limitations.
File Description
src/​internal.c Implements condition-variable locking while retaining fallback and single-threaded paths.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/internal.c
@LinuxJedi

Copy link
Copy Markdown
Member Author

Superseded by #254, which now carries this fix as its own commit ("Stop releasing WP11_Lock from a thread that did not take it"), plus a follow-up commit for the Copilot cancellation finding here ("Keep WP11_Lock acquisition free of cancellation points").

@LinuxJedi LinuxJedi closed this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants