Skip to content

Write only changed token objects to storage - #257

Merged
Frauschi merged 3 commits into
wolfSSL:masterfrom
LinuxJedi:token-store-changed-objects
Oct 19, 2026
Merged

Frauschi merged 3 commits into
wolfSSL:masterfrom
LinuxJedi:token-store-changed-objects

Conversation

@LinuxJedi

@LinuxJedi LinuxJedi commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

A token object's store id is its position in the token's object list, and both WP11_Session_AddObject and WP11_Session_RemoveObjectByHandle called wp11_Slot_Store, which writes the token record and every object (two records each). WP11_STORE_BATCH only covers the built-in file store, so with a custom store every record is a separate write. With wolfBoot's TrustZone flash store each record rewrite costs several sector erase/program cycles, so every create got slower as the token filled up.

  • wp11_Token_StorePass takes a WP11_STORE_PASS_* selector: all objects (as before), the token record plus one object, or one object only. wp11_Token_Store and wp11_Slot_Store keep their full-store behavior for C_InitToken, PIN changes, re-keying and C_Finalize.
  • Create: the new object goes to the head of the list and takes the next id, so no other record changes. Only the new object and the token record are written.
  • Destroy: the head (newest) object moves into the freed list position, taking the freed id. Only that object is rewritten, or nothing when the newest object is the one destroyed. Previously every newer object shifted down an id and the whole token was rewritten. The moved object and the token record are written first. Records are removed only once that pass has written everything: the destroyed head's records, or else the moved object's records under its old id, plus the destroyed object's key record when its store type differs from the moved object's. A pass that fails therefore leaves the previous records loadable (on master the destroyed object's records were removed first, and a failed pass left the token unloadable). The destroyed object's records can then stay behind, unreferenced. On the file store the moved object and the token record are one batch. A custom store without batching can still be interrupted between those two writes.
  • C_SetAttributeValue on a token object now writes that object (WP11_Object_Persist). Until now attribute changes reached storage only at the next full rewrite or C_Finalize. If the write fails the call still succeeds, since the change is already live, and the marked token (below) has it written by the next store pass or C_Finalize.
  • A store pass that is skipped (WOLFPKCS11_NO_STORE set at run time) or fails marks the token (storeDirty), so its next store pass writes the token record and every object. The full rewrites used to repair this on the next create or destroy. A destroy also keeps the moved object's old records when its new copy was not written.
  • The storage format is unchanged: ids stay contiguous, and the key-data AAD does not include the id.

Behavior changes worth reviewing:

  • After a destroy, C_FindObjects can return objects in a different order (the moved object changes place). PKCS#11 does not define the order.
  • Key records written before policy binding (unbound) are now rebound when that object is next written, not on any create or destroy.

New test tests/token_partial_store_test.c (file store). It checks that a create leaves the other object records untouched (same inode). It copies the store directory before C_Finalize, after creates, destroys (oldest, middle, newest) and a label change, then reloads from that copy. It checks the values, the renamed label, that destroyed objects are gone, and that no records are left behind. It fails on master at the first check. A second phase destroys, renames and creates objects while WOLFPKCS11_NO_STORE is set, creates one more after unsetting it, and reloads from storage before C_Finalize. It fails without storeDirty (the rename and the moved object are lost). In debug builds a third phase uses WP11_Test_StoreCloseFailAfter to fail the store of a destroy, a rename and a CKA_ENCRYPT change on the AES key. Each reloads unchanged from a copy taken before C_Finalize, and is caught up by the next create. The destroy case fails with the previous ordering (C_Initialize returns CKR_FUNCTION_FAILED).

Testing:

  • make check passes in the default, --enable-nss and --enable-debug configs; the pre-commit hook make test passes. Merged with current master, --enable-debug make check passes too.
  • On an STM32U585 at 160 MHz running wolfBoot TZ PKCS#11 (custom store), creating 8 token objects (5 certificates, 2 EC keys, 1 AES key) went from 0.77 s for the first to 5.4 s for the eighth (24.9 s total) to a flat 0.77 s each (6.3 s total). Destroying all 8 oldest first went from 31.0 s to 9.8 s. Objects reload and verify after reboot, including after destroying every other object.
  • Not tested with WOLFPKCS11_TPM_STORE. The TPM NV reservation of the token record still runs on every pass that writes the token record.

ZD 21908

Store ids are list positions, and C_CreateObject and C_DestroyObject
rewrote every token object. With a flash backed custom store, such as
wolfBoot's, each write got slower as the token grew.

A create now writes the new object and the token record. A destroy moves
the newest object into the freed position, so at most that one object is
rewritten. C_SetAttributeValue writes the changed token object, as the
next create or destroy no longer does.
Copilot AI balanced review requested due to automatic review settings October 16, 2026 11:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Storage pauses can cause inconsistent reloads, and persistence failures leave failed attribute updates applied.

2 open findings
What changed in this PR

Reduces token-storage writes by persisting only affected objects, while retaining full writes for token-wide operations.

Changes:

  • Adds selective storage passes for object creation and deletion.
  • Persists token-object attribute changes immediately.
  • Adds regression coverage for unchanged records and reload behavior.
File Description
wolfpkcs11/​internal.h Declares the object persistence helper.
tests/​token_partial_store_test.c Tests selective writes, deletion, and reloads.
tests/​include.am Registers and links the new test.
src/​internal.c Implements selective persistence and object relocation.
src/​crypto.c Persists successful attribute updates.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/internal.c
Comment thread src/crypto.c Outdated
With only changed objects written, a store pass that was skipped, because
WOLFPKCS11_NO_STORE was set, or that failed was no longer repaired by the
next create or destroy rewriting everything. Mark the token when that
happens so its next store pass writes all objects.

C_SetAttributeValue no longer fails when its write fails. The change is
already live, and the marked token has it written by the next store pass
or C_Finalize, as happened before attribute changes were written here.

Also fix the test build with GCC format truncation checks, and test
changes made while storage is turned off.

@Frauschi Frauschi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice speed-up, and moving the head into the freed slot keeps the format unchanged. I traced the relink by hand and the moved object lands on the freed id in every case, and the full test passes locally with --enable-debug. One durability point on destroy (pre-existing, but the description says otherwise) and a few smaller things inline.

Comment thread src/internal.c
Comment thread tests/token_partial_store_test.c
Comment thread src/internal.c Outdated
Comment thread src/internal.c
Comment thread tests/token_partial_store_test.c
A destroy removed the object's records before writing the token, so a
pass that failed left the token record pointing at a missing id and the
next C_Initialize failed. Write the moved object and the token record
first, and only then remove what is no longer used: the destroyed head's
records, or the moved object's records under its old id and the
destroyed object's key record when its store type differs from the
moved object's.

Also log a failed write of a changed object in debug builds, update the
slot store doc blocks, and test destroys and attribute changes whose
store fails.
@Frauschi
Frauschi merged commit 53ae1e5 into wolfSSL:master Oct 19, 2026
96 of 97 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants