fix: use random IVs for stored keys (put/get), matching the Dart SDK by cconstab · Pull Request #526 · atsign-foundation/at_python

cconstab · 2026-07-03T00:38:04Z

put/get encrypted every stored value under a static all-zero IV (the default
of aes_encrypt_from_base64/aes_decrypt_from_base64), i.e. IV reuse across all
data. This makes the Python SDK match the Dart reference at_client:

Shared keys: put always generates a random 16-byte IV and stores it as
ivNonce in the key metadata (serialized into the update command).
Self keys: also get a random IV (stored as ivNonce) — matching current Dart,
which randomizes the IV for every put in AtClientImpl._putInternal
(at_client_impl.dart:973) before dispatching to the encryptor. (Dart's per-type
SelfKeyEncryption still has a dead zero-IV branch; a port must randomize self keys
or it writes zero-IV data — which released atsdk does.) Interop-safe: get falls
back to the zero IV when ivNonce is absent.
UpdateVerbBuilder: fixed to carry iv_nonce — set_metadata/_build_metadata_str
silently dropped it, so a self-key ivNonce would never have been persisted.
Get: fetch metadata via llookup:all / lookup:all, use ivNonce when present,
else fall back to the legacy zero IV — so existing data (and Dart-written legacy
data) still decrypts.
Metadata: iv_nonce kept as raw bytes internally (decode incoming base64) so it
round-trips with generate_iv_nonce()/__str__.

Selection is purely by presence of ivNonce, matching Dart
(legacy_encryption.dart / legacy_decryption.dart: ivNonce != null ? fromBase64 : generateIVLegacy(); legacy IV = 16 zero bytes).

Tests

test/put_get_iv_test.py — network-free: AES round-trip with a random IV; legacy IV
is 16 zero bytes; Metadata iv_nonce base64<->bytes round-trip; _iv_from_fetched;
and _put_shared_key generates + persists a 16-byte IV.
Cross-SDK interop test in the lib — test/interop_test.py (with a Dart helper
under test/interop/) runs both directions against the Dart reference at_client:
- Python put shared → Dart get shared ✅
- Dart put shared → Python get shared ✅
  Guarded: skipped unless AT_INTEROP=1 + Dart on PATH, so the normal unittest
  run and fork CI are unaffected.

CI / CD note

.github/workflows/interop.yml runs this interop test on GitHub Actions: it stands up
the ephemeral-environment service container, adds the vip.ve.atsign.zone hosts entry,
installs Python + Dart, installs the SDK, onboards two atSigns via CRAM, and runs the
test with AT_INTEROP=1. It's currently workflow_dispatch (manual) so it's opt-in and
doesn't change the default pipeline.

It has been validated in a real GitHub Actions run — green in ~1m20s, with both
directions actually executing (Ran 2 tests ... OK, not skipped). Recommendation:
promote it into regular CI/CD — e.g. run on PRs that touch the crypto/atclient paths,
or on a nightly schedule — so Python↔Dart wire compatibility (IVs and beyond) stays
guaranteed rather than checked once. (Note: workflow_dispatch is only triggerable once
the workflow lands on the default branch.)

Backward compatibility: absent ivNonce ⇒ zero IV, so all pre-existing data
(Python- or Dart-written) still decrypts.

Previously put/get encrypted every stored value under a static all-zero IV (the default of aes_encrypt/decrypt_from_base64) — IV reuse across all data. Match the Dart at_client behavior: - shared keys: put always generates a random 16-byte IV, stored as ivNonce in the key metadata (serialized into the update command); - self keys: keep the legacy zero IV unless the caller set ivNonce (Dart does not auto-generate for self keys); - get: fetch metadata via llookup:all / lookup:all, use ivNonce when present, else fall back to the legacy zero IV — so existing data (and Dart-written legacy data) still decrypts; - Metadata: keep iv_nonce as raw bytes internally (decode incoming base64) so it round-trips with generate_iv_nonce()/__str__. Selection is purely by presence of ivNonce in metadata, matching Dart. Network-free unit tests in test/put_get_iv_test.py; cross-SDK interop verified separately against the Dart reference SDK.

…ilder Self keys now also get a random IV (stored as ivNonce), not just shared keys. Dart's current SelfKeyEncryption uses the zero IV for self keys, which is the same IV-reuse weakness; this is interop-safe because get falls back to the legacy zero IV when ivNonce is absent, and Dart's self decrypt honors ivNonce when present. Required fixing UpdateVerbBuilder, which silently dropped iv_nonce (set_metadata and _build_metadata_str never carried it) — so a self-key ivNonce would not have been persisted. Now round-trips. Tests: self put generates+persists a 16-byte IV; self encrypt->decrypt via ivNonce.

Move IV generation into put(), before dispatch to the per-type encryptor, matching Dart's structure (it randomizes ivNonce for every put in _putInternal). Keeps the per-type ??= backstop so direct calls stay safe. Adds a test for the put() layer.

…n CI test/interop_test.py exercises random-IV shared-key put/get both directions against the Dart reference at_client. Skipped unless AT_INTEROP=1 and Dart is on PATH, so the normal unittest run and fork CI are unaffected. Dart helper + pubspec under test/interop/. Draft manual workflow in .github/workflows/interop.yml starts the ephemeral env, onboards two atSigns, and runs it.

…gotcha Defensive rm of $HOME/.atsign/storage before the interop run (harmless on a fresh runner; avoids stale-key decrypt failures when re-running against a recreated EE).

…erge

… [skip ci]

cpswan

Code is OK, but a lot of nits where Claude hasn't been told how to be a proper Atsign developer.

cpswan · 2026-07-03T07:45:25Z

@@ -0,0 +1,85 @@
+# DRAFT — opt-in cross-SDK IV interop test (Python at_client <-> Dart at_client).


Has worked on test run, so this can go.

cpswan · 2026-07-03T07:45:34Z

@@ -0,0 +1,85 @@
+# DRAFT — opt-in cross-SDK IV interop test (Python at_client <-> Dart at_client).
+# Manual only (workflow_dispatch). Review before enabling on PRs: it starts the


cpswan · 2026-07-03T07:47:19Z

+          - 64:64
+          - 2500-2540:2500-2540
+    steps:
+      - uses: actions/checkout@v4


Action not pinned, and badly out of date.

Suggested change

- uses: actions/checkout@v4

- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0

cpswan · 2026-07-03T07:48:23Z

+      - name: Resolve the EE FQDN to localhost (matches the cert CN)
+        run: echo "127.0.0.1 vip.ve.atsign.zone" | sudo tee -a /etc/hosts
+
+      - uses: actions/setup-python@v5


Action not pinned and badly out of date.

Suggested change

- uses: actions/setup-python@v5

- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0

cpswan · 2026-07-03T07:49:12Z

+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'


Use latest stable release:

Suggested change

python-version: '3.13'

python-version: '3.14'

cpswan · 2026-07-03T07:59:40Z

@@ -0,0 +1,11 @@
+name: at_python_interop


Dependabot config needs to be updated to keep this (and the lockfile) up to date.

cpswan · 2026-07-03T08:03:25Z

@@ -0,0 +1,39 @@
+# Cross-SDK IV interop test


Markdown hasn't been linted:

./test/interop/README.md:10:1: MD013: Line length [Expected: 80, Actual: 86] (line-length) ./test/interop/README.md:13:1: MD022: Headings should be surrounded by blank lines. [Expected: 1; Actual: 0; Below] (blanks-around-headings,blanks-around-headers) ./test/interop/README.md:17:1: MD022: Headings should be surrounded by blank lines. [Expected: 1; Actual: 0; Below] (blanks-around-headings,blanks-around-headers) ./test/interop/README.md:18:1: MD013: Line length [Expected: 80, Actual: 84] (line-length) ./test/interop/README.md:33:1: MD013: Line length [Expected: 80, Actual: 87] (line-length) ./test/interop/README.md:36:1: MD022: Headings should be surrounded by blank lines. [Expected: 1; Actual: 0; Below] (blanks-around-headings,blanks-around-headers)

cpswan · 2026-07-03T08:09:05Z

+        client._put_self_key = MagicMock(return_value="ok")
+        me = AtSign("@alice")
+
+        sk = SharedKey("k", me, AtSign("@bob")); sk.set_namespace("test")


Linter:

test/put_get_iv_test.py:106:48: E702 Multiple statements on one line (semicolon) | 104 | me = AtSign("@alice") 105 | 106 | sk = SharedKey("k", me, AtSign("@bob")); sk.set_namespace("test") | ^ E702 107 | self.assertIsNone(sk.metadata.iv_nonce) 108 | client.put(sk, "v") |

cpswan · 2026-07-03T08:09:25Z

+        self.assertEqual(len(sk.metadata.iv_nonce), 16)
+        client._put_shared_key.assert_called_once()
+
+        selfk = SelfKey("k", me); selfk.set_namespace("test")


Linter:

test/put_get_iv_test.py:112:33: E702 Multiple statements on one line (semicolon) | 110 | client._put_shared_key.assert_called_once() 111 | 112 | selfk = SelfKey("k", me); selfk.set_namespace("test") | ^ E702 113 | self.assertIsNone(selfk.metadata.iv_nonce) 114 | client.put(selfk, "v") |

cpswan · 2026-07-03T08:09:51Z

+                   "metaData": {"ivNonce": base64.b64encode(iv).decode()}}
+        client.get_lookup_response = MagicMock(return_value=fetched)
+
+        k = SelfKey("selfdemo", AtSign("@alice")); k.set_namespace("test")


Linter:

test/put_get_iv_test.py:131:50: E702 Multiple statements on one line (semicolon) | 129 | client.get_lookup_response = MagicMock(return_value=fetched) 130 | 131 | k = SelfKey("selfdemo", AtSign("@alice")); k.set_namespace("test") | ^ E702 132 | self.assertEqual(client._get_self_key(k), "self value") |

Colin Constable added 7 commits July 2, 2026 16:11

test(interop): clear Dart local storage before run; note EE-recreate …

d512358

…gotcha Defensive rm of $HOME/.atsign/storage before the interop run (harmless on a fresh runner; avoids stale-key decrypt failures when re-running against a recreated EE).

ci(interop): TEMP push trigger to validate the interop workflow pre-m…

77617ff

…erge

ci(interop): revert temp push trigger, back to workflow_dispatch only…

1ddf934

… [skip ci]

cconstab changed the title ~~Fix/put get random iv~~ fix: use random IVs for stored keys (put/get), matching the Dart SDK Jul 3, 2026

cconstab requested a review from cpswan July 3, 2026 02:56

cpswan reviewed Jul 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: use random IVs for stored keys (put/get), matching the Dart SDK#526

fix: use random IVs for stored keys (put/get), matching the Dart SDK#526
cconstab wants to merge 7 commits into
trunkfrom
fix/put-get-random-iv

cconstab commented Jul 3, 2026

Uh oh!

cpswan left a comment

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

cpswan Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -0,0 +1,85 @@
		# DRAFT — opt-in cross-SDK IV interop test (Python at_client <-> Dart at_client).

		@@ -0,0 +1,85 @@
		# DRAFT — opt-in cross-SDK IV interop test (Python at_client <-> Dart at_client).
		# Manual only (workflow_dispatch). Review before enabling on PRs: it starts the

	- uses: actions/checkout@v4
	- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0

	- uses: actions/setup-python@v5
	- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0

Uh oh!

Conversation

cconstab commented Jul 3, 2026

Uh oh!

cpswan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants