Allow to disable internal string key caching for `json.loads()`

[nineteendo]

Feature or enhancement

Proposal:

Could we add an option to disable key caching like I added in jsonyx 2.2? Or should we tell people to use my library instead? We already have the option to disable checking for circular references in json.dumps() which is much more dangerous.

>>> b"abc".decode() is not "abc" # "abc" is not interned
True
>>> dcts = json.loads('[{"abc": 1}, {"abc": 2}, {"abc": 3}]')
>>> len({id(key) for dct in dcts for key in dct})
1
>>> dcts = json.loads('[{"abc": 1}, {"abc": 2}, {"abc": 3}]', cache_keys=False)
>>> len({id(key) for dct in dcts for key in dct})
3

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-131955: Allow to disable internal string key caching for json.loads() #131961

[picnixz]

What's the point of disabling key caching in general? when would this become an issue? checking for circular references is something different though and it's to prevent possible RecursionErrors.

[nineteendo]

Performance, I should benchmark again but I got around a 20% on my performance benchmarks.
That's the same reason why we allow disabling checking for circular references.

[picnixz]

Performance in terms of speed, maybe but can we lose in terms of memory? I assume that caching is done by interning the strings?

[nineteendo]

Up to 71% faster.

script

::json_cache_keys.bat
@echo off
echo 100 repeated keys
call main\python            -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 100)" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 100)" "json.loads(s)"
echo 10,000 repeated keys
call main\python            -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 10_000)" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 10_000)" "json.loads(s)"
echo 1,000,000 repeated keys
call main\python            -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 1_000_000)" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{'abc': 123}] * 1_000_000)" "json.loads(s)"

echo 100 different keys
call main\python            -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(100)])" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(100)])" "json.loads(s)"
echo 10,000 different keys
call main\python            -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(10_000)])" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(10_000)])" "json.loads(s)"
echo 1,000,000 different keys
call main\python            -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(1_000_000)])" "json.loads(s)"
call json_cache_keys\python -m timeit -s "import json; s = json.dumps([{str(i): 123} for i in range(1_000_000)])" "json.loads(s)"

100 repeated keys
10000 loops, best of 5: 27.7 usec per loop # before
10000 loops, best of 5: 25.2 usec per loop # after
# -> 1.10x faster
10,000 repeated keys
100 loops, best of 5: 2.58 msec per loop # before
100 loops, best of 5: 2.43 msec per loop # after
# -> 1.06x faster
1,000,000 repeated keys
1 loop, best of 5: 371 msec per loop # before
1 loop, best of 5: 392 msec per loop # after
# -> 1.06x SLOWER

100 different keys
10000 loops, best of 5: 29.1 usec per loop # before
10000 loops, best of 5: 24.5 usec per loop # after
# -> 1.20x faster
10,000 different keys
100 loops, best of 5: 3.03 msec per loop # before
100 loops, best of 5: 2.43 msec per loop # after
# -> 1.25x faster
1,000,000 different keys
1 loop, best of 5: 686 msec per loop # before
1 loop, best of 5: 401 msec per loop # after
# -> 1.71x faster

[picnixz]

Note: we're not using sys.intern I think. So the mechanism is a true caching my bad (I thought we were just using some interning but it appears we're using a memoization approach). Do you have a PoC for CPython and not jsonyx?

[nineteendo]

Yeah, I know. :)

Just comment out these lines:

diff --git a/Modules/_json.c b/Modules/_json.c
index cd8e697916..1381b95aac 100644
--- a/Modules/_json.c
+++ b/Modules/_json.c
@@ -710,10 +710,10 @@ _parse_object_unicode(PyScannerObject *s, PyObject *memo, PyObject *pystr, Py_ss
             key = scanstring_unicode(pystr, idx + 1, s->strict, &next_idx);
             if (key == NULL)
                 goto bail;
-            if (PyDict_SetDefaultRef(memo, key, key, &memokey) < 0) {
-                goto bail;
-            }
-            Py_SETREF(key, memokey);
+            // if (PyDict_SetDefaultRef(memo, key, key, &memokey) < 0) {
+            //     goto bail;
+            // }
+            // Py_SETREF(key, memokey);
             idx = next_idx;

             /* skip whitespace between key and : delimiter, read :, skip whitespace */

I'm currently trying to benchmark memory usage.

[nineteendo]

It uses 23% more memory.

script

#mysys.py
import sys

def _getsizeof(obj, seen=None):
    if (obj_id := id(obj)) in seen:
        return 0
    size = sys.getsizeof(obj)
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([_getsizeof(k, seen) + _getsizeof(v, seen) for k, v in obj.items()])
    elif isinstance(obj, list):
        size += sum(_getsizeof(v, seen) for v in obj)
    return size

def getsizeof(obj):
    size = _getsizeof(obj, set())
    for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
        if size < 1024:
            break
        size /= 1024
    return f"{size:.2f} {unit}"

::json_cache_keys.bat
@echo off
echo 100 repeated keys
call main\python            -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 100))))"
call json_cache_keys\python -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 100))))"
echo 10,000 repeated keys
call main\python            -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 10_000))))"
call json_cache_keys\python -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 10_000))))"
echo 1,000,000 repeated keys
call main\python            -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 1_x000_000))))"
call json_cache_keys\python -c "import json, mysys; print(mysys.getsizeof(json.loads(json.dumps([{'abc': 123}] * 1_x000_000))))"

100 repeated keys
18.94 KB # before
23.19 KB # after
# -> 1.22x more memory
10,000 repeated keys
1.84 MB # before
2.26 MB # after
# -> 1.23x more memory
1,000,000 repeated keys
183.53 MB # before
225.50 MB # after
# -> 1.23x more memory

[picnixz]

Well.. 23% is still huge IMO but maybe speed can be an convincing argument (I can see when it's useful: when you're loading lots of REST API data which either have lots of different keys (e.g., 1 result) or many objects with the same keys. Now, how does it fare in terms when having deeply nested structures?

And in terms of different keys, do we have a change in memory? we should gain something right? I'll also test tomorrow on my Linux to see if we have the same numbers (I'm assuming that you're using a PGO+LTO build but is this the case?)

[nineteendo]

Nesting shouldn't affect this, and if all keys are different memory usage will be higher during decoding (but I can't measure this easily).

I'm just using a regular build, but I expect similar numbers with an optimized build, after all we're executing less code.