-
-
Notifications
You must be signed in to change notification settings - Fork 34.5k
gh-140557: Force alignment of empty bytearray and array.array buffers
#140559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 9 commits
6b37e66
1fea8e5
b7eed2b
dd9e50b
9d454de
de09bbe
ad966dc
b036215
2dec490
92a5fff
d3ca57f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4447,6 +4447,43 @@ def test_bytearray_release_buffer_read_flag(self): | |
| with self.assertRaises(SystemError): | ||
| obj.__buffer__(inspect.BufferFlags.WRITE) | ||
|
|
||
| @support.cpython_only | ||
| @unittest.skipIf(_testcapi is None, "requires _testcapi") | ||
| @unittest.skipIf(struct is None, "requires struct") | ||
|
jakelishman marked this conversation as resolved.
Outdated
|
||
| def test_bytearray_alignment(self): | ||
| # gh-140557: pointer alignment of buffers including empty allocation | ||
| # should be at least to `size_t`. | ||
| align = struct.calcsize("N") | ||
| cases = [ | ||
| bytearray(), | ||
| bytearray(1), | ||
| bytearray(b"0123456789abcdef"), | ||
| bytearray(16), | ||
| ] | ||
| ptrs = [_testcapi.buffer_pointer_as_int(array) for array in cases] | ||
| self.assertEqual([ptr % align for ptr in ptrs], [0]*len(ptrs)) | ||
|
|
||
| @support.cpython_only | ||
| @unittest.skipIf(_testcapi is None, "requires _testcapi") | ||
| @unittest.skipIf(struct is None, "requires struct") | ||
|
jakelishman marked this conversation as resolved.
Outdated
|
||
| def test_array_alignment(self): | ||
| # gh-140557: pointer alignment of buffers including empty allocation | ||
| # should be at least to `size_t`. | ||
| align = struct.calcsize("N") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Chatted with a Rust export on the "Rust for CPython Discord". Came to the conclusion with them that we should align Rather than There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Chiming in to confirm that we had this discussion. 👍 for aligning to 8 bytes on all platforms; it seems simpler and less surprising than trying to marginally optimize for the subset of 4-byte platforms where Per https://en.wikipedia.org/wiki/Data_structure_alignment:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh yeah thanks - I was overly focused on how the buffer inside
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Max of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't fully paged in to this (awesome!) workstream yet, but I wanted to note that I'm incredibly inspired by the unstable rust With regards to SIMD, I have two ideas:
I'm also right now doing a cleanroom implementation of zstd in rust (I can explain my concerns with ruzstd) and making use of alignment for SIMD to improve over the zstd C implementation (currently working on a ring buffer component), which I very much intend to plug into a python native module. This comment is in full support of this PR and this workstream. Please let me know if I can follow up and learn more about this! Thanks! ^_^ |
||
| cases = [array.array(fmt) for fmt in ARRAY] | ||
| # Empty arrays | ||
| self.assertEqual( | ||
| [_testcapi.buffer_pointer_as_int(case) % align for case in cases], | ||
| [0] * len(cases), | ||
| ) | ||
| for case in cases: | ||
| case.append(0) | ||
| # Allocated arrays | ||
| self.assertEqual( | ||
| [_testcapi.buffer_pointer_as_int(case) % align for case in cases], | ||
| [0] * len(cases), | ||
| ) | ||
|
|
||
| @support.cpython_only | ||
| def test_pybuffer_size_from_format(self): | ||
| # basic tests | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| :func:`bytearray` and :func:`array.array` buffers now have the same alignment | ||
|
jakelishman marked this conversation as resolved.
Outdated
|
||
| when empty as when allocated. Unaligned buffers can still be created by slicing. | ||
|
jakelishman marked this conversation as resolved.
|
||
Uh oh!
There was an error while loading. Please reload this page.