Lut generation and schema search#49
Open
srikanth-iyer wants to merge 71 commits into
Open
Conversation
…/srikanth-iyer/CANA into LUT-generation-and-schema-search
fxcosta-phd
approved these changes
Oct 21, 2024
fxcosta-phd
left a comment
fxcosta-phd
left a comment
Contributor
There was a problem hiding this comment.
Great job @srikanth-iyer, those are necessary improvements.
Would you mind adding a description/disclaimer on the README file about what is in the automata folder?
This is because "automata" is more general than "boolean_node" in the sense that it could take more states. However, this is not what the code considers. We are still in the "boolean realm", but we are interested in specific features focusing on annihilation and generation.
Generalizes the selection of the middle input for annihilation and generation schemata for all k, instead of the hardcoded k=7 Initializes default xticks in both schemata plots to prevent missing tick errors when no schemata are drawn.
…es to improve clarity
…; add GP function to bools.py and corresponding tests
- Implemented Cython functions for fast computation of input symmetry means in `cboolean_node.pyx`, enhancing performance for boolean node operations. - Added `input_symmetry_mean_fast` and `input_symmetry_mean_annigen_fast` functions to calculate mean number of permutable inputs efficiently. - Introduced `_sum_group_lengths` helper function for summing group lengths in a list of Two-Symbol schemata. - Created `__init__.py` files for `bns` and `cell_collective` datasets, providing module-level documentation for bundled datasets in CANA.
…ath)
Ports the two-symbol coverage optimization to the LUT/schema-search branch.
This branch had already Cython-ized the per-call helpers (_expand_ts_logic,
_pi_covers, _ts_covers), but computes_ts_coverage still carried the core
algorithmic redundancy: it looped over all 2**k states and re-expanded every
schema for each state. The annihilation/generation ("anni_gen") coverage path
(get_anni_gen_coverage / input_symmetry_mean_anni_gen) builds on top of regular
ts_coverage, so it benefits transparently.
Changes (cana/canalization/boolean_canalization.py):
- computes_ts_coverage: invert the loop. Expand each schema once (via the
existing expand_ts_logic_fast Cython helper), enumerate the minterms it covers
directly (new _minterms_of helper), and mark coverage. Output is byte-for-byte
identical (same keys, key order, and per-state covering lists) to the previous
implementation.
- find_two_symbols_v2: memoize results on (k, frozenset(prime_implicants)) so
nodes with identical logic are not recomputed; returns a deep copy to keep the
cache immutable to callers. Fold the same_symbols computation into one pass.
Adds tests/test_ts_coverage.py, which locks computes_ts_coverage to a reference
reimplementation of the original per-state algorithm across arities up to k=9.
Measured ~2.9x faster coverage on a k=9 node (on top of the existing Cython
helpers); full test suite (78 tests) passes, plus 6 new coverage tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoojjrUJSx3qofcdWyUwNm
fill_out_lut expands a partial LUT into a full 2**k LUT. The old version rebuilt
each wildcard-expanded state via repeated O(k) string slicing and finished with a
full sweep plus an O(2**k log 2**k) sort. It also had three inconsistent hard-coded
wildcard lists, so '2' was accepted and entered the expansion branch but was never
expanded -- leaving spurious non-binary keys (e.g. "12") and dropping coverage.
Changes:
- Rewrite fill_out_lut (cana/utils.py) with a dense integer-indexed table:
* one canonical wildcard set (includes '2', fixing the bug);
* expand wildcards with integer bit math instead of per-bit string slicing;
* emit the LUT already in binstate order, dropping the final sort;
* a private UNSET sentinel (not '?') so an explicit '?' value stays
distinguishable from an unspecified state during clash detection.
Output is byte-for-byte identical to the original for all non-'2' inputs.
Adds a complete docstring and inline comments explaining each step.
- Fix the misleading comment in automata/schema_search_tools.py: with
fill_clashes=True the LATER entry wins (the code is correct; the comment claimed
the first entry wins).
- Add tests/test_fill_out_lut.py: randomized equivalence vs a verbatim copy of the
old implementation (k=2..8, with/without fill_clashes), plus explicit tests that
'2' now expands like '-' and leaves no junk keys.
~1.4-1.9x faster across k=7..15; full test suite (106 tests) passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoojjrUJSx3qofcdWyUwNm
…ocesses Prime implicants are stored as Python sets of pattern strings, and a set of strings iterates in a different order each process because str.__hash__ is salted (PYTHONHASHSEED). This leaked into schemata output: schemata_look_up_table() and related outputs returned rows in a per-process-random order. Row content is always identical (the prime-implicant set is mathematically unique); only order permuted. Sort at every point a PI set (or set-derived list) is exposed, keeping set-typed storage (still used for internal set operations): - schemata_look_up_table(type="pi") iterates sorted(pi). - find_two_symbols_v2 sorts its prime-implicant input and its returned two-symbol list, canonicalizing _two_symbols for all consumers (schemata ts LUT, TS coverage, canalizing map, drawing). - computes_pi_coverage builds covering lists in sorted order. - drawing/schema_vis.py renders PI rows in sorted order. Content is unchanged; only ordering is now stable and reproducible. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EoojjrUJSx3qofcdWyUwNm
…ering Spawns fresh subprocesses with different PYTHONHASHSEED values and asserts byte-identical schemata_look_up_table row order for both type="pi" and type="ts" across structured and random LUTs (k=3/5/7). Subprocesses are essential: within a single process the set iteration order is already deterministic, so an in-process test cannot catch the bug. Also asserts in-process that content is preserved and each output group is emitted in sorted order. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EoojjrUJSx3qofcdWyUwNm
master maintains no changelog, so this documents how the LUT-generation-and-schema-search branch diverges from master (branch point de25baf), reconstructed from the branch history and grouped Keep-a-Changelog style (Added / Changed / Fixed / Performance). Covers partial-LUT generation, annihilation/generation analysis, schema-search tools, Cython acceleration, drawing tools, the deterministic-ordering change, and the fill_out_lut and two-symbol-coverage work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EoojjrUJSx3qofcdWyUwNm
srikanth-iyer
force-pushed
the
LUT-generation-and-schema-search
branch
from
1906917 to
88767ef
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
added schema search rules, sample automata rules, tests, input symmetry for anni_gen in BooleanNode among others.
@fcphysics if you have time, i'll be happy to go through the changes with you. Please let me know if you can review these changes. I need them on the casci repo because some of the functions will be used in the noisy cellular automata GA.
@jcrozum also adding you here. I'm assuming that you're busy settling in, but in case you have time to check the code, please let me know. I'd love your input.
And sorry for the big pull request.