Skip to content

Measure the correlation between an opening and its neighbouring shut times - #8

Merged
remislp merged 2 commits into
masterfrom
ekdist-adjacent
Sep 8, 2026
Merged

remislp merged 2 commits into
masterfrom
ekdist-adjacent

Conversation

@remislp

@remislp remislp commented Sep 8, 2026

Copy link
Copy Markdown
Member

What this adds

ekdist.adjacent measures, from a record, what scalcs.sccurves and scalcs.scalcslib give in theory, so one can be superimposed on the other. It is what Figs 6D and 6E of Colquhoun, Hatton & Hawkes (2003) display, and what a fit to a single-channel record is checked against once the rate constants are in hand.

pairs puts each opening beside a neighbouring shut time
conditional_mean_open_time mean apparent open time for openings whose adjacent shut time falls in each of a list of ranges — the diamonds of Fig. 6D
dependency the observed dependency of Magleby & Song (1992) on a log-log grid — Fig. 6E
log_edges grid edges, per decade

Which neighbour, and why it is an argument

Every function takes side='next', 'previous' or 'adjacent', because the theory distinguishes them: scalcs returns the mean open time given the previous gap and given the next gap as two separate curves, and its HJC_dependency is the joint density of an open time and the shut time that follows it.

Passing the wrong one raises nothing. It quietly compares a measurement against the wrong curve, which is why it is a required-by-default parameter with a documented meaning rather than a convention buried in the code.

One bug found, with a regression test

The dependency's marginals are taken over every pair, not over the pairs that fall inside the grid.

Using the row and column sums of the joint histogram renormalises to whatever rectangle the grid happens to cover, and a shut-time grid never spans the distribution — the gaps between activations run to seconds. In a record of 200 000 intervals only 7 316 of 49 236 pairs were inside the grid, and the mistake turned a true dependency of zero into values of two and three.

test_marginals_are_taken_over_every_pair_not_only_the_grid pins it.

Tests

15, on constructions whose answer is known by hand or by symmetry rather than by comparison with another library — independent times must give zero dependency, correlated ones a positive diagonal, and the conditional means are checked against sums done by hand.

Agreement with the Q-matrix theory is checked separately, where a record can be simulated from a mechanism: on 150 000 intervals of the CHH 2003 AChR mechanism the observed dependency agrees over 21 cells with a mean difference of −0.004, against a theory spanning −0.90 to +2.58. That check needs scalcs and so lives outside this repository; nothing here imports it, and the module needs only numpy.

Also: the clean-install job was not doing what it says

Separate commit. The job is titled "Import every module with nothing else installed" and exists because setup.py has to name everything the package imports — scipy was missing from install_requires for a while and only a clean environment could reveal it.

But it imported a hand-written list, and the list had already fallen behind: ekdist.adjacent is not in it. So the job that catches an undeclared dependency would have missed one in any module added since the list was written.

pkgutil.iter_modules now finds whatever is actually installed. It reports 7 where the list had 6.

Provenance

Written on 2 September during the reproduction of CHH 2003, committed to a local master and never pushed. This is that work rebased onto origin/master, plus the CI fix.

🤖 Generated with Claude Code

remislp and others added 2 commits September 8, 2026 15:39
…times

ekdist.adjacent measures from a record what scalcs.sccurves and
scalcs.scalcslib give in theory, so that one can be superimposed on the other.
It is what Figs 6D and 6E of Colquhoun, Hatton & Hawkes (2003) J Physiol
547:699-728 display, and what a fit to a single-channel record is checked
against once the rate constants are in hand.

pairs() puts each opening beside a neighbouring shut time,
conditional_mean_open_time() gives the mean apparent open time for openings
whose adjacent shut time falls in each of a list of ranges, and dependency()
gives the observed dependency of Magleby & Song (1992) on a log-log grid.

Every function takes side='next', 'previous' or 'adjacent', because the theory
distinguishes them: scalcs returns the mean open time given the previous gap and
given the next gap as two separate curves, and its HJC_dependency is the joint
density of an open time and the shut time that FOLLOWS it. Passing the wrong one
raises nothing; it quietly compares a measurement with the wrong curve.

The dependency's marginals are taken over every pair, not over the pairs that
fall inside the grid. Using the row and column sums of the joint histogram
renormalises to whatever rectangle the grid covers, and a shut-time grid never
spans the distribution, because the gaps between activations run to seconds. In
a record of 200000 intervals only 7316 of 49236 pairs were inside the grid, and
the mistake turned a dependency of zero into values of two and three. There is a
regression test for it.

15 tests, on constructions whose answer is known by hand or by symmetry rather
than by comparison with another library: independent times must give zero
dependency, correlated ones a positive diagonal, and the conditional means are
checked against sums done by hand. Agreement with the Q-matrix theory is checked
separately, where a record can be simulated from a mechanism: on 150000
intervals of the CHH 2003 AChR mechanism the observed dependency agrees over 21
cells with a mean difference of -0.004, against a theory spanning -0.90 to
+2.58.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The job is titled "Import every module with nothing else installed" and exists
because setup.py has to name everything the package imports -- scipy was
missing from install_requires for a while and only a clean environment could
show it. But it imported a hand-written list, and the list had already fallen
behind: ekdist.adjacent is not in it.

So the job that catches an undeclared dependency would have missed one in any
module added since the list was written. pkgutil.iter_modules finds whatever is
actually installed; it reports 7 where the list had 6.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@remislp
remislp merged commit 6538193 into master Sep 8, 2026
6 checks passed
@remislp
remislp deleted the ekdist-adjacent branch September 8, 2026 15:07
remislp added a commit that referenced this pull request Sep 8, 2026
Fit a mechanism to many simulated records and the estimates do not always form
one cloud. Some records take the optimiser to a second maximum, typically one
where most events are missed, and the results come back as two groups an order
of magnitude apart. The mean of a set like that describes neither group.

mixture2 fits two Gaussian components in log space, for groups that overlap or
nearly do, and reports the proportion in each and their geometric means.
gap_split finds the largest multiplicative step between neighbouring sorted
values, for a few runaways cleanly separated from the body.

The point of both is that the split comes from the data. Choosing a threshold
by eye and counting either side makes the answer a property of the threshold;
these let an exclusion be stated -- "16 values above a 55-fold gap" -- and
checked by someone else. Log space because rate constants are positive with
multiplicative errors, so a group of them is log-normal rather than normal;
that also makes Mixture.means geometric means.

Written for the reproduction of Colquhoun, Hatton & Hawkes (2003), whose Fig. 2
is this situation exactly, but nothing in the module is specific to that paper
or to ion channels. numpy only.

23 tests, against data generated from a known mixture rather than against
values the function produced itself, because a mixture fit cannot look wrong:
it returns two plausible components whatever it is handed, and raises nothing.
Proportions are recovered at 10, 27 and 50 per cent so the tolerances are not
tuned to one case. Two tests keep the suite honest rather than checking
arithmetic: one records that a unimodal set is split without complaint, since
that is the mistake a user will make and the separation of the means is the
only thing that tells them; the other shows the recovery tolerances would
reject the most likely way for the fit to be broken while still looking
reasonable.

Suite goes from 75 to 98. The clean-install job needs no change -- it
enumerates modules since #8, and picks this one up as the eighth.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant