gh-142939: difflib.get_close_matches performance

[dg-pb]

S="
from difflib import get_close_matches
possibilities = [f'word{i}' for i in range(1000)]
"

$PYEXE -m timeit -s $S 'get_close_matches("word", possibilities)'
# current - 8.0 ms
# after `ratio()` - 7.2 ms
# after `__contains__` - 6.9 ms

• Issue: difflib.get_close_matches performance #142939

[johnslavik]

The code is now simpler, nice.

[aisk]

Hi, can you provide some microbenchmarks to ensure that there is really a performance boost with this change? It doesn’t need to be overly complex or highly precise.

[dg-pb]

Hi, can you provide some microbenchmarks to ensure that there is really a performance boost with this change?

Doesn't my small benchmark cover it?

Both changes are straight forward amendments that take place in a linear loop:

1. Avoid calling function twice when possible
2. Replace one way of doing containment with another

I don't think there is anything else that would provide additional information.

[aisk]

Ah, sorry, I missed the message in the main thread.

[hauntsaninja]

Looks good, thank you! I added a news entry

[tim-one]

Have to say I don't care about the speed of get_close_matches(). As the docs say,

  See also function get_close_matches() in this module, which shows how
simple code building on SequenceMatcher can be used to do useful work.

If it were written today, it would be a mere "recipe" instead (but the docs didn't have such things way back when).

That said, the code changes are good. Does no harm 😉. But I like switching to "in" more for clarity than for speed. There's no reason I can imagine for why "in" will always be faster. Capturing a bound method object (dict.__contains__) avoids the runtime expense of finding it anew each time membership is tested. The greater speed now for in comes from avoiding the Python-level function call to invoke the membership testing via the bound method object.

Tomorrow that advantage may vanish, or ever reverse. One reason for why, in general, micro-optimizations cost more in human time than they're generally worth.