Skip to content

fix: keep checkpoint lifecycle work off the event loop - #511

Open
jinliyl wants to merge 1 commit into
agentscope-ai:mainfrom
jinliyl:fix/nonblocking-checkpoint-lifecycle
Open

fix: keep checkpoint lifecycle work off the event loop#511
jinliyl wants to merge 1 commit into
agentscope-ai:mainfrom
jinliyl:fix/nonblocking-checkpoint-lifecycle

Conversation

@jinliyl

@jinliyl jinliyl commented Aug 31, 2026

Copy link
Copy Markdown
Member

Summary

  • move complete compressed checkpoint load/dump work for local stores, catalogs, and local/NetworkX graphs into worker threads
  • move BM25 snapshot creation and FAISS/Zvec sidecar lifecycle work off the event loop while preserving atomic publication and cancellation behavior
  • coordinate worker snapshots with component state locks and a dedicated checkpoint generation so embedding backfill cannot produce mixed-generation checkpoints
  • shut down the optional application thread pool without synchronously blocking the event loop
  • add real embedded ReMe.start() / ReMe.close() heartbeat coverage in addition to private-helper tests

Verification

  • pytest tests/unit/test_embedded_consumer_compat.py tests/unit/test_file_catalog.py tests/unit/test_file_graph.py tests/unit/test_keyword_index.py tests/unit/test_file_store_consistency.py -q (175 passed)
  • pytest tests/unit -q --deselect tests/unit/test_logging_config.py::test_stdlib_forwards_screen_and_file_logs_to_qwenpaw (1145 passed, 1 deselected)
  • pre-commit checks on every changed file: AST, whitespace, Black, Flake8, Pylint, and Pyroma passed
  • controlled 250 ms lifecycle probe improved from zero heartbeat ticks during both operations to 47 ticks during start() and 46 ticks during close()

Test environment note

The deselected logging test fails independently under the current Python 3.13 environment because nested stdlib logger forwarding is suppressed; this change does not touch logging code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant