[Piergiuseppe]

tinyjobs move --from URL --to URL (or --app) carries the jobs still to run from one store to another, so changing store is a deploy and a command instead of a stop. It works between SQLite, Postgres and Redis in any direction, and back.

How it works. A job is copied to the new store under its id, then deleted from the old one, a chain sent with after= at a time, parents copied first and children deleted first. The copy is the whole record of the move: a run that stops anywhere loses nothing, and the next one finishes it.

The protocol change. The first version took jobs out with cancel, and four rounds of review each found a way it lost or doubled a job: cancel leaves a cancelled row under a live id, cancels every job waiting behind, and stops a job a worker just started. So the backend protocol has a new method, delete(job_id): it removes a job that has not started and that nothing waits behind, and returns False for any other without touching it. It is abstract on BaseBackend, so a third-party store has to add it; that is why it comes before the 1.0 freeze. enqueue of an id a store already holds now returns the job it holds on every store (SQLite raised, Redis wrote over it and counted it twice).

What it refuses. A store in memory at either end, one store under two addresses, an old store that holds no job at all (what a mistyped path or prefix looks like), and a SQLite file that does not exist.

Addresses are not printed. Stores are named by kind, the old store (sqlite), in move, in verify and in argument errors, since an address can carry a password in more places than one masking rule finds.

Checked. Full suite green locally on Python 3.12 with Redis 7 and Postgres 17 (2,781 tests); the move tests run on all seven pairs of stores; the contract suite has fourteen new tests for delete and for enqueue of a stored id. By hand on a Linux VPS: 20,000 jobs, one in ten in a chain, from SQLite to Redis in 20 s and back in 25. Nine review rounds; the last found nothing above low.

To decide. __version__ is 0.19.0.dev0 and the changelog section is 0.19.0: the protocol break rides with the release that was meant for breaks.

Docs: docs/moving-to-another-store.md, and "Moving between stores" in docs/design.md for why it copies, then deletes.

[read-the-docs-community]

Documentation build overview

📚 tinyjobs | 🛠️ Build #35910184 | 📁 Comparing 750bb19 against latest (bb42dd7)

  🔍 Preview build  

7 files changed · + 1 added · ± 6 modified

+ Added

• moving-to-another-store/index.html

± Modified

• index.html
• design/index.html
• guarantees/index.html
• operating-it/index.html
• other-backends/index.html
• status/index.html