Repository navigation
Dispatcher job — idempotent task generation, backfill-capable #2
Description
The spine. Turns active recurring duties into tasks, once per period, forever, without a lock.
Blocked-by: #1 (period engine).
Files you own
src/jobs/dispatch.job.ts(new), added todulyJobsinsrc/jobs/index.tstest/dispatch.test.ts(new)
Do not touch objectstack.config.ts.
Shape
defineJob({
name: 'duly_dispatch',
label: 'Dispatch due tasks',
schedule: { type: 'cron', expression: '0 1 * * *', timezone: 'UTC' },
handler: /* … */,
retryPolicy: /* … */,
timeout: /* … */,
})Runs hourly-or-daily in UTC and resolves each duty in its own timezone — a single UTC pass covers every zone because the question asked per duty is "is this period's task due to exist yet", not "is it midnight here".
What it does, per active duty
Select duly_duty where status = 'active', form = 'recurring', and today is inside [effective_from, effective_to] (either bound may be null).
For each, for the current period and any period whose visible_from <= today:
| Field | Value |
|---|---|
subject |
duty.name copied, so renaming the duty never rewrites history |
duty, owner |
from the duty |
business_unit |
denormalised from the duty, so a later transfer does not move historical rollups |
source |
duty.source |
period_key |
periodKeyFor(...) |
due_date |
dueDateFor(...) |
visible_from |
visibleFromFor(due_date, duty.lead_days) |
status |
open |
Then set duty.last_dispatched_period to the latest key it created.
Idempotency — the part that matters
duly_task carries a unique index duly_task_dispatch_identity on (duty, owner, period_key), scoped to the organization. Rely on it. Attempt the insert and swallow the uniqueness violation; do not read-then-write, which is a race that two overlapping runs will lose.
This is the constraint that lets the job be ordinary: re-runnable, killable mid-run, safe to invoke twice concurrently. A run that inserts nothing because everything already exists is a successful run.
Explicitly not dispatched
form: 'standing'— a standing duty never generates a task. Skip it entirely; do not create and immediately close.form: 'one_off'— dispatched by hand or by the assignment fan-out (Assignment fan-out — one piece of work becomes N independent tasks #6), never here.- duties whose
statusispausedorretired.
Backfill
Accept optional job input { from?: string; to?: string } (ISO dates). When present, generate every period in the window via periodsBetween instead of just the current one. Same insert path, so backfilling twice is a no-op.
Acceptance
- running the job twice over the same clock produces zero second-pass inserts
- a duty in
Asia/Shanghaiand one inAmerica/Los_Angelesget the correct local period on the same UTC run - standing duties produce nothing, in any run
- a paused duty produces nothing; un-pausing it and re-running produces only the current period, not the gap
last_dispatched_periodadvances and never regresses- backfill over a 13-month window produces exactly the expected key set, and a second backfill inserts nothing
Gates
pnpm validate && pnpm typecheck && pnpm test && pnpm build.
Activity
os-warren commentedon Sep 1, 2026
Claim: PM loop round 2
Session:
session_01SqkTcrxUFci7nqXdbBSe2pBranch:
claude/issue-2-dispatcher-jobWorktree:
duly-issue-2Domain: n/a (single-package app repo)
File surface:
src/jobs/,test/dispatch.test.ts(stop on breach; explain in the report)Container & model: M,
mode:subagent,model: opusClause-②: no — application metadata; no protocol contract surface.
Serial constraints cleared: round 2 is #29 / #2 / #23 / #4, file surfaces disjoint (
AGENTS.md+guard test /src/jobs//src/objects/+docs /src/actions/). Premise re-verified on the merged ref:src/functions/period.tsat82f88c9exportsperiodKeyFor,periodBounds,dueDateFor,visibleFromFor,periodsBetween,FREQUENCIES,Frequency,DueAnchor— the API this card assumes.Generated by Claude Code
os-warren commentedon Sep 1, 2026
os-dev-report
{ "issue": 2, "status": "done", "branch": "claude/issue-2-dispatcher-job", "pr": "https://github.com/objectstack-ai/duly/pull/43", "premise_still_valid": true, "summary": "The dispatch job is implemented and every acceptance criterion is proven, but it DOES NOT DISPATCH AT RUNTIME YET, loudly and by design: measured on 17.2.0, a defineJob handler is invoked with exactly { jobId, data, bundle } and no data reach, so the one metadata shape the platform offers for scheduled work cannot write a record. Rather than reach for a module-scope global silently, runDispatch(engine, ...) takes its engine as an argument, bindDispatchEngine is the one named seam, and dulyDispatch refuses loudly naming the wiring until a host calls it; the one-line onEnable wire lands in src/actions/register-handlers.ts, which issue #4 owns and is editing right now, so it is filed as duly#42 with Blocked-by: #4 instead of breaching that surface. Shipped in two files so the metadata/imperative boundary is a file boundary: src/jobs/dispatch.plan.ts is pure (every scheduling decision, no I/O, no clock, no platform import, every period spelling delegated to src/functions/period.ts) and src/jobs/dispatch.job.ts is the defineJob metadata plus ~40 lines that talk to the engine. Idempotency is a bare insert plus a failure-path read: on any insert error it asks the DATA whether the identity triple is present (present = the obligation exists once, whoever won the race; absent = re-throw so the run fails), which is driver-independent and, unlike a blanket try/catch, does not turn a missing field or a dead store into a successful run that created nothing.", "tests": "All four gates green at merged head fe7941a (origin/main merged in for #36/#38), each exit code captured before any pipe. `pnpm validate` EXIT=0 -> its own verdict line \"Validation passed (318ms)\". `pnpm typecheck` (tsc --noEmit) EXIT=0, no output. `pnpm test` EXIT=0 -> \"Test Files 7 passed (7) / Tests 243 passed (243)\"; test/dispatch.test.ts contributes 55. `pnpm build` EXIT=0 -> \"Build complete (581ms)\", and the artifact was read back to confirm the job and handler actually land: dist/objectstack.json jobs[0].name = duly_dispatch, functions keys = [dulyDispatch]. The engine-backed layer runs on sqlite (databaseDriver 'sqlite', databaseUrl ':memory:'), NOT the memory driver: measured, InMemoryDriver.create stores no constraints, so two identical duly_task inserts both succeed and the table ends with 2 rows, whereas sqlite refuses the second (SqliteError, code SQLITE_CONSTRAINT_UNIQUE, 1 row). TWO ABLATIONS, each run from a committed tree (commit 20d1b2a) with a trap '<restore>' EXIT INT TERM, and each confirmed on disk by grep-counting BOTH the removed and the injected literal before running vitest -- an editor exit code was never treated as evidence. No rebuild is involved: vitest imports src/ directly, there is no dist in the resolution path. (1) sqlite -> memory driver in the test boot: on-disk check printed \"removed-literal count 1 -> 0; injected-literal count 1\" with a non-empty git diff --stat; result 4 failed / 51 passed, the failures being 'the index is real on this engine', 'the second pass inserts nothing', 'two overlapping runs produce one task' and 'a second backfill inserts nothing' -- so the sqlite choice is load-bearing and the suite cannot go green by absence. (2) made insertOnce update the existing duly_task row on the 'existing' path: on-disk check printed \"anchor 1 -> 0; injected 'engine.update(duly_task' count 1\"; result exactly 1 failed / 54 passed, the stagnation-clock guard. Both printed RESTORED and git status was empty afterwards. Independent evidence the suite bites: it caught a real bug on its first run -- with lead_days 0 the look-ahead passed the run's instant as `from` and midnight-of-today as `to`, so periodsBetween returned [] and EVERY zero-lead duty silently dispatched nothing; fixed, with the reason in the code.", "open_questions": [ { "question": "Should duly_dispatch ship `enabled: true` while duly#42 (the onEnable engine wire) is still open? As shipped, the nightly run will throw \"Job 'duly_dispatch' has no data engine ... No tasks were dispatched.\" every night at 01:00 UTC until #42 lands, and the retry policy will make that three failures per night rather than one.", "options": [ "A. Ship enabled (what this PR does). The failure is loud, recorded in sys_job_run, and names the exact missing wire. The moment #42 lands the job works with no further change.", "B. Ship enabled:false and flip it in #42. No failing runs in the interim, but a disabled job reads identically to a working one from every surface except the logs -- and this repo's whole posture (completed_at_required_when_done) is loud refusal over silent absence.", "C. Hold this PR until #42 can land, i.e. until #4 releases src/actions/register-handlers.ts." ], "recommendation": "A, because the interim state is short and the noise is the correct signal: 'the dispatcher is not wired' is exactly what an operator needs to see, and it is self-describing. B trades a visible failure for an invisible one, which is the trade this product exists to refuse. C serialises two independent pieces of work for no correctness gain -- the implementation and its tests are complete and reviewable now." } ], "out_of_scope_findings": [ "filed as objectstack-ai/objectstack#14094: a defineJob handler is invoked with { jobId, data, bundle } and no data reach (measured probe over a real boot: RUNS 7, context keys bundle/data/jobId), so the platform's only scheduled-work metadata shape cannot read or write a record; the documented module-scope escape also does not survive the artifact path, whose runtime module exports only { functions, meta } and whose bundle carries no onEnable", "filed as objectstack-ai/objectstack#14095: an application told to rely on a unique index cannot recognise the violation -- ObjectQL rethrows the raw driver error (SQLITE_CONSTRAINT_UNIQUE here, 23505 on Postgres) and isUniqueViolationError lives in @objectstack/types, which an app cannot resolve; the declarative half is worse, since create_record collapses the engine error to a string with no code so try_catch and fault edges can only swallow everything", "filed as objectstack-ai/duly#42 (Blocked-by: #4): the one-line bindDispatchEngine(ql) call in src/actions/register-handlers.ts, split out because #4 owns that file this round" ] }Generated by Claude Code
os-warren commentedon Sep 1, 2026
ACCEPT — PM review of #43, round 2. This card found the most consequential thing in the project so far.
Gates re-run with
origin/mainmerged: all fourEXIT=0, 278 tests. CIverifysuccesson the branch head. File surface held tosrc/jobs/+src/functions/index.ts+test/dispatch.test.ts.The platform gap is confirmed from the contract, not just your probe
I went and read it rather than taking the measurement on trust —
packages/spec/src/contracts/job-service.ts:No engine, no
ql, no data reach of any kind. Your probe saw{ jobId, data, bundle }; the declared type says{ jobId, data }. Either way the conclusion holds and it is a serious one: the platform's only metadata shape for scheduled work cannot read or write a record. objectstack#14094 is the right place for it and it is the most valuable finding of the session.Refusing loudly rather than reaching for a module-scope global was the correct response.
bindDispatchEngineas the one named seam, and a handler that throws naming the exact missing wire, means the interim state is self-describing instead of a job that appears to run and does nothing.The idempotency design is better than what I specified
I told you to swallow the uniqueness violation. You found that an application cannot recognise one — ObjectQL rethrows the raw driver error and
isUniqueViolationErrorlives in@objectstack/types, which an app cannot resolve (objectstack#14095) — and built something that does not depend on error codes at all: insert first, and only on failure ask the data whether the identity triple is present.That is stronger than my instruction in both directions. One round trip on the happy path with no check-then-write window, and a real failure still fails.
I ablated the half that matters. Replacing the guard with a blanket
return 'existing'— the obvious "simplification" a future reader will reach for:Exactly one test, and it is the one standing between a dead store and a run that reports success having created nothing. That failure mode — the dispatcher silently dispatching nothing, every night, with a green job record — is the worst thing this product could do, because the absence of tasks looks identical to a quiet month.
Your sqlite-over-memory choice is load-bearing for the same reason and your ablation proves it:
InMemoryDriverstores no constraints, so on memory the idempotency suite would pass while the index providing idempotency was never consulted.And the bug your own suite caught on first run —
lead_days: 0passing the run instant asfromand midnight asto, soperiodsBetweenreturned[]and every zero-lead duty dispatched nothing — is precisely the class this card exists to prevent.Open question: A, ship enabled
Agreed, and for your reason. A disabled job reads identically to a working one from every surface except the logs; a failing one names its own missing wire in
sys_job_run. This repo already chose loud refusal over silent absence incompleted_at_required_when_done, and the same call applies here. The interim is short — #4 has just merged, so #42 is unblocked as of now.The two-file split (
dispatch.plan.tspure,dispatch.job.tsthe engine boundary) makes the metadata/imperative line a file boundary, which is the right shape under rule 9.Merging, then dispatching #42.
Generated by Claude Code