Skip to content

[3.10] Resolve Quartz jobs inside the trigger's tenant scope - #9125

Open
sfmskywalker wants to merge 1 commit into
release/3.10.0from
cursor/quartz-tenant-job-factory-310-d1c1
Open

sfmskywalker wants to merge 1 commit into
release/3.10.0from
cursor/quartz-tenant-job-factory-310-d1c1

Conversation

@sfmskywalker

Copy link
Copy Markdown
Member

Cherry-pick of #9121 (main SHA 0cceb5a).

Fixes #9120 for 3.10.1.

This PR merges only after the 3.10.0 tag.

Purpose

Timers and cron triggers only ran against the default tenant when tenants have their own databases. Quartz resolved job constructor dependencies before any tenant was pushed, so stores and DbContexts bound to the default tenant.

Scope

  • Bug fix (behavior change)

Description

Problem

RunWorkflowJob, ResumeWorkflowJob and ExecuteBackgroundActivityJob used to push the tenant at the start of Execute. That is too late: IWorkflowStarter, stores and the DbContexts behind them are resolved by the job factory first. An Execute-time lookup (the previous fallback) is the same too-late push and silently reintroduces #9120.

Solution

TenantAwareJobFactory derives from Quartz 3.15.1's MicrosoftDependencyInjectionJobFactory and is registered once per container by JobFactoryRegistration.TryUseTenantAwareJobFactory:

  • Called from QuartzFeature, QuartzSchedulerFeature (UseQuartzScheduler()), and QuartzBackgroundActivitySchedulerFeature (same module path), plus the matching shell features.
  • Registers only when no host IJobFactory is present.
  • Quartz 3.15.1's AddQuartz default is TryAddSingleton(typeof(IJobFactory), typeof(MicrosoftDependencyInjectionJobFactory)). That default is treated as replaceable and does not count as "another factory".
  • A host IJobFactory (typed, instance, or factory delegate) is kept. Detection works when features run before or after AddQuartz, but a later UseJobFactory<T>() or UseMicrosoftDependencyInjectionJobFactory() call replaces the tenant-aware factory.

The factory:

  1. Reads TenantId from the trigger JobDataMap, falling back to the job JobDataMap.
  2. Finds the tenant.
    • Lookup succeeded and returned null: log that the job's triggers are now in ERROR and throw from NewJob.
    • Lookup threw (DB/transient): do not throw from NewJob. Return a job that defers the failure into Execute. Workflow jobs use IQuartzJobRetryScheduler; durable ExecuteBackgroundActivityJob deletes itself and rethrows via ExceptionDispatchInfo.
  3. Push that tenant, resolve the job from the base factory, and return a thin IJob wrapper that pushes the tenant again around Execute.
  4. A job with no TenantId keeps today's default-tenant behaviour.

Jobs no longer look up or push a tenant. If trigger/job data has a TenantId but the tenant in context during Execute is null or different, they log the TenantId and job key and throw. They never fall through to default-tenant stores.

Hosts that register their own IJobFactory keep it. A fail-loud guard then reports jobs that run without their tenant, instead of silently using the default tenant. Public constructors are unchanged.

Tests

  • Factory: tenant from trigger, tenant from job, trigger wins over job, missing tenant throws and names ERROR state, lookup exception defers to the retry scheduler, background-activity lookup failure deletes the durable job and does not retry, no TenantId uses default-tenant services.
  • Real in-memory scheduler: a job for tenant-b sees that tenant at construction and inside Execute.
  • Module path: UseQuartzScheduler() without UseQuartz(), real in-memory scheduler, tenant-a and tenant-b each see their own tenant at construction and in Execute.
  • Guard: jobs throw when the trigger tenant is missing from context or differs; they do not start work.
  • Host custom IJobFactory is kept (not overridden); the guard then fires loudly.
  • Registration: Quartz default factory is replaced; host typed/delegate factories are kept; features register the factory once.

Local:

  • UnitTests: 182 passed
  • ComponentTests: 7 passed

Checklist

  • The PR is focused on a single concern
  • Commit messages follow the recommended convention
  • Tests added or updated
  • No unrelated cleanup included
  • All tests pass
Open in Web Open in Cursor 

* fix(scheduling): resolve Quartz jobs inside the trigger tenant scope

Quartz constructed jobs before any tenant was pushed, so constructor
dependencies bound to the default tenant. A tenant-aware job factory now
reads TenantId from the trigger (or job) data, pushes that tenant into a
DI scope, and only then resolves the job.

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* fix(scheduling): keep tenant around Quartz Execute and restore job ctors

Address CR R1: derive TenantAwareJobFactory from
MicrosoftDependencyInjectionJobFactory, wrap the inner IJob so the
tenant is pushed again in Execute, throw from NewJob only when the
tenant is missing, defer lookup failures to IQuartzJobRetryScheduler,
restore the public job constructors for 3.10.1 binary compatibility,
and register the factory only on the Quartz feature.

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* fix(scheduling): fall back to per-job tenant push without the factory

Address CR R2: when TenantAwareJobFactory is not registered, each
workflow and background job looks up TenantId and pushes the tenant
for Execute. Lookup exceptions stay on the existing retry path.
TenantLookupFailedJob deletes durable ExecuteBackgroundActivityJob
instead of retrying, and rethrows with ExceptionDispatchInfo.

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* fix(scheduling): register tenant job factory on the scheduler path

Replace the per-job Execute tenant lookup with module-path factory
registration and a fail-loud guard. Quartz's default DI factory is
treated as replaceable; a host custom IJobFactory is kept.

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
(cherry picked from commit 0cceb5a)
@sfmskywalker sfmskywalker added area:runtime Quality loop area: runtime bug Something isn't working elsa 3 This issue is specific to Elsa 3 prio high Is on the roadmap for the near-future labels Oct 29, 2026 — with Cursor
@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: d2b3eb6e-3192-461a-81fd-11a7d49c18a0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@sfmskywalker

Copy link
Copy Markdown
Member Author

@elsa-workflows-bot review

@elsa-3-lead elsa-3-lead Bot added this to the 3.10.1 milestone Oct 29, 2026
@sfmskywalker

Copy link
Copy Markdown
Member Author

@greptileai

@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes have no outstanding issues requiring changes before merging.

Summary

Moves tenant setup ahead of Quartz job construction so job dependencies see the scheduled tenant.

  • Quartz jobs resolve their dependencies inside the scheduled tenant.
  • Scheduled jobs stop when their tenant context does not match.

Intentional behavior noted by sfmskywalker: missing tenants place triggers in ERROR; host factories are preserved, with the tenant guard rejecting jobs that lack the scheduled tenant rather than silently using default-tenant stores; background activity lookup failures delete the job and rethrow rather than retry.

No blocking issues were found in the reviewed changes. The PR is safe to merge.

Reviews (1) · Last reviewed commit: "Resolve Quartz jobs inside the trigger's..." · Reviewed by Greptile

@elsa-workflows-review-bot elsa-workflows-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsa 3 Code Review: APPROVE + HIGH @ 2878c3a

Code Review, Round 1/6

This is a clean pick of #9121 (main 0cceb5ac) onto release/3.10.0 for 3.10.1. #9121 was approved in Round 3 at 6ce7b8cf, and this PR carries exactly that change: no extra edits, no conflict resolution. Approved for 3.10.1; per the PR body it merges only after the 3.10.0 tag.

Verified

  • Same change as main. git patch-id --stable is identical for 0cceb5ac and 2878c3a9 (2c7a817f). git range-diff shows only the added (cherry picked from commit 0cceb5ac...) trailer in the message. Both touch the same 23 files (+1110 / -124).
  • Same starting point. The 23 touched files have identical content on release/3.10.0 (775cb601) and on main just before the #9121 merge (f295d8ba), and identical content on this head and on 0cceb5ac. So the picked files equal main after #9121, byte for byte.
  • Build and tests on this head. Elsa.Scheduling.Quartz builds for net8.0, net9.0 and net10.0 with no errors. Elsa.Scheduling.Quartz.UnitTests: 182/182 pass. Elsa.Scheduling.Quartz.ComponentTests: 7/7 pass.
  • Quartz version. release/3.10.0 pins Quartz.Extensions.Hosting and Quartz.Serialization.Json at 3.15.1, the same as main, so the AddQuartz / UseJobFactory<T> descriptor behaviour that JobFactoryRegistration relies on, and the MicrosoftDependencyInjectionJobFactory.InstantiateJob override, are the same here. CShells is 0.0.28 on both.
  • Feature and shell setup. The set of files that call AddQuartz, register or replace an IJobFactory, or call TryUseTenantAwareJobFactory is the same on this head and on main: QuartzFeature, QuartzSchedulerFeature and QuartzBackgroundActivitySchedulerFeature, their shell counterparts, and the unchanged EF Core store features. Under extensions/src/scheduling the only difference from main is one package reference in Elsa.Scheduling.Quartz.EFCore.SqlServer.csproj, tracked separately and not touched here.
  • No public API break. JobExecutionExtensions is internal, so dropping GetTenantAsync affects no consumers. TenantAwareJobFactory (internal sealed) and JobFactoryRegistration (internal static) are internal. The RunWorkflowJob, ResumeWorkflowJob and ExecuteBackgroundActivityJob constructors are unchanged, tenantFinder included, so they stay binary compatible.
  • PR body. It no longer tells hosts to use the internal factory, and it states that a later UseJobFactory<T>() or UseMicrosoftDependencyInjectionJobFactory() call replaces the tenant-aware factory. Both match the code.

Non-blocking (carried over from #9121, relevant for a patch release)

  • 3.10.1 release note. On a multi-tenant host whose IJobFactory is not TenantAwareJobFactory, tenant jobs now fail loudly instead of quietly running under the default tenant. That covers a host's own factory, and UseJobFactory<T>() or UseMicrosoftDependencyInjectionJobFactory() called after Elsa's Quartz features. One-shot resume and background jobs that fail this way are not retried. This is intended, but it is a behaviour change in a patch, so please add a line to the 3.10.1 notes.
  • Follow-ups (main first, then pick if wanted): a bounded timeout or cache for the synchronous tenant lookup in NewJob, one shared TenantId reader for the factory and the guard, guard tests that assert no retry and no DeleteJob for the workflow jobs, and the CS9107 capture warning in TenantAwareJobFactory.

Threads

There are no review threads on this PR. CodeRabbit skips non-default bases. Greptile's summary (5/5) raises no issues.

CI

All checks on this head are green: ubuntu-latest, prove, import-rehearsal-tests, retained-testing-helper, select-tests, GitGuardian, solution grouping and Greptile. run-affected-tests was skipped.

— Code Review (Elsa 3), via Elsa Review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:runtime Quality loop area: runtime bug Something isn't working elsa 3 This issue is specific to Elsa 3 prio high Is on the roadmap for the near-future

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant