Skip to content

Reuse the compiled provider across runs#592

Draft
dnagoda wants to merge 1 commit into
dc.batch-3-failure-handlingfrom
dc.batch-4-reuse-provider
Draft

Reuse the compiled provider across runs#592
dnagoda wants to merge 1 commit into
dc.batch-3-failure-handlingfrom
dc.batch-4-reuse-provider

Conversation

@dnagoda

@dnagoda dnagoda commented Jul 6, 2026

Copy link
Copy Markdown

📚 Stack (bottom → top)

  1. Add batch mode for JSONL function runs #595 Add batch mode for processing multiple inputs via JSONL
  2. Preserve JSON input object order #590 Preserve JSON input object order
  3. Refine batch failure handling #591 Refine batch failure handling
  4. Reuse the compiled provider across runs #592 Reuse the compiled provider across runs
  5. Compute humanized bytes lazily #593 Compute humanized bytes lazily
  6. Add minimal batch output with --batch-full-output opt-in #594 Add minimal batch output with --batch-full-output opt-in

Stacked PRs — review bottom-up.


Why

Batch mode already reuses the loaded function module, but Javy/provider
functions still recompiled the embedded standard provider for every input
row
. At scale, provider compilation dominated batch runtime.

What

  • Store the compiled provider Module on Provider and compile it once,
    when the ValidatedModule is built.
  • FunctionRunParams now carries a ValidatedModule (built once in main), so
    single and batch runs reuse the already-compiled function and provider
    modules — per-row work is just instantiation.
  • Removes the now-redundant CompiledProvider type, the parallel
    *_with_compiled_provider entry points, and the per-row
    Module::from_binary fallback.
  • Add batch coverage for a Javy v3 function to exercise the provider path.

Performance

Measured locally: 5,000 js_function_javy_plugin_v3 rows complete in
~1.2s, versus ~52s when the provider was recompiled per input row.

Testing

  • cargo test
  • cargo build --release

Batch mode reuses the loaded function module, but Javy/provider functions
still recompiled the embedded standard provider for every input row, so
provider setup dominated batch runtime.

Store the compiled provider Module on Provider and compile it once when the
ValidatedModule is built. FunctionRunParams now carries a ValidatedModule
(built once in main), so single and batch runs reuse the already-compiled
function and provider modules and per-row work is just instantiation. This
drops the separate CompiledProvider type, the parallel
*_with_compiled_provider entry points, and the per-row Module::from_binary
fallback.

Add batch coverage for a Javy v3 function to exercise the provider path.

Measured locally: 5,000 js_function_javy_plugin_v3 rows complete in ~1.2s,
versus ~52s when the provider was recompiled per input row.

Verified with:
- cargo test
- cargo build --release

Assisted-By: devx/c659e918-9568-4750-b122-e3890447348a
@dnagoda dnagoda force-pushed the dc.batch-4-reuse-provider branch from a8174bb to 9d39117 Compare July 6, 2026 23:33
@dnagoda dnagoda force-pushed the dc.batch-3-failure-handling branch from 388e7a4 to 3e431cf Compare July 6, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant