Repository navigation
build(deps): keep in-process mode off the RPC and HTTP stacks - #527
Merged
anoop-narang merged 1 commit intoOct 16, 2026
Merged
anoop-narang merged 1 commit into
anoop-narang merged 1 commit into
Conversation
liquid-cache-datafusion-local pulled in arrow-flight, tonic, axum, hyper, h2, reqwest and rustls although in-process mode never uses the network. - Gate liquid_cache_common::rpc behind a new `rpc` feature, off by default; client and server enable it. - Move ExecutionMetricsResponse, a plain serde struct and the only rpc item the in-process crates used, to liquid_cache_common::metrics. - Drop object_store's `http` feature from liquid-cache-datafusion, which does not use it. The client calls parse_url_opts and now enables `http` itself. - Give liquid-cache-common's tests tokio `macros` directly; they had relied on tonic enabling it. Closes datafusion-contrib#522
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #527 +/- ##
=======================================
Coverage ? 81.38%
=======================================
Files ? 81
Lines ? 13666
Branches ? 13666
=======================================
Hits ? 11122
Misses ? 2216
Partials ? 328 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
anoop-narang
added a commit
to hotdata-dev/liquid-cache
that referenced
this pull request
Oct 16, 2026
Sync upstream (datafusion-contrib#524–datafusion-contrib#527): adopt upstream's file-id fix
|
🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #522.
In-process mode (
liquid-cache-datafusion-local) no longer pulls in the client/server network stack. Its normal dependency graph drops from 387 to 343 crates;arrow-flight,tonic,axum,hyper,h2,reqwestandrustlsare all gone.Changes
liquid-cache-common: newrpcfeature, off by default. It gatespub mod rpcand makesarrow-flightandprostoptional.liquid-cache-datafusion-clientandliquid-cache-datafusion-serverenable it.ExecutionMetricsResponsemoves toliquid_cache_common::metrics. It is a plain serde struct and the only part ofrpcthe in-process crates used.extract_execution_metricsstays where it is, with the same signature.liquid-cache-datafusionno longer enablesobject_store'shttpfeature. Nothing in that crate uses it, and it was what pulled inreqwest/hyper/h2/rustls, so gatingrpcalone would leave those in place.liquid-cache-datafusion-clientcallsobject_store::parse_url_optsand was relying on this transitively, so it now enableshttpitself. The server, benchmark and examples already did.liquid-cache-commongetstokiowithmacrosas a dev-dependency. Its#[tokio::test]s only compiled before becausetonicenabled that feature.liquid-cache-commonwithrpc, and its README notes the feature.prostremains in the graph throughvortex; that is outside this change.Breaking changes
liquid_cache_common::rpc::ExecutionMetricsResponseis nowliquid_cache_common::metrics::ExecutionMetricsResponse. The serialized JSON is unchanged.rpcis now opt-in. Code that usesliquid_cache_common::rpcdirectly, or through thecommonre-exports ofliquid-cache/liquid-cache-datafusion/liquid-cache-datafusion-local, needsliquid-cache-common = { ..., features = ["rpc"] }.http. Users ofliquid-cache-datafusionor-localwho registeredhttp(s)://object stores and gotobject_store'shttpfeature through this crate need to enable it themselves.Testing
cargo test --workspace --exclude dev-toolsgives 236 passed, 0 failed. The test list is identical tomain(239 entries).cargo clippy --workspace --all-targets --all-features -D warnings,cargo fmt --check,cargo docandcargo shearare clean.cargo check -p <crate> --all-targetspasses for every crate. Workspace feature unification would otherwise hide a missing feature declaration.liquid-cache-datafusion-local:examples/datafusion-local.rsflow onnano_hits.parquet; the second run is served from cache andextract_execution_metricsreports correctly.cargo treeshows none of the crates listed above.liquid-cache-datafusion-clientregistershttp://andhttps://object stores.httpline removed, the same build fails at runtime withfeature for Http not enabled, so that line is required.bench_serverwithclickbench_clientreturns correct query results.https://parquet file./execution_metricsendpoint returns the same JSON keys as before.