Repository navigation
Impossible to access PutPayload variant of HTTPRequestBody from custom HttpConnector #558
Description
Describe the bug
When trying to implement a custom HttpConnector, one needs to implement HttpService. That service needs to operate on HttpRequest, whose type of body is HttpRequestBody.
The implementation of HttpRequestBody is an enum with either a single or multiple buffers:
arrow-rs-object-store/src/client/http/body.rs
Lines 111 to 112 in 12ef9bc
But the public API of HttpRequestBody presents no way to access the PutPayload variant. There's an as_bytes public method, but that returns None for Inner::PutPayload.
To Reproduce
(maybe this is a feature request instead of a bug report?)
Expected behavior
Should be able to access all internal variants through the public API.
Additional context
I'm prototyping integrating the HttpConnector into obstore — Python bindings to object_store — so that someone can define an arbitrary Python-based HTTP client to make the requests initiated by object_store.
Activity
tustvold commentedon Dec 1, 2025
The intention is for implementations to use https://docs.rs/http-body/1.0.1/http_body/trait.Body.html
kylebarron commentedon Dec 1, 2025
The
HTTPRequestBodyis I think always known upfront and is never a stream, so it would be nice if there were a synchronous way to access the payload instead of having to go throughBodytustvold commentedon Dec 2, 2025
I think we need to keep Inner crate-private, in particular the index tracking on the PutPayload variant would be a little odd to expose.
I can see a few options:
as_payload(&self) -> Option<&PutPayload>From<HttpRequestBody> for PutPayloadimpl Iterator<Item=&Bytes> for &HttpRequestBodyHowever, these would all potentially constrain us down the line. Whilst we don't currently support streaming writes, this isn't necessarily always going to be the case. #504 #281
I'm not familiar with where pyO3 's async story has gotten, but is this a hard blocker or just a nice-to-have?
InnerfromHttpRequestBody#606 as a duplicate of this issue