Repository navigation
Allow request cancellation for wrapped C++ methods - #342
xyzconstant wants to merge 6 commits into
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline and AI policy for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
3300995 to
0986a13
Compare
|
CI failures seem unrelated |
|
Nice work! I've had a number of ideas about this feature over the years so it's really interesting to see it implemented. I will just give some quick thoughts for now so I don't get nerdsniped and wind up spending all day or more on this. Thoughts:
EDIT: Added |
|
Thanks for your feedback @ryanofsky. Sorry for the late response. I took some time to fill some C++ gaps (mostly to understand whether client-side detection of the
This makes complete sense. I believe I now grasp the original idea. This is cleaner for consumers and non-libmultiprocess clients, which don't need a
Yes, you're correct here as well.
This is interesting. I think we could combine this with
I prefer this over the dedicated hooks. However, I don't fully understand how it would handle the client side. For So I think the hook can tell
Yes, definitely we can cover both sides in this PR. I need to think about this more though, together with the mpgen issue above.
Ok perfect. I'll drop it.
Thanks, I really thought it wasn't caused by this PR, but I guess I had it wrong. I have some WIP rework locally, so these errors may be different the next time I push. |
Hmm yeah. Actually when I wrote this I wasn't thinking about the need to unregister, and the
You're right, I wasn't thinking about the client side of this very clearly. I was thinking as the client code was processing parameters it could use So I think a capnproto method annotation of some kind might be necessary. I was thinking of adding annotations anyway to support more flexible mapping of c++ parameters to capnproto parameters like: in context of #282. But something less general could also work well here. |
The vector of callbacks exists because another callback (pre-existing) is registered in
So during a cancellable request, there are two registrations: one internal, and another the wrapped method itself registers. But I do agree it looks a bit awkward. The raw pointers are there so
Good, this settles the client side then. I'd like to start with something minimal that we can scale up later into something more general. I was thinking of a method annotation (your waitValue @7 (context :Proxy.Context, timeoutMs :Int32) -> (result :Int32) $Proxy.extraParam("cancel"); |
4c8b3db to
4cfc251
Compare
4821d99 to
4731f5d
Compare
4731f5d to
f6944b2
Compare
|
@ryanofsky Just addressed your comments, rebased with master, and pushed. Added 9422b97, which supports the Updated the description too. Thanks for your feedback :) |
|
Thanks for the updates! And approach ACK f6944b2. I think it's good to add client and server cancellation support together in the same PR. Quickly skimming changes, the It would also be great to see this put to use in bitcoin core by dropping the |
d2014d9 to
22000b4
Compare
|
FWIW: I just pushed changes to bitcoin/bitcoin#36097 to sync with this PR's current state. I also updated the description here to reference it. |
89bc79c to
aa15445
Compare
aa15445 to
5deb80e
Compare
5deb80e to
d8bf70b
Compare
d8bf70b to
f1e28d0
Compare
Add a `$Proxy.extraParam` method annotation that declares an extra C++-only parameter in the generated method signature. The parameter has no corresponding capnp parameter and is not serialized or sent over RPC. The annotation value names the parameter in generated C++ code. Client behavior: - A matching `CustomBuildExtraParam(TypeList<T>, ClientInvokeContext&, T&&)` overload MUST be implemented. The parameter is passed to it before the RPC message is dispatched. Server behavior: - A matching `CustomReadExtraParam(TypeList<T>, ServerContext&)` overload MUST be implemented. No data arrives for this parameter so this overload reconstructs the parameter value on the server side. The test checks the values the client passes reach the client overload and the ones the server reconstructs arrive instead.
…`request_mutex` The mutex guards the request's params and results structs, not the cancellation itself. The old names would be confusing next to the CancelState class added in the following commits. Pure rename, no behavior change.
Add two optional hooks to `ClientInvokeContext` that client overloads can set: - `set_canceler` receives the `kj::Canceler` object wrapping the request promise once it is sent, and null when the request finishes and the canceler is destroyed. `clientInvoke` only creates the canceler (heap-allocated and attached to the promise) if the hook is set. - `handle_error` receives the `kj::Exception` of a failed request and lets clients throw a custom exception, which `clientInvoke` rethrows to the caller. For example, `kj::Canceler::cancel` raises `DISCONNECTED`, the same error a disconnect produces, so only the parameter that canceled can tell them apart.
Add two members to `ServerInvokeContext` that server overloads can use: - `request_mutex` is the mutex `request_lock` refers to, set together with it by the `Proxy.Context` `PassField` overload, and null for methods running on the event loop thread. Overloads lock it to touch `cancel_fn`. - `cancel_fn` holds a callback that the cancel monitor runs on the event loop thread, when the request is canceled while the method executes. `ServerCall` clears it after the method returns.
The `$Cxx.allowCancellation` annotation used by the cancellation tests does not exist in older versions. Remove the configure-time checks that only covered them, and move the olddeps CI config to 1.0.0.
- One test cancels an in-flight `ProxyClient` call from another thread. - Another drops the response promise mid-execution, imitating non-libmultiprocess clients. Additionally, this commit adds a reference implementation of a cancel parameter and its corresponding `CustomBuildExtraParam`/`CustomReadExtraParam` overloads that applications could replicate to fully support cancellations.
f1e28d0 to
7d1decd
Compare
Currently, libmultiprocess requests are blocking, with no way to cancel them from either the client or server side. Downstream code works around this by pairing each blocking method with a dedicated
interruptX()method whose only purpose is to wake it up. For example, bitcoin/bitcoin#33676 introducedBlockTemplate::interruptWait()specifically to wake an in-progressBlockTemplate::waitNext()call.Cap'n Proto provides a useful cancellation mechanism that libmultiprocess can use to support cancellation from both non-libmultiprocess and libmultiprocess clients. When a promise is dropped, it sends a cancellation request. The server may either cancel immediately or ignore the signal.
This PR implements approach 4 from bitcoin/bitcoin#33575, adding cancellation support on both sides:
ProxyClientmethod calls to be canceled from another thread.ClientInvokeContextprovides two hooks an application overload can set:set_canceler, which receives thekj::CancelerthatclientInvokewraps the request promise in, andhandle_error, which receives thekj::Exceptionfrom a failed request and decides what the caller sees. The second hook is how a cancel parameter turns theDISCONNECTEDthat akj::Cancelerraises intoInterruptException.The PR also adds a new capnp annotation,
$Proxy.extraParam, which declares C++-only parameters that are not sent through RPC. These parameters are handled byCustomBuildExtraParamon the client side andCustomReadExtraParamon the server side. Paired with the cancellation mechanism, this feature lets applications define their own cancel parameter types. A reference implementation lives undertest/mp/test/foo-types.handtest/mp/test/foo.h, used in tests.For example, given the following capnp schema method:
and this C++ method with an application-defined cancel parameter:
The caller (client) passes a callback that receives the function that cancels the call:
The callee (server) registers a callback that interrupts its wait and retains the returned guard until the wait ends:
Detecting a dropped promise on the server requires the
$Cxx.allowCancellationannotation on the method, file, or interface. Without it, Cap'n Proto runs the abandoned call to completion. The annotation requires Cap'n Proto 1.0 (see the "Breaking change" section in https://capnproto.org/news/2023-07-28-capnproto-1.0.html), which this PR also sets as the minimum supported version.NOTE: bitcoin/bitcoin#36097 builds on this PR by adding cancel arguments to the blocking mining methods (
waitTipChanged,createNewBlock,BlockTemplate::waitNext) and deprecating theinterrupt()/interruptWait()workarounds.