Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ sequenceDiagram
SF1->>serverInvoke: return
```

Destroy methods are a special case: a capnp interface can declare a `destroy` method that is handled by `ServerDestroy` instead of `ServerCall`. Rather than dispatching through the `ServerField`/`ServerRet`/`ServerCall` chain to a C++ interface method, `ServerDestroy` calls `invokeDestroy()` on the `ProxyServer`, which resets `m_impl` and runs any registered cleanup functions, giving the client a way to synchronously destroy the wrapped object on the server side.
Destroy methods are a special case: a capnp interface can declare a `destroy` method that is handled by `ServerDestroy` instead of `ServerCall`. Rather than dispatching through the `ServerField`/`ServerRet`/`ServerCall` chain to a C++ interface method, `ServerDestroy` calls `invokeDestroy()` on the `ProxyServer`, which resets `m_impl` and runs any registered cleanup functions, giving the client a way to synchronously destroy the wrapped object on the server side. The destroy request fails if other requests on the object are running or queued, and requests sent after it fail. Clients that use Cap'n Proto directly should normally drop the capability instead, which is safe even with requests in flight: the server frees the object after they finish.

## Advanced Features

Expand Down
15 changes: 14 additions & 1 deletion include/mp/proxy-io.h
Original file line number Diff line number Diff line change
Expand Up @@ -699,10 +699,23 @@ ProxyServerBase<Interface, Impl>::~ProxyServerBase()
//! Context.thread parameter was passed, this method will run on the worker
//! thread specified by the client. Otherwise it will run on the EventLoop
//! thread, like other server methods without an assigned thread.
//!
//! Non-libmultiprocess clients can send destroy while other requests are
//! running, so it is rejected in that case.
template <typename Interface, typename Impl>
void ProxyServerBase<Interface, Impl>::invokeDestroy()
{
m_impl.reset();
// A destroy request with an mp.Context runs on a worker thread and is
// counted in m_active_calls itself.
const int self_count{std::this_thread::get_id() == m_context.loop->m_thread_id ? 0 : 1};
// Move m_impl out on the event loop thread, where serverInvoke checks it,
// and destroy it after returning to this thread.
std::shared_ptr<Impl> impl;
m_context.loop->sync([&] {
if (m_active_calls == self_count) impl = std::move(m_impl);
});
if (!impl) throw std::runtime_error("destroy called while other calls are in progress");
impl.reset();
CleanupRun(m_context.cleanup_fns);
}

Expand Down
4 changes: 4 additions & 0 deletions include/mp/proxy-types.h
Original file line number Diff line number Diff line change
Expand Up @@ -835,6 +835,10 @@ kj::Promise<void> serverInvoke(Server& server, CallContext& call_context, Fn fn)
MP_LOG(loop, Log::Trace) << "request data: "
<< LogEscape(params.toString(), server.m_context.loop->m_log_opts.max_chars);

if (!server.m_impl) {
MP_LOG(loop, Log::Error) << "IPC server error request #" << req << ", method called after destroy";
throw std::runtime_error("method called after destroy");
}
try {
using ServerContext = ServerInvokeContext<Server, CallContext>;
using ArgList = typename ProxyClientMethodTraits<typename Params::Reads>::Params;
Expand Down
6 changes: 6 additions & 0 deletions include/mp/proxy.h
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,12 @@ struct ProxyServerBase : public virtual Interface_::Server
* wrapped. */
std::shared_ptr<Impl> m_impl;
ProxyContext m_context;
//! Number of requests queued or executing on worker threads.
//! Incremented in the mp.Context PassField overload before being posted and
//! decremented when completed or dropped. Requests executing on the
//! event-loop thread are excluded because they run to completion without
//! allowing another request to run.
int m_active_calls{0};
};

//! Customizable (through template specialization) base class which ProxyServer
Expand Down
5 changes: 4 additions & 1 deletion include/mp/type-context.h
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,16 @@ auto PassField(Priority<1>, TypeList<>, ServerContext& server_context, const Fn&
auto& server = server_context.proxy_server;
EventLoop& loop = *server.m_context.loop;
int req = server_context.req;
++server.m_active_calls;
auto active_call = std::shared_ptr<void>{nullptr, [&server](void*) { --server.m_active_calls; }};
// Keep a reference to the ProxyServer instance by assigning it to the self
// variable. ProxyServer instances are reference-counted and if the client
// drops its reference and the IPC call is canceled, this variable keeps the
// instance alive until the method finishes executing. The self variable
// needs to be destroyed on the event loop thread so it is freed in a sync()
// call below.
auto self = server.thisCap();
auto invoke = [self = kj::mv(self), call_context = kj::mv(server_context.call_context), &server, &loop, req, fn, args...](CancelMonitor& cancel_monitor) mutable {
auto invoke = [self = kj::mv(self), active_call = kj::mv(active_call), call_context = kj::mv(server_context.call_context), &server, &loop, req, fn, args...](CancelMonitor& cancel_monitor) mutable {
MP_LOG(loop, Log::Debug) << "IPC server executing request #" << req;
if (loop.testing_hook_async_request_start) loop.testing_hook_async_request_start();
KJ_DEFER(if (loop.testing_hook_async_request_done) loop.testing_hook_async_request_done());
Expand Down Expand Up @@ -176,6 +178,7 @@ auto PassField(Priority<1>, TypeList<>, ServerContext& server_context, const Fn&
// scope when it happens.
cancel_monitor.m_on_cancel = nullptr;
auto self_dispose{kj::mv(self)};
auto active_call_dispose{kj::mv(active_call)};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "proxy: Reject destroy() during active calls and calls after destroy()" (0e96139)

Note: I agree with both differences from the fix suggested in #378. Counting only mp.Context requests is enough, because a request without one runs entirely inside serverInvoke on the event loop thread, results included, so a destroy can't run in the middle of it. A mutex isn't needed because the counter is only incremented, decremented and checked on the event loop thread.

Decrementing here, in the same sync() call that drops self, also does what I wanted from decrementing "when the request's promise settles": it happens after fn.invoke() has filled in the results and before the response is sent, so a client that has received the response to a call can always destroy the object afterwards.

if (erase_thread) {
// Look up the thread again without using existing
// iterator since entry may no longer be there after
Expand Down
120 changes: 120 additions & 0 deletions test/mp/test/test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -757,5 +757,125 @@ KJ_TEST("Call async IPC method without thread or pool errors correctly")
KJ_EXPECT(error_thrown);
}

KJ_TEST("Calling IPC method after destroy and destroy during a call")
{
// Regression test for bitcoin-core/libmultiprocess#378 where a
// non-libmultiprocess client could:
//
// 1. Call a method after destroy, making the server call it through a
// null m_impl pointer.
// 2. Call destroy while another call on the same object was running,
// making the server free m_impl under that call.
//
// This test builds requests directly through Cap'n Proto, mimicking the
// behavior of a buggy client.
TestSetup setup;
ProxyClient<messages::FooInterface>* foo = setup.client.get();
EventLoop& loop = *foo->m_context.loop;
foo->initThreadMap();

// Create two pool threads, so the destroy() and value() calls run on
// different threads.
std::promise<void> pool_ready;
loop.sync([&] {
auto pool_req = foo->m_context.connection->m_thread_map.makePoolRequest();
pool_req.setCount(2);
loop.m_task_set->add(pool_req.send().then([&](auto&&) { pool_ready.set_value(); }));
});
pool_ready.get_future().get();

// Create two Bar objects, one for each case below. BarInterface declares a
// destroy method, unlike FooInterface.
std::optional<messages::BarInterface::Client> bar1;
std::optional<messages::BarInterface::Client> bar2;
std::promise<void> bars_ready;
loop.sync([&] {
auto request{foo->m_client.listBarsRequest()};
request.initContext();
request.setN(2);
loop.m_task_set->add(request.send().then([&](auto&& results) {
bar1.emplace(results.getResult()[0]);
bar2.emplace(results.getResult()[1]);
bars_ready.set_value();
}));
});
bars_ready.get_future().get();

////////////////////////////////////////////////////////////////////////////
// Case 1: method called after destroy.
////////////////////////////////////////////////////////////////////////////

std::promise<void> destroyed;
loop.sync([&] {
auto request{bar1->destroyRequest()};
request.initContext();
loop.m_task_set->add(request.send().then([&](auto&&) { destroyed.set_value(); }));
});
destroyed.get_future().get();

// Calls after destroy are rejected.
std::promise<std::string> value_error;
loop.sync([&] {
auto request{bar1->valueRequest()};
request.initContext();
loop.m_task_set->add(request.send().then(
[&](auto&&) { value_error.set_value(""); },
[&](kj::Exception&& e) { value_error.set_value(e.getDescription().cStr()); }));
});
KJ_EXPECT(value_error.get_future().get().find("method called after destroy") != std::string::npos);

// Release bar1.
loop.sync([&] { bar1.reset(); });

////////////////////////////////////////////////////////////////////////////
// Case 2: destroy called while another call is running.
////////////////////////////////////////////////////////////////////////////

// Block the next async request, the value() call, before it calls m_impl.
std::promise<void> value_started;
std::promise<void> value_release;
std::atomic<bool> blocked{false};
loop.testing_hook_async_request_start = [&] {
if (blocked.exchange(true)) return;
value_started.set_value();
value_release.get_future().wait();
};
std::promise<int> value;
loop.sync([&] {
auto request{bar2->valueRequest()};
request.initContext();
loop.m_task_set->add(request.send().then([&](auto&& results) { value.set_value(results.getResult()); }));
});
value_started.get_future().get();

// Destroy fails if value() request is being executed.
std::promise<std::string> destroy_error;
loop.sync([&] {
auto request{bar2->destroyRequest()};
request.initContext();
loop.m_task_set->add(request.send().then(
[&](auto&&) { destroy_error.set_value(""); },
[&](kj::Exception&& e) { destroy_error.set_value(e.getDescription().cStr()); }));
});
KJ_EXPECT(destroy_error.get_future().get().find("destroy called while other calls are in progress") != std::string::npos);

value_release.set_value();
KJ_EXPECT(value.get_future().get() == 1);
Comment thread
ryanofsky marked this conversation as resolved.
Resolved

// Destroy succeeds after the value() call has finished.
std::promise<std::string> destroy_result;
loop.sync([&] {
auto request{bar2->destroyRequest()};
request.initContext();
loop.m_task_set->add(request.send().then(
[&](auto&&) { destroy_result.set_value(""); },
[&](kj::Exception&& e) { destroy_result.set_value(e.getDescription().cStr()); }));
});
KJ_EXPECT(destroy_result.get_future().get() == "");

// Release bar2.
loop.sync([&] { bar2.reset(); });
}

} // namespace test
} // namespace mp
Loading