New drag and drop API by eira-fransham · Pull Request #4571 · rust-windowing/winit

eira-fransham · 2026-05-19T17:12:57Z

This PR implements a new API for drag and drop, with a DataTransfer type which abstracts over the various clipboard/drag and drop APIs across different platforms. I built this on top of #2429 in order to ensure @SludgePhD gets credit if it gets merged, although admittedly I ended up removing pretty much all of their work while I was reworking the design.

This is being built in order to help support drag-and-drop work in Slint's winit backend. As part of that work, I did extensive research on how drag-and-drop and clipboard APIs are implemented across different platforms, and wrote a (still WIP) research document that can be found here.

Some platforms (Wayland, X11) always transfer bytes with a MIME type, other platforms have a set of standardised transferrable types. However, all types that are supported cross-platform (images, RTF, HTML, plaintext, URIs/URI lists) are cleanly expressible using MIME types on all supported platforms. As part of this PR, I've written up a quick-and-dirty summary of which types are supported on different platforms (EDIT: the Windows section of that gist was based on WinRT, but I've since discovered that winit uses win32 and therefore can only send/receive file paths).

Design

The new API is inspired by the browser's DataTransfer API. The main complexity comes from supporting both the common set of capabilities (the types and traits in winit-core/src/data_transfer.rs) while also allowing a consumer to use the platform-specific APIs.

The design may look somewhat complex, and I am open to suggestions for simplifying it, but OS drag-and-drop/clipboard/etc APIs are just fundamentally complex. Unfortunately, there's going to be a lot of complexity here no matter the implementation. This article by a Wayland maintainer describes it as "arguably one of the most complicated parts of the core Wayland protocol", and from researching the design for other platforms it seems like Wayland actually has the simplest API.

In general, the API is designed around the idea that the user should be able to supply/read both cross-platform types and platform-specific types. Not all of the platform-specific types are fully implemented, although many of them are.

Current state

Both receiving and initiating a drag operation are implemented on Windows, Wayland and macOS. X11 supports receiving dropped data, but initiating a drag on X11 is not planned as part of this PR.

Example

The drag-and-drop example has been updated, so both sending and receiving drag operations is now shown off.

macOS

Sending and receiving a dragging operation:

2026-06-03.17-28-00.mov

Dragging styled text (via the HTML clipboard type) into the notes app on macOS:

2026-06-03.17-38-15.mov

Wayland

Video_2026-06-04_17-29-06.mp4

Windows

2026-06-04.18-41-50.mp4

AI disclaimer

Unsure if this is important to anyone reading this PR, but I thought I should mention just in case since I know that it's a common issue for OSS projects:

AI was NOT used for the development of the core winit API, nor the X11, Wayland, or macOS implementations, nor any other code, documentation, or comments contributed to this branch by me. It was also not used to write the PR description, nor any comments made by me. The Windows implementation was done by a colleague, so I do not know for sure if AI was used. I can ask him if the reviewer would like to know, but I can at least confirm that I've manually reviewed every line of his PR to my branch.

kchibisov · 2026-05-20T17:44:26Z

I haven't read very much into impl. details, but from what I've seen:

I think operations for fetching should be on ActiveEventLoop. You can not fetch clipboard on a window without loop on Wayland at all.
The transfer/mime stuff should work with clipboard, because Drag and Drop and system clipboard is the same. So once winit supports clipboard, it should use the similar-ish system.
We should keep in mind that on Wayland, if you get 2-3 types mime types, data that will be provided for each will be different, thus picking what to load should be up to user, and also, the loading of data from the Dnd/Clipboard is async operation, so the data should come back in terms of Event back to user. So, something like ActiveEventLoop::request_data_transfer_data(DataTransferId) -> Result<()> and a callback with data Vec<u8>, mime: Mime on ApplicationTrait should come later once we done reading the data back. Any blocking operation won't work, unfortunately.

In the current API, we have this URI list preloaded, but I think this is solved, we should only load with async-ish API once user asks, I think, if you want to follow Wayland.

If the API is not lazy, I'm not sure how it should be done, so if yo can provide tl;dr it would help (haven't found clear wording in your research).

This article by a Wayland maintainer describes it as "arguably one of the most complicated parts of the core Wayland protocol"

Well, because it's a lot of work, you need to negotiate what you want to read (image/text/html/whatever), then both ends should do non-blocking writing/reading to an FD, thus implying that you plug that FD into epoll based event loop or something. And ensure that you don't hang each other.

And compositor does very little here actually, it just lets you exchange FDs with other end, but all this negotiation + writing the right mime type is on the client.

So for example, when something drags object/pastes, you have a bunch of mime types you can use to query data, e.g. image/text/audio, then you ask for audio, and other end should provide audio.

Then you want to initiate drag and drop, and you started dragging something, but this something is either text or image, and then depending on what the other end picks(e.g. you drag from winit to firefox, and firefox picks one of two mimetypes you gave to it), winit must reply with the right data. Note that you don't create buffer for both image and text before hand, you only create them once you get event what type of data other end wants. It can also ask you for both, or ask you something from time to time as long as you have advertised something.

eira-fransham · 2026-05-21T08:50:51Z

@kchibisov Thanks for your response. I appreciate you clarifying the way that data transfer (i.e. clipboard and drag-and-drop) works, but it might be worth reading through the PR since I have an extensive doc comment explaining the exact concerns that you bring up and how the API addresses them. In particular:

The API already handles type multiplexing. That is not Wayland-specific, it's done on all platforms
The API is already lazy and asynchronous (in the general sense, not in the async-keyword sense). See the docs for fetch_data_transfer + the DataTransferResult event

The implementation in this PR only addresses drag-and-drop, but it is specifically designed to support clipboard operations in the future. Clipboard operations are planned in a follow-up PR, and only left unimplemented for now for two reasons:

I wanted to focus on the new API rather than adding too much new implementation (see also the note about removing the Wayland implementation in the PR description). This is already going to be a reasonably-large PR and I didn't want to overload the winit team.
The most-pressing concern from the side of Slint is drag-and-drop. Implementing the clipboard via a side-channel is a lot simpler and more self-contained than trying to do so for drag-and-drop.

The type hierarchy is like so:

DataTransfer - the offer of multiple typed views of some data (clipboard or drag-and-drop). None of the data is guaranteed to be resolved at this time. Corresponds to wl_data_offer, NSPasteboard, Windows.ApplicationModel.DataTransfer.DataPackage, etc.
TransferType + TypeHint - the platform-specific and cross-platform representation of a type, respectively. In most cases, TypeHint is enough. It covers the set of advertisable types which have some equivalent in the data transfer mechanism on every platform.
TypedData (could potentially do with a better name) - the data resulting from the resolved request for some specific type of a DataTransfer. Like with DataTransfer and TransferType, this can be interacted with generically or downcast to a platform-specific type.

The API flow as-implemented by this PR is like so:

Instead of receiving a parsed value (as in the design on master), the user just gets an ID.
The user can use that ID to request the types of the data transfer synchronously, and/or request a certain type of the data of the transfer asynchronously.
The user can accept or reject a drag operation by using that same DataTransferId. Even though a DataTransferId could also refer to clipboard data, in this case I figured it was better to just error out if the ID of a data transfer that wasn't part of a drag operation. Alternatively, a separate DragOperationId could be introduced to avoid the overloading, but I don't see a reason to do so.
When the fetch of some type in a data transfer has completed, a dyn TypedData is passed using the DataTransferResult event. The user may read this TypedData using the specified data format, with helpers for plaintext and URI lists since those are special-cased on some platforms and have platform-specific encoding (UTF-8 vs UTF-16) that users should not have to handle manually.

Note that we cannot just return a potentially-blocking io::Read impl when fetching a specific type, as at least on X11 the data is transferred as part of the event loop. TypedData must have the invariant that directly reading it on the event loop may stall the application but can never cause a deadlock. That's also why types can be fetched synchronously - all platforms support reading the types offered by another application without stalling the event loop.

Regarding moving fetch_data_transfer (+ the other data transfer related stuff) to the event loop: the event loop feels like the most-natural place for it, but X11 handles selection transfer on a per-window basis rather than per-event-loop, so putting it on the window was the lowest common denominator w.r.t. cross-platform use. I believe that X11 is the only platform that does it this way though, so I wouldn't be against putting it on the event loop instead.

kchibisov · 2026-05-21T13:29:34Z

Regarding moving fetch_data_transfer (+ the other data transfer related stuff) to the event loop: the event loop feels like the most-natural place for it, but X11 handles selection transfer on a per-window basis rather than per-event-loop, so putting it on the window was the lowest common denominator w.r.t. cross-platform use. I believe that X11 is the only platform that does it this way though, so I wouldn't be against putting it on the event loop instead.

But we can pass WindowId to which window deliver callback. Clipboard is the same, it can not work without window on Wayland, but still, transfer itself is event loop based. Window could be implementation detail of a transfer/etc. For X11 people tended to use hidden empty util window(for clipboard for sure, but it's a bit slow).

Also, X11 is nearly dead, so no point in design around it.

eira-fransham · 2026-05-21T15:07:32Z

Yeah, as of the latest few commits I’ve moved the whole drag-and-drop API to the event loop. Fetching still needs to be done asynchronously to support X11 though unfortunately, there’s not really a way around that without running the risk of deadlocking the event loop. The user can always immediately try to read the data without waiting for the DataTransferResult event, that’ll work on most platforms.

I’ve also updated the dnd example to show how to use the new API.

kchibisov

Generally using event will be fine, I guess. The API should certainly be async, sync won't work on Wayland as well.

eira-fransham · 2026-06-08T08:12:42Z

This PR is now ready for review. The commit history is still really messy, my usual preference would be to either squash-and-merge, or (if the project doesn't squash PRs) manually clean the history up as the final step before merging so I can ensure that any changes in response to PR comments are included in the cleaned-up commits and I don't have to clean up the history multiple times. If you'd prefer that I clean the history up before review then just say so and I'll do that first, I know this is a big block of work so I'm happy to do anything on my end that makes it easier for you to review.

Most of the new API surface is in winit-core/src/data-transfer.rs and winit-core/src/event_loop/mod.rs, so that would be the best place to start. The general API design philosophy that this PR follows is that everything should provide both platform-agnostic and platform-specific APIs, which leads to a lot of uses of dyn Trait and AsAny since everything needs to be object-safe.

eira-fransham · 2026-06-08T09:53:27Z

Looks like CI / Test nightly Windows 32bit MSVC (pull_request) failed for reasons unrelated to the PR, but it won't let me manually re-run it. Does anyone know what happened there, and what I can do about it?

kchibisov · 2026-06-12T16:57:37Z

The general API you propose LGTM.

The only minor things I have are 1) Maybe Cancel of operation should be a bit more explicit, thus removing None? 2) On platform where mime type is a string, generally, maybe we should accommodate for those cases somehow?

kchibisov

I've looked more or less through wayland and somehow x11

For Walyand, I really suggest to move all the transfers via calloop and non-blocking APIs, since winit should never block on IO and right now PR uses write_all.

Also, I'll suggest to e.g. try to drag some big data (more than unix PIPE limit) to itself(e.g. drag and drop between 2 winit windows of the same application), outside.

As part of re-implementing drag-and-drop, we want to re-add each platform as a separate commit. To prevent making the history too messy and to ensure that each commit compiles separately (to allow `git bisect` to work better in the future), we remove the old implementation first.

This implements a `DataTransfer`-style API for sending and receiving clipboard and drag-and-drop data. Implementations for each platform will be added in follow-up commits

This prepares for implementing the `DataTransfer` style drop API for Windows.

This adds an implementation of the `DataTransfer`-style drag-and-drop/clipboard API for Windows.

This adds an implementation of the `DataTransfer`-style drag-and-drop/clipboard API for macOS.

tronical

@kchibisov @madsmtm From my POV this is good to go. I believe the comments have been addressed (separating out the code, recent utf-16 removal, etc.). I'm good with this and would like to merge that in the coming days, unless there are any objections.

eira-fransham · 2026-06-25T07:55:04Z

The only minor things I have are 1) Maybe Cancel of operation should be a bit more explicit, thus removing None? 2) On platform where mime type is a string, generally, maybe we should accommodate for those cases somehow?

@kchibisov Yeah, I changed it so cancel is an explicit separate event instead of reusing OutgoingDragFinished, you're right that it was confusing. As for 2, you can just pass a winit_wayland::MimeType instead of a TypeHint. That usecase is explicitly supported and it's the reason that types are represented with a trait instead of only supporting TypeHint. I also re-exported MimeType from the winit_wayland root for this reason.

This adds an implementation of the `DataTransfer`-style drag-and-drop/clipboard API for Linux + Wayland.

This adds an implementation of the `DataTransfer`-style drag-and-drop/clipboard API for Linux + X11.

kchibisov

So other than this pipe thing I'm concerned, and code that handles data device drop, PR should be fine, I guess.

eira-fransham · 2026-07-03T08:48:59Z

@kchibisov So I think that setting up/tearing down the data device along with the touch capability (rather than just pointer) on Wayland is the last thing, right? What do you think about that, is that a blocker? I mentioned this in the thread but I'll say again here so it's easier to find my reasoning later: I don't have a way to test touch on Wayland, so I'd rather deliberately leave it unimplemented and only support DnD for devices with the pointer capability.

kchibisov · 2026-07-03T09:46:29Z

: I don't have a way to test touch on Wayland, so I'd rather deliberately leave it unimplemented and only support DnD for devices with the pointer capability.

Sure, just in case, I can test touch/tablet, etc myself, and I don't think it matters that much now.

eira-fransham · 2026-07-06T08:48:24Z

Anyone here qualified to fix the cargo deny failure? I can figure out a fix myself if no-one else wants to handle it.

nicoburns · 2026-07-06T11:57:12Z

@eira-fransham For the time being you should probably just add the RUSTSEC id's the ignore list like was done here for Servo: https://github.com/servo/servo/pull/46280/changes#diff-1040309c64844eb1b6b63d8fd67938adbf9461f1b3c61f12cf738f064a02d3deR38

We should also update those dependencies in Winit. But it probably doesn't make sense for that to be coupled with this PR.

eira-fransham · 2026-07-06T14:37:14Z

So CI is succeeding and this looks ready to go. Does anyone have any more changes they'd like to make before merging, and if not does someone want to merge it?

ogoffart

I asked an AI to review this. And it found some valid issues.

ogoffart · 2026-07-06T16:35:07Z

+    newline_reader: Cursor<&'static [u8]>,
+}
+
+fn percent_encode_into(out: &mut Vec<u8>, value: &[u8]) {


(please double check)
percent_encoding::NON_ALPHANUMERIC encodes :, / and . too, so file:///tmp/a.png is sent as file%3A%2F%2F%2Ftmp%2Fa%2Epng which is not a valid URI.

Hmm, interesting. I'll look into it.

ogoffart · 2026-07-06T16:47:50Z

+        // application can accept further types by fetching the data, but
+        // this will at least mean that waiting until the drop to start
+        // fetching data won't prevent the drop from working at all.
+        state.accept(state.transfer_id().into_raw() as _, accepted_type);


Should that be state.serial()

Thank you, I changed how this worked and I thought I'd tracked down all the places that needed changing.

ogoffart · 2026-07-06T16:52:39Z

 wayland-client = "0.31.10"
-wayland-protocols = { version = "0.32.11", features = ["staging", "unstable"] }
+# We need to pin the protocols to maintain compatibility with Rust 1.85
+wayland-protocols = { version = "=0.32.12", features = ["staging", "unstable"] }


This is not the way to maintain compatibility with older MSRV.
Use pinning on the CI.

(Alternatively, i suggest to use resolver = "3" in Cargo.toml)

Could you elaborate on "pinning on the CI"? I think there are a few ways to skin a cat here, I chose the one which rocked the boat the least. resolver = "3" would be my preferred solution, if @kchibisov, @nicoburns et al are ok with that.

The problem with pining like this is that it forces this version for all the downstream, even if they have a higher MSRV.
And it does cause conflicts and errors if some other dependency require a newer version or a differetn version.
So library crates should never pin their third party dependency with "=".

By pinning on the CI, i mean adding it there:

winit/.github/workflows/ci.yml

Line 139 in 27e17e3

run: cargo generate-lockfile && cargo update -p smol_str --precise 0.3.2 && cargo update -p unicode-segmentation --precise 1.12.0 && cargo update -p wayland-protocols --precise 0.32.12

(In fact, it is already there, so it doesn't need further pinning)

Yeah, you're right. I guess someone else fixed it in master between me doing making that change and now. I'll remove it.

resolver = 3 is better, I think, all this pinning started way before resolver = 3.

ogoffart · 2026-07-06T16:57:37Z

+
+                let surface = state.compositor_state.create_surface(&self.queue_handle);
+                buffer.attach_to(&surface).ok()?;
+                surface.offset(icon.offset_x, icon.offset_y);


wl_surface.offset is a v5 request.
So it needs to be gated with if surface.version() >= 5

Great catch, thank you

ogoffart · 2026-07-06T17:00:02Z

+#[derive(Default, Debug)]
+pub struct UriListEncoder {
+    uris: <Vec<OsString> as IntoIterator>::IntoIter,
+    // Weird system with two fields since otherwise we get lifetime errors.


What kind of error with lifetime?

ogoffart · 2026-07-06T17:44:57Z

+    /// set of valid actions supplied using this method, combined with the set of valid actions
+    /// on the drag source. If the drag is rejected at the point that the user finalizes the drop,
+    /// the application will receive [`DragLeft`](crate::event::WindowEvent::DragLeft) instead
+    /// of [`DragDropped`](crate::event::WindowEvent::DragDropped)`.


Suggested change

/// of [`DragDropped`](crate::event::WindowEvent::DragDropped)`.

/// of [`DragDropped`](crate::event::WindowEvent::DragDropped).

ogoffart · 2026-07-06T17:45:34Z

-        /// (x,y) coordinates in pixels relative to the top-left corner of the window. May be
-        /// negative on some platforms if something is dragged over a window's decorations (title
-        /// bar, frame, etc).
+    /// The position of an ongoing drag operation has .


unfinished sentence.

Suggested change

/// The position of an ongoing drag operation has .

/// The position of an ongoing drag operation has changed.

(presumably)

ogoffart · 2026-07-06T17:46:36Z

+        /// [`crate::event_loop::ActiveEventLoop::data_transfer`].
+        id: DataTransferId,
+    },
+    DataTransferReceived {


Missing documentation for thie enum

ogoffart · 2026-07-06T17:48:15Z

+    /// If this value is not readable as URIs, return `None`.
+    ///
+    /// The format of the returned URIs is simply a vector of strings. No validation is done
+    /// to ensure that the URIs are valid or in the format


Sentence ends mid-thought.

Maybe mention what format is expected on each platform (eg: file:// or just path)

ogoffart · 2026-07-06T17:49:38Z

 impl fmt::Debug for WindowId {
    fn fmt(&self, fmtr: &mut fmt::Formatter<'_>) -> fmt::Result {
-        self.0.fmt(fmtr)
+        write!(fmtr, "{:?}", self.0)


This clears the formatter flags (eg {:#?}) Is that intentional?

kchibisov reviewed May 22, 2026

View reviewed changes

Comment thread winit/examples/dnd.rs Outdated

Parth mentioned this pull request May 23, 2026

wayland DnD situation lockbook/lockbook#4607

Open

eira-fransham force-pushed the drag-n-drop branch 2 times, most recently from 650e7ee to 2e99f38 Compare May 26, 2026 12:36

eira-fransham force-pushed the drag-n-drop branch from a0c95c8 to ea394ea Compare June 3, 2026 14:50

eira-fransham marked this pull request as ready for review June 3, 2026 15:25

eira-fransham requested a review from madsmtm as a code owner June 3, 2026 15:25

eira-fransham changed the title ~~WIP: New drag and drop API~~ New drag and drop API Jun 3, 2026

eira-fransham requested a review from kchibisov June 3, 2026 18:05

eira-fransham mentioned this pull request Jun 5, 2026

WIP: Initiating and receiving drag and drop using winit API slint-ui/slint#11966

Open

ogoffart reviewed Jun 8, 2026

View reviewed changes

tronical reviewed Jun 8, 2026

View reviewed changes

Comment thread winit-appkit/src/app_state.rs Outdated

Comment thread winit-appkit/src/dnd.rs

Comment thread winit-core/src/data_transfer.rs

Comment thread winit-core/src/data_transfer.rs Outdated

Comment thread winit-core/src/event_loop/mod.rs Outdated

kchibisov requested changes Jun 12, 2026

View reviewed changes