App kit

AGENTS.md

@@ -26,11 +26,14 @@ The design lives in `docs/design/sync.md`; the live protocols (the server's live
 
 ```text
 crates/
-  proto     craft-cloud-proto     wire types: names, IDs, hashes, manifests, commit ops, API bodies, limits, join codes and join strings (`join.rs`)
+  api       craft-cloud-api       what apps need of the protocol, with no postcard: IDs, names, limits, the agent's local API (`local.rs`, `/v1/blobs` too), error codes and the account's devices (`api.rs`)
+  proto     craft-cloud-proto     wire types: hashes, manifests, commit ops, API bodies, join codes and join strings (`join.rs`), the live protocol (`live.rs`); re-exports `craft-cloud-api` under the same paths
   chunker   craft-cloud-chunker   FastCDC `fastcdc-v1` + BLAKE3, frozen by golden vectors
   store     craft-cloud-store     blob store: disk, S3, memory; checks every blob in and out
   meta      craft-cloud-meta      metadata on SQLite or Postgres; the batch commit; the change feed; invites and pairings (`join.rs`); live documents' logs and snapshots (`live.rs`)
   sync      craft-cloud-sync      the sync engine over `Fs`/`Transport` traits; `MemFs`; scenario tests and the simulation
+  kit       craft-cloud-kit       the app kit (`docs/kit.md`): an app's model live through the agent; the `LiveDocument` trait and `Session` (`session.rs`: open, the `local_change` hook, `poll`, undo through Loro's undo manager, reconnect and the 64 MiB queue, presence, saves), the change writer that cuts commits under 512 KiB (`writer.rs`), the mappings (`tree.rs`, `text.rs`, `table.rs`, `timeline.rs`, `opaque.rs`, `list.rs`), values, chunk references and blob stores (`value.rs`)
+  egui      craft-cloud-egui      Craft Cloud's piece of an app's window: the account line, the shared recents, a live document's badge, and `CloudWatch` that keeps them current
   app       craft-cloud-app       what apps link: a blocking client of the agent's local API (status, sign-in, joining, computers, pause, autostart, recents, live documents at the level of bytes: `live.rs`)
 apps/
   craft-cloud-server              HTTP API (`/api/v1`), the live service (`/api/v1/live`, `live.rs`: logs, documents by import, snapshots and compaction, presence), join codes (`join.rs`: pairings, join, the `/join/{code}` page), auth, admin CLI (`init`, `serve`, `user` with `invite`, `device`, `reset-epoch`, `live`)
@@ -41,14 +44,17 @@ apps/
   craft-cloud-desktop             the Craft Cloud app (`craft-cloud`, eframe/egui): the pure model (`model.rs`), the agent behind a trait and calls on threads (`agent.rs`), starting the agent (`start_agent.rs`), one app per user (`single_instance.rs`), the window (`ui/`, `app.rs`), the tray (`tray/`: tray-icon in the window's process on Windows and macOS; on Linux a StatusNotifierItem over zbus in a process of its own, `craft-cloud --tray`, `tray/process.rs`, since Wayland can't hide a window), the app's own start at login (`autostart.rs`), its icons (`assets/`, drawn by `cargo xtask icons`)
     tests/snapshots/              every screen as kittest draws it (`UPDATE_SNAPSHOTS=1` redraws them)
   craft-cloud-setup               the Windows setup (`craft-cloud-setup.exe`): install, upgrade, move from a hand-made install, `--uninstall`; the exes built in by `cargo xtask package`
+  craft-cloud-kit-demo            the app kit's demo (never packaged or installed): a story and a layer tree, live through the kit; its model and adapter (`model.rs`), its window (`ui.rs`); tests drive two windows on one fake hub, and time the kit on a big document
 packaging/linux/                  the Linux tarball's `install.sh` and `uninstall.sh` (into `~/.local`)
 xtask/                            cargo xtask ci | licenses | icons | package | version
 docs/                             design documents and the self-hosting guide
 ```
 
-Layering: `proto` and `chunker` depend on nothing else in the workspace. `store` and `meta` depend on `proto`.
+Layering: `api` and `chunker` depend on nothing else in the workspace; `proto` depends on `api` only. What apps link
+(`app`, `kit`, `egui`) depends on `api`, never on `proto`, so it brings no crate a Craft app doesn't already lock but
+Loro's own (`cargo xtask suite-deps <the apps' Cargo.lock files>` checks). `store` and `meta` depend on `proto`.
 The server depends on all three; nothing depends on the server outside tests. `sync` depends on `proto` and `chunker`
-(its tests also use `meta` and `store` as an in-process server). `app` depends on `proto` only and has no async
+(its tests also use `meta` and `store` as an in-process server). `app` depends on `api` only and has no async
 runtime. The agent depends on `proto`, `sync` and `app` (its `recents` commands are clients like an app); its
 tests also run the server on a loopback port, and `tests/network/` runs the server's router and agents over
 `turmoil` (a dev-dependency: the shipped binaries never link it). That test target includes sync's test harness

Cargo.toml

@@ -3,7 +3,7 @@ resolver = "3"
 members = ["crates/*", "apps/*", "xtask"]
 
 [workspace.package]
-version = "0.0.1"
+version = "0.1.0"
 edition = "2024"
 license = "MIT OR Apache-2.0"
 rust-version = "1.94"
@@ -17,6 +17,7 @@ dbg_macro = "warn"
 todo = "warn"
 
 [workspace.dependencies]
+craft-cloud-api = { path = "crates/api" }
 craft-cloud-proto = { path = "crates/proto" }
 craft-cloud-chunker = { path = "crates/chunker" }
 craft-cloud-store = { path = "crates/store" }
@@ -25,6 +26,8 @@ craft-cloud-server = { path = "apps/craft-cloud-server" }
 craft-cloud-sync = { path = "crates/sync" }
 craft-cloud-agent = { path = "apps/craft-cloud-agent" }
 craft-cloud-app = { path = "crates/app" }
+craft-cloud-kit = { path = "crates/kit" }
+craft-cloud-egui = { path = "crates/egui" }
 argon2 = "0.6"
 async-trait = "0.1.89"
 # `ws`: the server's live service and the agent's local API are WebSockets (tokio-tungstenite, MIT).

README.md

@@ -7,8 +7,8 @@ all your machines, and the folder `~/Craft Cloud` stays in sync, with version hi
 There is no hosted service and no account with anyone but you. Nothing phones home.
 
 > **Status: early development.** File sync, shared recent files and the Craft Cloud app work. Live
-> documents (an edit shows on your other machines as you type) work in the agent and the server; the apps take
-> them up next.
+> documents (an edit shows on your other machines as you type) work in the agent and the server, and the app kit
+> lets a Craft app take them up with one adapter; the apps take them up next.
 
 - **Big files sync quickly.** Files are split into content-defined chunks, so a change to a 20 GB project
   uploads only what changed, and an interrupted upload resumes.
@@ -38,6 +38,7 @@ that is signed in, and enter its code on the new one.
 - [Running the server](docs/server.md): reverse proxy, Postgres and S3, accounts, invites, backups
 - [The agent on the command line](docs/agent.md): `login`, `run`, `sync`, `join`, start at login
 - [Development](docs/development.md): how it fits together, building, testing, the sync engine's rules
+- [The app kit](docs/kit.md): how a Craft app makes its documents live
 - Design: [`sync.md`](docs/design/sync.md) and [`live.md`](docs/design/live.md). Contributors and AI agents
   start with [`AGENTS.md`](AGENTS.md).
 

apps/craft-cloud-agent/Cargo.toml

@@ -59,6 +59,7 @@ path = "src/main.rs"
 winresource.workspace = true
 
 [dev-dependencies]
+craft-cloud-kit.workspace = true
 craft-cloud-meta = { workspace = true, features = ["test-support"] }
 craft-cloud-server = { workspace = true, features = ["test-support"] }
 craft-cloud-store.workspace = true

apps/craft-cloud-agent/src/cli.rs

@@ -922,6 +922,9 @@ fn show_time(ms: i64) -> String {
     }
 }
 
+/// How soon a pass that left a Craft save waiting is followed by another.
+const SAVE_WAIT_RECHECK: std::time::Duration = std::time::Duration::from_secs(2);
+
 pub(crate) fn now_ms() -> i64 {
     Timestamp::now().as_millisecond()
 }
@@ -1435,6 +1438,8 @@ impl Session {
         let transport = HttpTransport::new(account.server.clone(), &account.token)?;
         let config = Config::new(account.device_name.clone(), account.device);
         let mut engine = Engine::new(fs, transport, std::mem::take(&mut saved.state), config, conflict_stamp)?;
+        // What Craft apps saved from live documents (their local API's `saved`): see `live::saves`.
+        engine.set_save_records(std::sync::Arc::new(crate::live::saves::SaveBook::new(data.path(), account.space)));
         // `state.json` is also written at the pass's checkpoints, so a crash or a kill loses at most what the pass did
         // since the last one (the last pass's outcome stays as it was until this one ends).
         let last_pass = std::sync::Arc::new(std::sync::Mutex::new(saved.last_pass));
@@ -1496,6 +1501,15 @@ impl Session {
             (Ok(_), Some(stability), Some(started)) => stability.pass_finished(started),
             _ => None,
         };
+        // A Craft save waiting for its record, or two whose order isn't known yet: looked at again soon (a record that
+        // lands asks for a pass by itself; a wait that runs out doesn't).
+        let waits = |why: &String| why.starts_with(craft_cloud_sync::SAVE_RECORD_WAITS) || why.starts_with(craft_cloud_sync::SAVE_ORDER_WAITS);
+        if let Ok(report) = &result
+            && report.skipped.iter().any(|(_, why)| waits(why))
+        {
+            let soon = SAVE_WAIT_RECHECK;
+            self.recheck_in = Some(self.recheck_in.map_or(soon, |at| at.min(soon)));
+        }
         let (outcome, summary) = match &result {
             Ok(report) => {
                 let summary = PassSummary::from_report(report);

apps/craft-cloud-agent/src/daemon.rs

@@ -71,6 +71,15 @@ pub struct RunOptions {
     /// Where the local API's `autostart` puts the agent's start-at-login entry (and a sign-out removes it from):
     /// this user's, or nowhere / a temporary place in tests, which must not touch the machine's own.
     pub autostart: crate::autostart::Target,
+    /// Tests: how long the live hub holds every app's save record back before taking it (a slow agent, as under load).
+    #[doc(hidden)]
+    pub record_delay: Duration,
+    /// How long after an app session had a document open an unannounced change of its file is still waited for as a
+    /// Craft save on its way (`live::saves::SESSION_GRACE`, 60 s; tests shorten it).
+    pub session_grace: Duration,
+    /// Tests: how late the live hub reads every app's save announcement (a hub busy elsewhere).
+    #[doc(hidden)]
+    pub announce_delay: Duration,
 }
 
 impl Default for RunOptions {
@@ -85,6 +94,9 @@ impl Default for RunOptions {
             poll_busy: Duration::from_secs(5),
             fail_watch: false,
             autostart: crate::autostart::Target::User,
+            record_delay: Duration::ZERO,
+            session_grace: crate::live::saves::SESSION_GRACE,
+            announce_delay: Duration::ZERO,
         }
     }
 }
@@ -745,6 +757,9 @@ pub async fn run_loop(data: &DataDir, options: RunOptions, mut stop: mpsc::Unbou
             token: account.token.clone(),
             machine: account.device_name.clone(),
             folder: account.folder.clone(),
+            record_delay: options.record_delay,
+            session_grace: options.session_grace,
+            announce_delay: options.announce_delay,
         };
         let hub = match Hub::start(hub_config, control.clone(), user_paused) {
             Ok(hub) => Some(hub),

apps/craft-cloud-agent/src/doc.rs

@@ -105,6 +105,10 @@ fn session(
     };
     let mirror = LoroDoc::new();
     mirror.import(document.state()).map_err(|e| usage(format!("the document's state doesn't read: {e}")))?;
+    for update in document.state_updates() {
+        // (An update this copy can't take is the other side's; the agent has it, and so will a fresh open.)
+        let _ = mirror.import(update);
+    }
     let text = |mirror: &LoroDoc| mirror.get_text(TEXT).to_string();
     // What came right after the state (updates still waiting for their history), before anything else.
     let take = |mirror: &LoroDoc, event: &Event| {
@@ -128,7 +132,7 @@ fn session(
             let left = until.saturating_duration_since(Instant::now());
             match document.recv_timeout(left) {
                 Some(Event::Acked(acked)) if acked == seq => break,
-                Some(Event::Refused { seq: refused, message }) if refused == seq => {
+                Some(Event::Refused { seq: refused, message, .. }) if refused == seq => {
                     return Err(usage(format!("the agent refused the change: {message}")));
                 }
                 Some(Event::Closed(why)) => return Err(usage(format!("the connection to the agent ended: {why}"))),
@@ -157,7 +161,7 @@ fn session(
                 say(format!("the connection to the agent ended: {why}"));
                 return Ok(Exit::Failed);
             }
-            Some(Event::Presence(_) | Event::Acked(_) | Event::Refused { .. }) | None => {}
+            Some(Event::Presence(_) | Event::Acked(_) | Event::Refused { .. } | Event::File(_)) | None => {}
         }
     }
 }

apps/craft-cloud-agent/src/http.rs

@@ -40,7 +40,7 @@ use craft_cloud_proto::api::{
 };
 use craft_cloud_proto::limits::{MAX_BATCH_OPS, MAX_CHUNK_BYTES, MAX_MANIFEST_BYTES, MAX_PAGE_NODES};
 use craft_cloud_proto::local::MAX_MACHINE_BYTES;
-use craft_cloud_proto::{ChangeSet, ChunkId, DeviceId, ManifestId, Op, PageCursor};
+use craft_cloud_proto::{ChangeSet, ChunkId, DeviceId, ManifestId, NodeId, Op, PageCursor, SaveRecord, VersionId};
 use craft_cloud_sync::{Transport, TransportError};
 use reqwest::header::{AUTHORIZATION, CONTENT_LENGTH, CONTENT_TYPE, HeaderMap, HeaderValue, RETRY_AFTER, USER_AGENT};
 use reqwest::{Client, Method, StatusCode, Url};
@@ -749,6 +749,13 @@ impl<C: Connection> Transport for HttpTransport<C> {
         let req = self.request(Method::POST, "commit", MAX_SMALL_JSON_BYTES).json(&CommitRequest { ops: ops.to_vec() })?;
         json(self.send(req).await?, MAX_SMALL_JSON_BYTES).await
     }
+
+    async fn attach_save(&self, node: NodeId, version: VersionId, save: &SaveRecord) -> Result<(), TransportError> {
+        let body = craft_cloud_proto::api::SaveAttach { node, version, save: save.clone() };
+        let req = self.request(Method::POST, "saves", MAX_SMALL_JSON_BYTES).json(&body)?;
+        let _: CommitResponse = json(self.send(req).await?, MAX_SMALL_JSON_BYTES).await?;
+        Ok(())
+    }
 }
 
 #[cfg(test)]