diff --git a/development/serverless/library-models.mdx b/development/serverless/library-models.mdx new file mode 100644 index 0000000000..b781db5bb7 --- /dev/null +++ b/development/serverless/library-models.mdx @@ -0,0 +1,163 @@ +--- +title: "Workspace Library Models on a Comfy API Deployment" +sidebarTitle: "Library Models" +description: "Upload a LoRA or other .safetensors model to your workspace library and run it on a Comfy API deployment with no new Build, Release, or redeploy." +icon: "folder-open" +--- + +Comfy API runs models from your workspace library, even when the deployment's Release doesn't include them. Upload a LoRA or another `.safetensors` model to the library, and run it on your deployment without a new Build, Release, or redeploy. + +Use the workspace library to try a LoRA you trained on your deployment, or to pick a different LoRA for each API request by changing one file name in the Workflow. + +## Before you start + +Before you upload a model to the workspace library, make sure you meet these requirements: + +- **Plan**: Your workspace needs an active Pro, Creator, Team, or Enterprise plan. +- **Model file**: The file must be a `.safetensors` file of 2 GB or less. Larger base models, such as SDXL or Flux, still go in the Build. +- **An active deployment**: The deployment must be in the same workspace as the library. + + + Some deployments created before workspace library models were available can't load them. To use workspace library models, create a new deployment. On an older deployment, a job can fail with `Library model "" is unavailable on this deployment. Retry the job. If it still fails, contact support with the job ID.` + + +## Upload a model to the workspace library + +You can upload a model with any of three tools: + +- **Cloud editor**: upload a file, or import from a Civitai or Hugging Face link. +- **comfy-cli**: upload from a terminal or a script. +- **Developer Platform**: upload from a Build's models step. + +Pick the tab for the tool you use. + + + + 1. In the deployment menu to the left of **Run**, make sure **Comfy Cloud** is selected under **Run on**. + 2. In the **Left sidebar**, click **Model Library**, then click **Import**. + 3. Click **Choose a file**, or drag a `.safetensors` file into the dialog. + 4. Under **What type of model is this?**, keep **LoRA** for a LoRA, or select the type that matches the file. + 5. Click **Upload**. When the upload finishes, click **Finish**. + + ![Import a model dialog with my_style_lora_9e2417.safetensors (97.37 MB) chosen, What type of model is this? set to LoRA, and the Back and Upload buttons](/images/development/serverless/library-models-editor-import.png) + + To import a model from a Civitai or Hugging Face link instead: + + 1. In the **Import** dialog, paste the model's link in **Paste link here**. + 2. Click **Continue**. + 3. Under **What type of model is this?**, select the model type. + 4. Click **Import**. + + Only `.safetensors` files run on a deployment. A link import in another format, such as `.ckpt`, imports into the library but doesn't run on a deployment. + + + Upload the file and set its model type with `--type`: + + ```bash + comfy assets library upload my_style.safetensors --type loras + ``` + + Check that the file is in the library: + + ```bash + comfy assets library ls --tags models,loras + ``` + + + 1. On the Developer Platform, open the models step of a Build and click **Upload Models**. + 2. Select **Workspace library**. + 3. Drag a `.safetensors` file onto the drop area, or click the area to browse. + 4. Under **Model type**, keep **LoRA** for a LoRA, or select the type that matches the file. + 5. Click **Upload**. + + ![Upload Models dialog on a Build's Which models ship inside the build? step, with the Public, Private, and Workspace library tabs, Workspace library selected, the drop zone, Model type set to LoRA, a line saying the file is in your workspace library, and the Cancel and Upload buttons](/images/development/serverless/library-models-platform-upload.png) + + + +Every member of the workspace can use the model once it is in the library. The library takes any model type, not only LoRAs. Pick the type that matches the folder your loader Node reads, such as `loras` for a LoRA or `vae` for a VAE. + +## Run the model on your deployment + +Once the model is in the library, you can run it on your deployment from the Cloud editor or through the API. Pick the tab for how you send jobs. + + + + 1. Click the deployment menu to the left of **Run**. Under **Run on**, select your deployment. The editor reloads, and your tabs stay open. + 2. Add a `Load LoRA (Model and CLIP)` Node, or open a Workflow that has one. + 3. In the Node's `lora_name` list, select your file. The list shows your library models next to the Release's models. + 4. Click **Run**. + + On the first run, the job's row in the queue shows `Downloading my_style.safetensors` while the worker downloads the file. When the job downloads several models, the row shows `Downloading ( of )`. + + ![Workflow with Load Checkpoint and Load LoRA (Model and CLIP) Nodes, the uploaded LoRA selected in lora_name, and the queue panel's top row reading Downloading my_style_lora_9e2417.safetensors 0 B of 97.37 MB](/images/development/serverless/library-models-editor-downloading.png) + + + In the [API-format Workflow](/development/api-development/workflow-api-format), set the `lora_name` input of the `LoraLoader` Node to the library file name, including `.safetensors`: + + ```json + "10": { + "class_type": "LoraLoader", + "inputs": { + "lora_name": "my_style.safetensors", + "strength_model": 1.0, + "strength_clip": 1.0, + "model": ["4", 0], + "clip": ["4", 1] + } + } + ``` + + Send the Workflow with `comfy deploy run`, the [Comfy SDKs](/development/api-development/sdks), or the [Comfy API v2](/api-reference/v2/overview): + + ```bash + comfy deploy run --deployment --workflow workflow_api.json + ``` + + To use a different LoRA, change `lora_name` to another file name from your library in the next request. You don't need a new Build, Release, or redeploy. + + + + + The first job on a worker that uses a library model takes longer, because the worker downloads the model first. Later jobs on the same worker use the cached copy. A new worker downloads the model again, and so does a worker that removed the model from its cache to make room for other models. To skip the download, add the model to the Build. + + +## Follow the limits and naming rules + +Besides the file size and format limits in [Before you start](#before-you-start), these limits apply: + +| Limit | Value | +| --- | --- | +| Workspace storage | 1 TB of models per workspace, including link imports | +| Library model names per Workflow | 100 | + +A loader that names a 101st library model fails as if the file were not in the library. + +Follow these rules so a job finds the right file: + +- **Use the folder the loader reads.** A loader only finds library files of its own model type. A `Load LoRA` Node finds files uploaded as `loras`, not as `checkpoints`. +- **Use the bare file name.** Set the loader input to the file name only, such as `my_style.safetensors`, with no folder in front of it. +- **Don't reuse a Release model's name.** If the Release already has a file with that name, the worker uses the Release's file, not your library file. +- **Keep names unique across folders.** If the library has the same file name in two folders that one loader reads, a job sent through the API loads neither file. +- **Expect the newest file to win.** If the library has two files with the same name in one folder, jobs use the one added last. +- **Match the base model.** A LoRA must match the base model the Workflow loads. +- **Use a loader the deployment has.** The loader Node must be available on the deployment. Library models load only through loader Nodes that Comfy Cloud has. If your loader comes from a Custom Node that Comfy Cloud doesn't have, add its models to the Build. + +## Change or delete a library model + +When a deployment is selected under **Run on**, the **Model Library** lists only the Release's models. To change or delete a library model, switch back to Comfy Cloud first: + +1. Click the deployment menu to the left of **Run**. Under **Run on**, select **Comfy Cloud**. +2. In the **Left sidebar**, click **Model Library**. + +To change the model type: + +- Hover over the model's card, click **Model Info**, and select a new **Model Type**. + + + Deleting a model removes it from the workspace library for every member. A job or API request that still names the file fails with a `Value not in list` error on the loader input. + + +To delete the model: + +1. Hover over the model's card, click **More**, then click **Delete**. +2. In **Delete this model?**, click **Delete**. diff --git a/development/serverless/overview.mdx b/development/serverless/overview.mdx index 9351486a08..49c5e5351d 100644 --- a/development/serverless/overview.mdx +++ b/development/serverless/overview.mdx @@ -18,6 +18,8 @@ Comfy API lets your application run ComfyUI workflows through a managed endpoint Workflows supported by [Comfy Cloud](/development/deploy/cloud) can run there without creating a Build or deployment. +To add a model to a deployment without a new Build, upload it to your workspace library. See [Workspace library models](/development/serverless/library-models). + Start with an agent or choose another way to provide your environment or workflow. @@ -119,7 +121,7 @@ comfy deploy events --deployment dep_123456 Storage billing starts with a deployment: - - When you deploy a release, its models are staged onto network storage shared by the deployment's workers. The storage is read-only after deployment and billed per GB-month for as long as any deployment of the Build exists in that region, including while a deployment is paused with `comfy deploy stop`. + - When you deploy a release, its models are staged onto network storage shared by the deployment's workers. The storage is read-only after deployment and billed per GB-month for as long as any deployment of the Build exists in that region, including while a deployment is paused with `comfy deploy stop`. Models a job uses from your [workspace library](/development/serverless/library-models) are not staged there. The worker downloads them for the job. - Each worker also gets a fixed 50 GB container disk. It is ephemeral and is only billed while the worker is running, as part of the worker's compute cost. - Deleting a deployment releases its compute. Staged network storage is cleaned up shortly after the last deployment using it in that region is deleted, and billing for it ends. diff --git a/images/development/serverless/library-models-editor-downloading.png b/images/development/serverless/library-models-editor-downloading.png new file mode 100644 index 0000000000..a5a9ccbf6f Binary files /dev/null and b/images/development/serverless/library-models-editor-downloading.png differ diff --git a/images/development/serverless/library-models-editor-import.png b/images/development/serverless/library-models-editor-import.png new file mode 100644 index 0000000000..968aa868e1 Binary files /dev/null and b/images/development/serverless/library-models-editor-import.png differ diff --git a/images/development/serverless/library-models-platform-upload.png b/images/development/serverless/library-models-platform-upload.png new file mode 100644 index 0000000000..b18787b176 Binary files /dev/null and b/images/development/serverless/library-models-platform-upload.png differ