Skip to content

About

No description, website, or topics provided.

Resources

Stars

8 stars

Watchers

1 watching

Forks

Repository files navigation

IntelligentMixVideo

An intelligent video remixing agent system built on Alibaba Cloud IMS, with custom templates, server-side text segmentation, media retrieval, and agent-generated orchestration and Remotion effects.

GitHub Stars Latest release Project validation status AGPL-3.0-only

English · 中文


What It Does

IntelligentMixVideo connects desktop template editing, server-side media processing, and cloud video rendering. Create reusable visual templates, then use the server APIs to combine scripts, narration, and video assets into finished videos.

  • Visual template editing: Add decorative text, speech bubbles, filters, visual effects, transitions, and text animations. Adjust text, font size, position, and effect parameters.
  • Top and bottom color bars: Select the mask tab to add full-width bars with separate switches, heights, and colors. They cover the video for its full duration while existing text and effects remain above them; preview and cloud composition use the same geometry.
  • Preview and timeline: Preview template effects, set object start times in seconds or as a percentage of the video duration, and choose a fixed duration or let an object run to the end of the video.
  • Media and avatar together: Choose separate windows, an avatar over full-canvas material, or an automatically located circular head. Overlay layouts keep the complete material with blurred padding. Position the avatar with canvas X/Y and size; only matched material intervals use the layout, while gaps retain the original full-screen avatar. See the template API documentation.
  • Cloud template library: Templates are saved to MySQL through the server and shared by everyone connected to it.
  • Avatar opening and background removal: Automatically locate a fixed head circle for a timed opening, then restore the original material timeline. Optional IMS matting removes the avatar background during shared-screen intervals. The editor configures the layout; final effects are verified in cloud output. See the template API documentation.
  • Text and audio processing: ASR transcription, text segmentation, and keyword processing provide data for subtitles and composition timelines. Titles and subtitles can each enable Chinese–English text with independent styles; composition translates the actual content and synchronizes both languages. Disabled English settings are retained for reuse and omitted from output.
  • Asynchronous video composition: Depending on the inputs, use the standard digital avatar workflow or compose media assets with or without narration. Retrieve results through task queries and callbacks.
  • Remotion text effects: Generate effects with an agent, preview successful versions, and save them as assets for cloud templates.

Video composition is currently accessed mainly through the server API; the client does not yet have a dedicated page for composing complete videos. Isolated generation of new Remotion effects requires a Linux server and the rendering environment. See Supported Capabilities for details.

Workspace

Page Purpose
Home Browse cloud templates, create templates, and open existing work
Template editor Select assets on the left, preview in the center, edit object parameters and timing on the right, and reorder videos on the timeline below
Remotion effects Request new effects, view successful versions and history, adjust parameters, and save assets
Settings Manage the backend URL; Debug also provides module settings

Templates store object configurations and timing rules separately from the preview video. Replacing the preview video recalculates object visibility intervals based on the new video duration.


Workflow

Template editing, video composition, and Remotion effects each handle a different stage:

  • Template editing: Add text and effects in the client, set object intervals, and save to the cloud library. Only the template configuration is saved, not the preview video.
  • Video composition: Call the server API with a cloud template ID, script, and media URLs. Based on the input mode, the server performs ASR, text segmentation, asset matching or sequential arrangement, submits the timeline to IMS, transfers the rendered video to ZOS, and returns the result through queries or callbacks.
  • Remotion effects: Generate and validate effects with an agent, save successful versions as fixed assets, and add them to a cloud template timeline. A separate overlay rendering API exports the full set of Remotion clips as a transparent video.

Quick Start

Choose a setup based on what you want to do:

Goal Requirements
Edit templates or use server APIs A client or API caller, an accessible server, and MySQL
Generate new Remotion effects A Linux server with model configuration and an isolated rendering environment
Compose videos in the cloud IMS, ZOS, and any ASR, segmentation, or asset matching services required by the selected mode

The SDK, fonts, and sample media used for template previews still require network access. Cloud features also need valid credentials and configuration after installation.

Using an Installer

Get official versions from Releases. Development builds are available under Artifacts for the corresponding GitHub Actions run and are generally retained for 14 days.

Platform Architecture Installer
Windows x64 NSIS .exe, .msi
Linux x64 .deb, .AppImage
macOS Apple Silicon, Intel .dmg for the corresponding architecture

The two installer variants use different backend setups:

Package Backend Usage
Standard Connects to an external API Templates, effects, and composition require an available backend
Debug Bundles an API, a private database, and runtimes On first launch, extracts the runtimes, initializes the database, and waits for the API to become ready; connects to an external backend instead if an explicit URL is saved

Debug installer artifacts are named intelligent-mix-video-debug-<platform>-<commit SHA>. They use optimized builds; the main difference is the bundled backend. The Linux bundled backend requires glibc 2.35 or later. First-time initialization may take a while.

After installing and opening the client, check the backend URL and related configuration in Settings, then follow Basic Usage to create a template.

Official clients expose version checks and one-click updates in Settings. Compatible, validated UI updates keep the window open; native updates install and restart automatically (deb uses a download guide). Configure release signing and record platform validation before enabling hot updates; see the client update guide.

Windows code signing and macOS notarization are not currently configured. See the client documentation and Builds and Releases for platform setup and build details.


Running from Source

Client development requires Bun, Rust stable, and the Tauri 2 platform prerequisites. Use the Bun version specified by packageManager in client/package.json. The server requires Python 3.12 or later, uv, and MySQL.

  • Windows: Install Visual Studio C++ Build Tools, the Windows SDK, and WebView2. ARM64 hosts also need the matching ARM64 C++ tools.
  • macOS: Install Xcode Command Line Tools and prepare the environment as described in the Tauri documentation.
  • Linux: Install the system dependencies for your distribution as described in the Tauri documentation.

If you have not cloned the repository yet, run:

git clone https://github.com/HsiangNianian/IntelligentMixVideo.git
cd IntelligentMixVideo

The following steps start at the root of the checked-out repository. If you already have a local checkout, use it directly. Available features and configuration depend on the documentation included with the selected branch or version.


Running the Server

Templates, effects, and the APIs all require the server:

  1. Start MySQL.
  2. For initial setup, copy server/.env.example to server/.env. If the configuration file already exists, edit it directly.
  3. Set DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME.
  4. Run the following from a terminal at the repository root:
uv sync --locked --project server
uv run --locked --project server server

The server listens on 0.0.0.0:20070 by default. Once it starts, open the API documentation on the same machine. Startup creates the database if it is missing, so the configured database account needs permission to create it.

Remote clients must use the server IP address or domain name, and the server port must be accessible. See the server documentation for database, API, and configuration details.


Running the Client

Open another terminal and enter client/ from the repository root. For initial setup, you can copy client/.env.example to client/.env. If the API URL is unset or empty, it defaults to http://localhost:20070.

cd client
bun install --frozen-lockfile
bun run tauri dev

This opens the desktop development window. With the server running, create or select a template from Home.

You can also use browser development mode from client/:

bun run dev

Then visit http://localhost:1420. Templates require a server connection in both browser and desktop modes.


Basic Usage

Creating and Saving a Template

  1. On Home, create a cloud template and enter its name and description.
  2. A new template includes one title and one subtitle. Select styles and effects from the asset panel on the left; VFX is limited to one object, and multiple filters are allowed.
  3. Select an object and adjust its text, font size, position, style, and animation on the right.
  4. Set object timing in the inspector on the right and preview it in the center. The subtitle always covers the full video. Drag video clips on the timeline to change their order.
  5. Click Save. The first save creates the template record; later saves update the same template.
  6. Return to Home and confirm that the template appears in the template list.

Switching templates or creating a new one prompts you to handle unsaved changes. Editing does not provide autosave or crash recovery.

The preview uses the first ten seconds of each of three videos, or the full source when it is shorter; local files picked in the dialog keep their full length. Load all three URLs through the preview dialog, or pick a local file for any of them (kept only on this machine; the desktop client keeps it across restarts). The first transition connects videos 1 and 2; the second connects videos 2 and 3. Edit their effects and durations in the inspector. The timeline lists title, subtitle, bubbles, transitions, VFX, filters, and videos from top to bottom. Preview sources and video order are independent of saved templates.

Default and custom sources both use their actual media durations. Template rules and media information produce a shared timing result: preview duration subtracts transition overlaps, while composition preserves the fixed audio and material timing. Transitions require a boundary and duration; unavailable boundaries produce an explanation. Saving enforces the object count and timing rules. Invalid stored templates remain visible with their ID, validation reason, and individual delete action; valid templates remain usable. Upgrade the client and server together.

New compositions apply transitions at the nearest cuts strictly before and after each subtitle keyword appears; a simultaneous cut is excluded, and shared cuts receive only one effect. Mixed avatar videos search avatar/material switches, while material-only videos search material cuts. With no spoken keywords anywhere in the video, every cut is eligible. Sequential selection cycles through configured effects; random selection allows unlimited reuse. Existing tasks retain their accepted rules when resumed.

See template behavior for timing rules, asset operations, and save constraints.

Using Remotion Effects

Configure Remotion Agent on the server (or in Debug client Settings) and confirm that the connected Linux server has the isolated rendering environment. Then open Remotion effects to request a generation, view successful versions, and adjust parameters.

Save a successful version as an asset, then add it to a cloud template timeline from Remotion assets. Previewing effects and saving assets do not submit a full cloud video composition task.

See the Remotion client documentation for usage and version restrictions.

ASR and Text Segmentation

ASR provides a standalone asynchronous Python function and a command-line entry point. It uses Fun-ASR in the Beijing region to process direct HTTPS audio URLs and returns the original JSON with character and word timestamps. There is currently no standalone HTTP transcription endpoint.

Set DASHSCOPE_API_KEY in server/.env, then run from the repository root:

uv run --locked --project server python -m server.asr "https://example.com/audio.wav"

Replace the example URL with a direct audio URL accessible to the cloud service. The result is written to asr_result.json in the working directory, overwriting any file with the same name.

POST /segmentations uses a correct script and an existing ASR result to produce segments with timestamps and keywords; it does not call ASR. Full segmentation requires a valid script and the original Fun-ASR result with exactly one transcripts entry. Word-level begin_time/end_time values are in milliseconds; returned segment times are in seconds.

Segments longer than 8 characters, excluding punctuation and whitespace, are split further in code. Without ASR, a request containing only title extracts only a title keyword. See text segmentation and ASR transcription for examples and constraints.

Composing Videos through the API

Create a task with POST /api/v1/video-compositions, then query its result with GET /api/v1/video-compositions/{task_id}. You can also provide callbackUrl to receive a terminal-state notification.

The server selects and saves the workflow from the script field and top-level narration URL; do not supply compositionMode:

Input Workflow
Non-blank text and a top-level audioUrl Standard mode; requires videoUrl. Omitting materials uses only the digital avatar, while an explicit array participates in asset matching
No non-blank text, but a top-level audioUrl Media-only mode with narration; optional copy supplies subtitles. Requires a non-empty asset list and composes assets in order, regardless of videoUrl
Neither a narration URL nor a non-blank script Media-only mode without narration; requires a non-empty asset list and a valid target duration, and does not generate subtitles

text and copy are distinct inputs: both being non-blank, or either being non-blank without audioUrl, returns HTTP 422 before task creation. Missing, null, empty, and whitespace-only scripts count as absent. The internal mode is saved in video_compositions.data.request.compositionMode before copy is normalized to text; subsequent processing uses that saved mode. Older tasks without it retain the original URL-based interpretation.

A successful creation response means the request was accepted, not that rendering is complete. Its data field contains the task ID. Save that ID and retrieve the result through the query endpoint or terminal callback.

styleId must identify a saved cloud template. Standard mode still requires a valid script, videoUrl, and audioUrl. Omitting materials skips asset matching; explicitly supplying an empty or non-empty array invokes matching. materials: null is not accepted.

For asset matching, COMPOSITION_PUBLIC_BASE_URL should be an IMV address accessible to the matching service, which uses it to send results back. If empty, the base URL of the composition request is used. The request field callbackUrl is the business-system address that receives the final video result.

Terminal callbacks contain taskId, status, videoUrl, and errorMessage. A successful callback uses succeed, while a successful query result uses succeeded. Notification failures do not revert the composition result. Callers can query through GET as a fallback and handle notifications idempotently by task ID.

Required fields, external services, and subtitle behavior differ by mode. Before submitting a request, read the video composition API documentation, especially the distinction between omitting materials, supplying an empty array, and supplying a non-empty array.

The composition integration also renders dynamic Remotion titles and subtitles into one transparent video and adds it to the IMS Timeline. See the integration and validation notes for the current template-slot behavior and compatibility limits.

Rendering Transparent Overlays

POST /api/v1/overlay-renders renders the fixed Remotion clips saved under a given styleId into one transparent VP9 Alpha WebM, using their timing and stacking order. The output contains no audio, and an accessible URL is returned after upload to ZOS.

The request accepts styleId and optional audioUrl, duration, and callbackUrl fields. Total duration comes first from the full audio duration at audioUrl, then from duration, or otherwise from the latest clip end time. Audio is used only to determine the duration; it is not mixed into the output.

Query the status with GET /api/v1/overlay-renders/{taskId}. This API always requires a Linux sandbox and the rendering dependencies. See the Remotion server documentation for configuration and limitations.


Configuration

Common Configuration Options

Configuration Location Purpose and defaults
VITE_API_URL client/.env Default API URL for source development and builds; defaults to http://localhost:20070 when empty
VITE_PREVIEW_VIDEO_1_URL, VITE_PREVIEW_VIDEO_2_URL, VITE_PREVIEW_VIDEO_3_URL client/.env Default URLs for videos 1, 2, and 3; leave empty for the corresponding 10-second sample bundled with the client, or use a resource path under public/
DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME server/.env MySQL connection settings; configure before using the server
PORT server/.env or process environment API listening port; defaults to 20070, with a valid range of 1–65535
DASHSCOPE_API_KEY server/.env API key for Fun-ASR transcription in the Beijing region
IMV_LLM_BASE_URL, IMV_LLM_API_KEY, IMV_LLM_MODEL server/.env Text segmentation model connection, configured separately from Remotion Agent
IMV_ACTOR_BASE_URL, IMV_ACTOR_API_KEY, IMV_ACTOR_MODEL, IMV_VISION_MODEL server/.env or Debug client Remotion Agent settings Effect generation and vision models; the client can supply configuration for the current task
SEGMENT_MATCH_BASE_URL server/.env External asset matching URL used when standard mode explicitly supplies materials
COMPOSITION_PUBLIC_BASE_URL server/.env Externally accessible server callback base URL; uses the request base URL when empty
ALIBABA_CLOUD_ACCESS_KEY_ID, ALIBABA_CLOUD_ACCESS_KEY_SECRET server/.env or Debug client Shanghai IMS settings IMS rendering credentials; optional ALIBABA_CLOUD_SECURITY_TOKEN; region and endpoint default to Shanghai
ZOS_API_ENDPOINT, ZOS_BUCKET, ZOS_ACCESS_KEY_ID, ZOS_SECRET_ACCESS_KEY, ZOS_WEB_URL server/.env Read only by the server; upload and public access URLs are configured separately

See server/.env.example and client/.env.example for the full field list. Media URL access requirements and asset field constraints are covered in the video composition API documentation.

Standard client settings show only General, with the backend URL and environment information, without requesting the module catalog. Debug mode retains all module settings, including Remotion Agent and Shanghai IMS; database settings are not part of the settings panel. Dynamic forms depend on the configuration schema returned by the backend; displayed defaults are not the actual values in the server .env file.

Save applies only the current page and closes the dialog after success; validation or write failures keep it open with the input preserved. Cancel closes the dialog without saving. Closing discards any other unsaved module drafts without undoing values already saved.

When Changes Take Effect

  • Server running from source: Configuration is read from the fixed server/.env path. Precedence is constructor arguments, process environment variables, configuration file, then field defaults. Restart the server after changes.
  • Frontend environment files: Restart the development server after editing client/.env. Rebuild an already-built client. Other Vite environment files may override variables with the same names.
  • Client backend URL: Save it under Settings → General. New requests use the new URL; restart the client to switch existing session connections.
  • Debug client model and IMS settings: Saved values are used by the corresponding business requests. Previously saved module values are preserved in standard clients; without saved settings, server defaults apply.
  • Advanced settings for the Debug bundled server: Quit and reopen the client after saving; the settings are loaded at startup. This does not apply to manually started or remote servers, and database configuration is outside this settings scope.

Client settings do not modify the shared server .env file. See the client settings documentation for the UI, storage locations, and differences between modes.


Supported Capabilities

Capability Current scope
Desktop client Windows x64, Linux x64, macOS Apple Silicon / Intel
Browser development mode Cloud features through the API
New Remotion effect generation Isolated generation currently requires a Linux server, unprivileged user namespaces, and rendering dependencies
Adding Remotion assets to templates Supported in cloud templates
Standalone ASR Python function and command-line entry point; no standalone HTTP transcription route
Video composition Asynchronous server API; no dedicated client page for complete video composition yet
Composition task scheduling Currently uses a single instance and process; see the server documentation for recovery limitations
Cloud template access The shared template API currently has no login or user isolation

Windows and macOS Debug packages include runtimes for the bundled API, template storage, and existing previews. Bundled rendering dependencies do not mean that the machine supports isolated generation of new effects.

Template Compatibility

Templates require a tracks array, with each object's parameters stored in tracks[].editor. The old top-level editor format is not converted automatically.

Desktop local templates from earlier versions are no longer shown or edited. Their file is left in place and is not migrated; recreate any template you still need as a cloud template.


FAQ

Cloud Templates Fail to Load

Confirm that the server and MySQL are running and that the client backend URL points to the correct API. If server startup fails, check the database connection and database creation permissions. For access from another machine, also confirm that the port is reachable.

Effect Generation Says the Service Is Not Ready

Confirm that the backend has a working model configuration and a Linux environment for isolated validation and rendering. After updating Agent settings, reconnect from the effects page. Advanced bundled-backend settings require a client restart. See the Remotion server documentation for dependencies.

Settings Shows Only General, or the Module List Fails to Load

Standard clients intentionally show only General. In Debug mode, the module list comes from /api/settings/plugins; if loading fails, check the backend URL and service status, then reopen Settings. Browser settings are stored only in page memory and are lost on refresh.

Dependency Installation Reports an Unavailable Version

The server uses the official PyPI index and locks dependencies in server/uv.lock. Check for local uv index overrides and mirror synchronization first. Do not work around the issue by immediately downgrading dependencies or deleting the lockfile.

Where Are Video Composition Logs Stored?

In a source checkout, new task logs are saved under server/.log/video_composition/<task_id>/, covering requests, templates, ASR, segmentation, asset matching, timelines, and ZOS.

Each task has request.json, template.json, asr.json, segmentation.json, matching.json, timeline.json, and zos.json, all organized into input/output/execute_log/error_log. Repeated content may refer to other records in the same task through $log_ref.

Logs are written sequentially in the background. Only when the entire .log directory exceeds 50 MB are task directories removed from oldest to newest, and only if both composition and notification have finished. Running tasks and tasks with queued log writes are retained. Legacy database log tables are no longer read, written, or migrated; business task tables remain.

Historical logs may still exist under server/src/server/.log/; newer versions do not migrate or delete that directory automatically. Script text and signed media URLs in logs may still contain business data. See execution logs for the structure, retention policy, and troubleshooting.

Windows build fails with link: extra operand

This usually means Cygwin's link.exe was used by mistake. In Visual Studio Installer, enable Desktop development with C++ and install the matching MSVC tools and Windows SDK. Rebuild from the Visual Studio Developer PowerShell for the target architecture.

Linux AppImage playback stutters on Wayland

Run these commands separately under the same conditions to compare the Wayland and X11 backends:

IMV_GDK_BACKEND=wayland ./app.AppImage
IMV_GDK_BACKEND=x11 ./app.AppImage

Replace app.AppImage with the actual filename. This option compares display backends; it does not disable hardware acceleration by default. Passing build or artifact checks does not verify playback frame rates on a real desktop.

Linux AppImage cannot open a Chinese input method

The AppImage loads only its bundled GTK input method modules, never the ones installed on the system (their GTK/GLib versions differ, and mixing them can crash). AppImages up to 0.9.1 bundle no fcitx or ibus module, so no input field can switch to Chinese; upgrade to a newer version. The .deb uses the system GTK and is not affected. Newer AppImages bundle the fcitx5 module (which also answers to GTK_IM_MODULE=fcitx) and the ibus module. With another input method framework, start the AppImage with the bundled XIM module:

GTK_IM_MODULE=xim ./app.AppImage

XIM requires the input method framework to set XMODIFIERS (for example @im=fcitx).


Data and Credentials

  • Earlier versions stored desktop local templates at data/template/templates.json under the application data directory (or a custom path set in Settings). The client no longer reads, deletes, or migrates that file.
  • Desktop settings are stored at data/settings/settings.json under the application data directory. API keys are currently saved in plaintext alongside other settings; there is no system credential-store integration yet. Browser settings stay in page memory and are lost on refresh.
  • Model and IMS credentials supplied through client requests are used per task and are not written to the shared .env, work history, or task logs. Logs still contain the actual script and media URLs; check business data before sharing them.
  • New composition tasks upload the finished video and a PNG of its third frame to ZOS and make those objects publicly readable. GET responses and successful callbacks return the video URL. The URL stops working if the object is deleted or removed by a lifecycle policy.

See client settings and video composition for storage details and scope.


Development and Documentation

Project Structure

Directory or file Contents
client/ Rust + Tauri 2 + React + TypeScript desktop client, using Tailwind CSS 4 and shadcn/ui
server/ Python + FastAPI + MySQL server providing template, segmentation, effects, and composition APIs
proto/ Protobuf message definitions shared by the client and server
.github/ Validation, cross-platform build and release workflows, and supporting scripts
docs/ Design documents and API agreements for specific areas

The client is organized into pages, feature components, base UI components, and shared utilities. See client/README.md for the structure and AGENTS.md for minimal-change and source documentation requirements.

Documentation Index

Topic Documentation
Running the client, template editing, and previews client/README.md
Server startup, database, and APIs server/README.md
Text segmentation and keywords Text segmentation
Standalone audio transcription ASR transcription
Video composition modes, fields, and request examples Video composition API
Composition execution logs Execution logs
Client settings, credentials, and when changes take effect Settings module
Remotion page and asset usage Remotion client
Remotion server and rendering environment Remotion server
Transparent Remotion overlay rendering Overlay rendering
Collaboration, structure, and build conventions AGENTS.md
Contribution and PR requirements CONTRIBUTE.md
Template protocol and compatibility

See Template Compatibility for the data format and handling of old files. Template configuration is saved separately from preview media.

The cloud template API accepts Protobuf binary save requests under /template and defaults to JSON for list, detail, and save responses. Responses follow the Protobuf JSON mapping (camelCase fields and RFC 3339 timestamps); legacy clients sending only Accept: application/x-protobuf still receive binary responses. New clients decode either format by Content-Type and ignore unknown JSON fields. The root buf.yaml manages message definitions and compatibility checks; buf.gen.yaml generates client TypeScript and server Python code. After changing the protocol, run from the repository root:

buf lint
buf generate

Commit the generated files and run buf breaking --against <baseline> against an existing protocol version, replacing <baseline> with the actual comparison baseline. See the template API for the interface format.

Pre-commit Checks

Run checks appropriate to your changes:

Change Command Working directory
Frontend core logic bun run test client/
Frontend types and build bun run build client/
Server uv run --locked --project server pytest server/tests -v Repository root
Python package uv build --project server Repository root
Rust formatting cargo fmt --manifest-path client/src-tauri/Cargo.toml --check Repository root
Desktop application and installers bun run tauri build client/
CI / release scripts bun test ./.github/scripts Repository root
Release workflow regression checks bun .github/scripts/release-smoke.mjs Repository root
GitHub Actions workflows actionlint .github/workflows/*.yml Repository root
Client/server version consistency bun .github/scripts/validate-release.mjs Repository root
Basic file checks uvx pre-commit run --all-files Repository root
Diff formatting git diff --check HEAD Repository root

Optionally install local Git hooks with uvx pre-commit install.

For documentation-only changes, check the content, links, Markdown preview, and diff formatting; a full application build is unnecessary. Frontend builds and native compilation checks do not replace installer testing or real desktop playback checks. State which checks you ran and what remains unverified.


Builds and Releases

Standard local installers are written to client/src-tauri/target/release/bundle/, or the corresponding target subdirectory when a target is specified. Workflow entry points are available in GitHub Actions.

Cross-platform CI

Trigger rules, validation scope, and build entry points
Event Checks and builds
Development branch push Checks based on changes; skips duplicate push validation when the same commit already has an open PR
PR Runs frontend, server, and any required native compilation checks across four platform/architecture targets based on changes; does not package installers
Default branch push Validates integration as needed; client or CI changes produce standard installers
main / dev push Debug client additionally produces builds with a bundled backend, independently of standard validation and builds
Official version tag Full cross-platform packaging, attachment validation, and Release publication
Manual run Validate project runs checks, Build client creates standard packages, and Debug client creates packages with a bundled backend

The regular entry point is validation.yml, and builds reuse client-build.yml. The aggregate PR check is named CI result. Branch protection should require it and pre-commit.ci rather than individual platform jobs that may be skipped based on changed paths.

Release preflight explicitly installs the Python version selected by setup-uv before checking the uv lockfile offline; selecting a version alone does not install the interpreter.

When Validate project runs manually, integration-tests is enabled by default and uses temporary MySQL and an isolated rendering environment. Regular push / PR runs do not start these heavier tests. Reports and artifacts are retained for 14 days. Enable build-installers to create installers in the same run. Set api-url for the standard package API URL; leaving it empty defaults to http://localhost:20070.

See the client build CI conventions for scheduling based on client or CI changes, lockfile validation, native checks across four platform/architecture targets, and the integration environment.

Debug bundled backend, data directories, and platform details

Debug client runs on main / dev pushes and supports manual runs; it does not run on PRs. Builds set the bundled-backend flag through debug-backend: true. Standard packages and official releases do not bundle the backend by default. To use the manual web entry point, the workflow must already exist on the default branch; then select the branch to build.

Debug packages include the complete server, Python, uv, a private MySQL instance, Node, Bun, Remotion, Chrome, FFmpeg/ffprobe, and fonts. Linux also includes isolation tools such as bubblewrap/prlimit. First launch extracts resources, initializes the private database, waits for the API to become ready, and connects to its loopback port. A port is assigned automatically if none is saved. Exiting the client shuts down the API and database.

Platform Bundled backend data directory
Linux ${XDG_DATA_HOME:-~/.local/share}/com.intelligentmixvideo.client/backend/
Windows %APPDATA%/com.intelligentmixvideo.client/backend/
macOS ~/Library/Application Support/com.intelligentmixvideo.client/backend/

These directories contain the private database, configuration, and server.log; Remotion data lives in the corresponding remotion/ directory. Initial configuration is created from an example without credentials. Existing configuration and data are preserved, and installers do not include a developer's .env or database. See the build conventions for runtime cache locations and testing scope.

On Unix, the database uses a private socket. On Windows, it uses a random password and a dynamically assigned loopback port. Neither depends on a host MySQL installation. macOS builds are produced separately for Apple Silicon and Intel; Windows uses an x64 native runtime.

Isolated generation of new Remotion effects currently supports Linux servers only. For local development on macOS, set IMV_ALLOW_UNSANDBOXED_WORKER=true on the server to run effect generation without isolation. This switch is off by default and ignored on other systems; when enabled, generated code runs outside a sandbox. Transparent overlay rendering always requires the Linux sandbox and is unaffected by this switch. See the Remotion server documentation for dependencies and configuration.

Linux AppImages exclude bundled Wayland and PulseAudio libraries, use the host versions, and retain the required GStreamer plugins. They bundle the fcitx5 and ibus GTK3 input method modules (the startup hook pins GTK to the bundled module list), and the modules, their libraries and their registration are checked before upload. Startup defaults to WEBKIT_GST_DMABUF_SINK_DISABLED=1 and WEBKIT_GST_USE_PLAYBIN3=1 while preserving explicit user settings. Actual playback behavior still needs verification on the target machine.

Tag Releases

The client and server share one version, and source versions must match the official tag. Before release, use the script to synchronize and commit the manifests and lockfiles. After merging, create the vX.Y.Z tag at the corresponding commit. Builds validate versions rather than rewriting source versions during the build.

Version synchronization and release commands

Run from the repository root. The following v0.6.0 is only an example; replace it with the version you intend to release:

RELEASE_TAG=v0.6.0 bun .github/scripts/validate-release.mjs --write
bun .github/scripts/validate-release.mjs

The script synchronizes the project versions in the client package.json, tauri.conf.json, Cargo.toml, and Cargo.lock, and the server pyproject.toml and uv.lock, while preserving third-party dependency versions. In PowerShell, first run $env:RELEASE_TAG = "v0.6.0", then run the same script.

Review and commit the changes, merge them into main through a PR, and confirm that the local checkout is at the intended release commit before creating and pushing the tag:

git tag -a v0.6.0 -m "Release v0.6.0"
git push origin v0.6.0

Here, origin must point to the project repository that actually publishes releases. Pushing a tag to a personal fork does not release a version of the upstream project.

release.yml runs on official version tags. After successful cross-platform builds, it validates the installers, signed update manifests, UI resources, macOS update archives, and CHANGELOG.md before making the Release public. Release builds require the IMV_UPDATE_PUBLIC_KEY variable and TAURI_SIGNING_PRIVATE_KEY secret (plus the password secret when applicable). Only stable versions are accepted, not -beta / -rc versions, and versions must satisfy Windows MSI numeric limits.

The workflow generates changelogs from Conventional Commits. Publishing uses the built-in GITHUB_TOKEN and requires the appropriate write permissions. Failed runs can be retried and missing draft attachments can be uploaded, but reruns do not overwrite published releases. Do not move published tags.

See the versioning conventions and Release and CHANGELOG conventions for version synchronization, concurrent releases, and recovery.


Contributing

Before starting, read CONTRIBUTE.md and AGENTS.md, check existing issues and PRs, and create a separate working branch from the current contribution target branch.

  • Use English for commit messages, PR titles, and descriptions. Follow Conventional Commits, for example docs: clarify setup instructions.
  • Use the PR template to explain the changes, rationale, validation, and compatibility.
  • Include appropriate tests for features and bug fixes. Documentation should match the actual behavior of its target branch.
  • Preserve the existing license and third-party notices, and review the file scope and configuration content before submitting.

When reporting an issue, include the project version or commit, operating system, reproduction steps, and relevant logs.


License

IntelligentMixVideo is licensed under the GNU Affero General Public License, version 3 only (AGPL-3.0-only). See LICENSE.md for the full terms.

About

No description, website, or topics provided.

Resources

Stars

8 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages