Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

cozy-runtime (v2)

Cozy Runtime supplies the Python author API, model construction and execution helpers, TensorD-launched executors, and the package CLI. TensorD makes model weights available to those executors and coordinates machine budgets, storage lifetime and process lifecycle. TensorFS is a library used inside both TensorD and Runtime.

TensorD is a role of the cozy binary (cozy machine serve, from cozy-creator/cozy); no Runtime wheel carries it. See Publishing Runtime for explicit source and approved-prebuilt releases.

TensorD owns accepted executions and their durable journal behind cozy.machine.v1. Runtime reports executor facts over the private control/descriptor channel. The retired Python worker and its protobuf facade are not part of this topology. See Machine and Runtime roles for the weight-data and memory boundaries, and the tracker for current work.

Compatibility status

Package code uses cozy_runtime.author, the shared numerical helpers under cozy_runtime.derive, and TensorFS's scoped tensorfs.derived API. Underscored author modules, Runtime internal/cli/protocol, and administrative TensorFS Store operations are not package APIs. Package ranges must be requalified for coordinated author API cuts.

Runtime executes packages on the installed standard CPython 3.12, 3.13 and 3.14 window. TensorD embeds its Rust dependencies; each package environment has its own Python closure. Current Runtime Python/TensorFS bounds are in pyproject.toml. TensorFS parses headers and performs weight-plane operations inside the Runtime executor; managed store operations and shared buffer lifetime are coordinated by TensorD. Accepted executions receive scoped source/output descriptors and budget grants. A model source supplies actual metadata and byte access, while an installed library or a model identifier alone supplies no managed-store authority.

cozy-runtime --json python-interpreters reports the packaged rolling policy and actual installed interpreters. By default discovery uses uv python find --no-python-downloads; operators may set COZY_PACKAGE_PYTHONS to a colon-separated finite list of absolute interpreter paths. TensorD preparation selects the lowest compatible installed version from the package's requires-python range and optional exact captured version before installing its complete private dependency closure. No Python download occurs during preparation. Content identity records the actual Python patch and ABI, and retained environments stop reopening when their Python minor leaves the supported window. Package requires-python declarations may remain broader than this execution policy.

Python package compatibility is expressed through dependency ranges. TensorD's machine API and the private executor channel use their own operation/capability contracts; a Python package version is not a machine-wide authority grant.

Published dependency declarations allow patch upgrades, including optional extras. Minimum versions (>=) are preferred; major/minor compatibility ranges are permitted. Exact development and CI versions belong in uv.lock; worker images retain their exact wheel and image digests. The Python ABI bound is independent of dependency version policy.

For media inputs (handles, reads, preparation), see Media inputs.

Layout

package role
cozy_runtime.author typed author API and invocation services
cozy_runtime.internal executor, model loading, weight policy and scoped storage adapters
cozy_runtime.cli package CLI

cozy_runtime.author also holds the INVOCATION KERNEL (_invoke.py): decode -> resolve -> preflight -> hydrate -> invoke -> validate -> finalize. It is pure surface semantics — no device, config, environment, protocol or network — so it lives beside the types it enforces, and internal depends on author, never the reverse. Every door (an entrypoint or a CPU attempt) is a user experience over that one function, and the per-service fakes in cozy_runtime.author.fakes are capability adapters over it rather than a parallel runtime.

internal/config.py is the only module permitted to read process environment: one read at the entrypoint, a frozen RuntimeConfig passed explicitly thereafter. Second read refuses. internal/child_env.py holds the device-child env projection allowlist as data; internal/spawn.py executes the seal (ERASE -> IMPOSE -> SEAL) around it. internal/exits.py is the one shared exit-code matrix.

The executor and the machine

TensorD ──control + descriptors──▶ Runtime executor (Python/PyTorch)

TensorD uses TensorFS to obtain verified stored weights and prepare/retain shared CPU buffers. Runtime constructs the model, plans its weight layout and uses TensorFS's local plane/device APIs for GPU allocation, copies, mapping and unmapping. TensorD may retain exported GPU allocation handles between executors; Runtime attaches those allocations and binds tensor views in its own process. GPU tensors, prepared CPU buffers, and serialized weight objects are different representations with different lifetimes.

TensorFS has no separate daemon in this arrangement. Runtime also has explicit standalone Store helpers for tools/tests; managed executors consume TensorD's scoped sources and buffers.

module role
internal/seam.py bounded control framing and native descriptor transfer
internal/trampoline.py exec-first stage and process settings
internal/executor.py disposable device process and package invocation
internal/weights_writer.py scoped native source/output client and broker
internal/fill.py project source headers into model rows and read plans; explicit standalone Store helpers
internal/weights.py, internal/plane.py executor weight layout, residency policy and TensorFS GPU/CPU buffer operations

One attempt kernel (§8). The invocation kernel has exactly one caller (the device executor).

Observability is bounded, or it is not observability. author/_observations.py holds the ONE record shape, a ring bounded by count AND bytes that sheds oldest first and counts what it shed, and an O(1) time accumulator (Attribution) that makes inference step time attributable without one retained row per step — 324 ns per step against a ~100 ms handler leg. Progress frames are never retained; their durable form is that accumulator plus the monotone position the silence detector reads. Credential-shaped VALUES — signed URLs, bearer and key spellings — are redacted at the emit boundary and counted. Nothing is refused: observability may not fail the thing it observes, and field NAMES are not inspected at all, because the name regex that used to be here false-positived on output_tokens and eos_token.

Progress is one optional call: ctx.progress(step, total) after each step of a loop, under the innermost tel.stage(name). The machine places the whole run's bar from its own time estimate. Invalid progress is omitted and diagnosed on the same bounded lossy lane; it cannot route, settle, bill, or otherwise control an attempt.

Previews are one optional call too: preview(x0, "flux2") (cozy_runtime.author) after a step, with the loop's estimate of the clean latents (or its latents) in a named VAE's space. While a client watches the run, the Runtime projects them to RGB with that VAE's linear factors on their device, at most twice a second, and encodes a small WebP (an animated one for video) off the loop's thread; the machine keeps the newest for the client. With no client watching it returns at once. A shape or VAE it can't show is said once in the run's log and never fails the run.

The machine owns durable run outcomes and output custody. Runtime returns structured executor facts and consumes the native capabilities it is granted. The former Python worker triage ledger, destination-grant table and worker publication service are retired; see the machine API for current control and custody behavior.

The executor's network claim is Python-deep and says so. internal/executor.py:main installs sandbox.refuse_network, so package code that dials is refused typed at the first NETWORK_EVENTS spelling. That bounds a BUGGY package (law 15's same-tenant rung). It is not a kernel bound: the executor runs in the machine's network namespace with no seccomp filter, and a C extension or ctypes reaches connect(2) underneath the hook. The egress arms observe both halves against one reachable address, so the claim cannot quietly grow. Granted input bytes are supplied by the machine and handed over as verified paths; the executor is never given an address.

Progress is observed, never promoted into a timer-driven failure. Advancing monotone positions and measured process work are reported; an unchanged sample stays silent. Process exit, a dead peer, typed outcomes, explicit cancellation and enclosing caller deadlines decide lifecycle; work that is alive but still is shown, not killed (tracker/design/stalls.md). No number of seconds kills anything on its own. Observation cannot reach residency or placement.

Component residency is the runtime's, and the package has no vocabulary for it. An package declares the possible heavyweight set of each public model operation with @uses_components and nothing else. Entering that scope ADMITS and MATERIALIZES the set before the body runs — freeing non-declared components, LRU over refcount-0 leases whose CUDA events have fired, until the plan's activation envelope fits ALLOCATABLE capacity — and leaving it records the fence and makes the bytes evictable. Eviction FREES; there is no host residence tier, and a re-stage is an ordinary cr-005 fill from the page-cached CAS mapping. The vocabulary is unreachable from package code rather than merely unused.

Encoder results are memoized by marking the method. @memoize makes a method eligible; the Runtime looks the call up before the scope opens, so a hit stages nothing. The key is the method's code and declared helpers, the environment, the components' weights, the arguments (values: strings, numbers, host tensors, arrays, images, assets, records) and the numerics. A method drawing from the default RNG refuses memo_rng. Results go to the executor's LRU, and to the machine's disk tier only when small and measured costly to produce (memo: in runtime.yaml; tracker design pipeline-memoization.md).

Runtime's compiled CUDA kernels (kernels/) ride its native wheels, one per OS and CPU (Linux x86_64 and aarch64, Windows x86_64), under cozy_runtime/_kernels: the 8-bit op, FlashAttention-3's forward and RMSNorm + split-half RoPE (author.kernels.rms_rope_split_half), every GPU target's cubins plus PTX, nanobind on the stable ABI, no torch ABI. Each kernel serves once it passes its probe on the card (internal/verify.py). kernels.yaml builds and checks the modules, cached by a hash of everything that shapes their bytes; task wheel adds them to the pure wheel. The pure wheel (macOS, any other platform) carries none.

Every model-specific optimization first benchmarks torch.compile/Inductor as a discovery oracle and records profile identity, graph breaks, generated kernels, compile cost, warm latency, peak VRAM and output equivalence. Reusable fusion becomes a maintained in-package kernel; otherwise qualification may produce one exact AOT artifact. Runtime never compiles on each invocation, and eager execution remains the fallback.

The PlanChooser walks a closed ordered rung set against MEASURED free VRAM. Every coefficient declares its provenance (uncalibrated | bound | measured | ledger); claiming measured without a source refuses. An uncalibrated coefficient may DEEPEN placement — which changes weight placement only, never the bits — and may never refuse. A shortfall deepens to the deepest rung with a quantified confession rather than refusing.

The ledger carries one row per byte class, each declaring its KIND (arithmetic for the worker's own bookkeeping, measured for a named device or kernel interface, unreadable for a class this worker genuinely cannot read — never a zero standing in for one) and the METHOD that produced it. Every attempt closes by reconciling every class against the line taken before its envelope was granted, and the record is recorded.

The package interface

internal/static_interface.py derives the interface from SOURCE (cr-114, decision #713 — package code is untrusted). cozy-runtime describe runs it by default: ast over the module pyproject's cozy.application entry point names, following module-scope imports inside the tree (and cozy-runtime's own cozy_runtime.derive.* source), a constant folder over literals, names, integer arithmetic, tuple(…)/list(…) and one comprehension over a folded sequence, and a CLOSED vocabulary — msgspec.Struct classes with their tag/tag_field/kw_only keywords and msgspec.field(name=…), Annotated carrying msgspec.Meta/FileBound/ModelDefault/Preflight, Literal, Enum/IntEnum/StrEnum, unions, list/tuple/dict/Sequence/Mapping/Set, the asset classes, Files[...] with FileLimits, Tree, ModelArtifact, Model subclasses (encoded_leaves=, min_bits=, phase=, @sequence_parallel, @uses_components), Weights[M], and @app.entrypoint with literal keywords and WeightsOutput(...) tuples. A function's request is its one struct parameter, or else its plain parameters as fields (immutable scalar defaults). A value the package owns as DATA rather than as code — MiniMax-H3's served denoise steps are its committed task plan's, and decision #705 forbids a second copy in serving code — is spelled data_values(__file__, "plan.json", "schedules", "transformer_evaluations"): the reader and the imported app take it from the same bytes through the same cozy_runtime.author function, confined to the calling module's own directory and to JSON scalars, so reading data is never running code (cr-115). Each declaration is rebuilt as a real type (msgspec.defstruct, enum.Enum) and rendered by the SAME internal/schema.py the import path uses, so the two readings cannot drift. Nothing of the package is imported, executed or evaluated, and anything outside the vocabulary (a call or comprehension in an interface position, a type from outside the tree, a star import, computed registration, or a function-local import of an interface name) is a typed refusal naming file:line and the construct: static_unresolvable, static_computed, static_unsupported, static_module. Never a guess.

A package may build its surface out of an INSTALLED cozy package's declarations — a Model subclass, a shared request struct; diagnostics/h3-vae-roundtrip subclasses MiniMax-H3's H3Model and reuses its request type. That dependency's source ships in its wheel, so the reader reads it out of the environment it is itself installed in, confined to distributions declaring a cozy.application entry point, which is what makes one an app. torch declares none, site-packages is never a root, and a sibling package supplies TYPES only: its own callables are its own, never scanned as registrations (cr-117).

A script is an ordinary package (xs-099): its PEP 723 block is its pyproject.toml and it declares app = App() and its functions as any package does, so nothing here reads it differently. Every attempt may call children: the executor's Start carries the call interfaces.

The reader is also what the DEVELOPMENT placement is built from (the worker session's own staging): it supervises and does not execute, so it reads the source and takes its model slots off the interface document.

The static reader is the only describer: publish, install, Calcifer and TensorD all take its document, and nothing derives an interface by importing a package. internal/discovery.py is the executor's import of the ONE installed App export, to serve it; it describes nothing.

internal/package_interface.py is the schema home for PackageInterface: only the launch and invocation facts with current readers. It carries no Runtime-global service inventory, diagnostics, config UI metadata, or tensor requirements, and _fixed_point proves over the built document that no release digest, build timestamp, machine path or mutable selection reached it. The document has one entrypoints list; an entry writes description (its docstring's first paragraph, on one line), internal (a private function) and weights_outputs when set and accelerator only as false, and its memo flag rides invocable.memoize, with invocable.memo_requires naming its optional Seed fields. A slot is {path, class, component_use, encoded_leaves}, plus default, min_bits and phase when declared and weights: true for a Weights[M] argument. descriptor_id (sha256 of one entry) is derived by cozy-runtime describe <function>, never stored. Deleted members are simply unknown fields; there is no legacy decoder or empty-value compatibility shim. Packages declare no GPU count, VRAM floor or lane: the machine picks a slot's lane from its GPU, the request's bits and the slot's min_bits/phase. Actual resource demand comes from the selected lane's tensor schema/encoding, qualification measurements and the worker's observed machine; the RecordOwner owns placement. The interface reader rejects the retired resources key as an unknown field; format /1 has one current grammar and no compatibility decoder. Any function may produce models without another workflow language. A slot's capability is its annotation (cr-077): model: M is constructed in a warm session; source: Weights[M] (bare Weights: any layout) is the raw checkpoint, never constructed, read through components()/tensor()/read_into()/read_values_into() or handed to ctx.tensorfs_source and derive.quantize_artifact. Authors may name a slot's default release through @app.entrypoint(defaults={"model": "sdxl@1.0.0"}): [org/]model@release, never a lane. Omitting the org names a first-party model, the root namespace's (sdxl@1.0.0). A run's model.<slot>= wins over the default. Selection resolves before dispatch; the executor receives exact artifacts. See authored defaults. The same decorators take components={"model": ("dit", "vae")}: the slot's components names the components that callable runs (absent means all), so a machine fetches those first. A binding that selects only some components leaves the others on meta; a method using one refuses component_unbound. A call chooses a callee slot's model with ModelChoice; None keeps the parent's:

await render(prompt=p, unet=ModelChoice(lora=(Lora("org/style", 0.8),)))

ref (a name as cozy run model.<slot>= takes, or a ModelArtifact) replaces the slot's model and its LoRAs. Without ref, lora stacks after the parent's LoRAs for the slot (the request's --lora): the same LoRA again takes the new strength, and strength 0 drops it. TensorD resolves each name as --lora does, composes the call's adapter view and keys its memo by the stack. A machine whose Start lacks model_choices (an older TensorD) refuses such a call: cozy machine update. Known-next calls can prepare their models in the background with prefetch(callee, models={...}) while another model runs, and a request done with its GPUs can release them early with ctx.release_gpus(). Independent items go as one call each, awaited together (asyncio.gather): TensorD runs a request's calls on every GPU it holds, an idle GPU before a busy one and one call per GPU at a time, and keeps each executor warm for the request's later calls. Each WeightsOutput declares only its name and new-byte ceiling; the produced CozyTensors header is the sole tensor-schema and encoding authority. The executor still receives only verified model capabilities, never the foreign source credential or destination authority. Multiple transformations are ordinary code inside one bounded function—there is no model_productions graph, compatibility reader, or second scheduler. internal/canonical.py is the RFC 8785 writer — tensorfs (tfs-013) owns the rules.

The package interface is derived, not committed to the source tree. cozy-runtime --json --dir PROJECT describe writes canonical bytes to stdout from source alone — no torch, no venv, no package import. Creator places those exact bytes at metadata/package-interface.json in the ordinary uploaded package file tree, and cozy package publish refuses when its own static describe disagrees with them. package_interface_digest is SHA-256 over those canonical bytes.

Commands

uv sync
uv run cozy-runtime version          # tag, commit, wire + python semvers
uv run cozy-runtime new <name>       # scaffold a minimal valid package
uv run cozy-runtime describe              # the surface, from source alone
uv run cozy-runtime --json describe       # canonical publication bytes on stdout

Per-package uv environments

Every published package, editable project and unpublished script must use a private uv environment with its complete selected dependency closure, locally and remotely. Baked Python distributions are reuse optimizations, not package version constraints. The accepted migration and mandatory development gates are in Package dependency isolation.

The current implementation still contains inherited-system-site and protected-family policies being replaced under proto-057/cr-186. Do not use those restrictions as a contract for new code. Preserve atomic generations, immutable byte custody, storage admission and executor process/credential isolation while removing image-dependent resolution. Exact captured locks preserve reproducibility; authors declare compatible ranges. Dependency repair must solve and validate the full closure before dispatch.

A model has exactly three lifecycle methods — load(loader), optional warm(ctx), unload(loader) (model-lifecycle.md, #708). warm is the author's one place for post-fill, pre-serving work: the runtime calls it once per construction fill, after the fill and its verification and before the placement is DISPATCHABLE, on every worker including dynamic placements, with the device lease held and real tensors in place — never again on a component stage or evict, and again only when the construction is rebuilt. Whether to compile at all is the author's choice; inside warm the runtime refuses only the name-renaming torch.compile(module) spelling (the in-place module.compile() is allowed), the Context carries the device, the fill's deadline and cancellation and can reserve no package call, and a raise is a typed construction failure (warm_failed, naming the exception) that keeps the placement OFFLINE. Its wall time is the warm leg of the prepare, on the fill's boot record. There is no entrypoint-level warm pass and no ctx.boot_warmup.

A local package's captured wheels are uploaded to TensorD and installed into a generation through its package installer. The executor imports that installed environment. Header-declared model assets travel in the CozyTensors runtime closure and reach package construction only through Loader's bounded read-only asset capability; a temporary private read-only tree exists only for third-party path-based constructors.

A writable delegated cgroup-v2 child remains the preferred executor lifecycle boundary. Ordinary RunPod containers instead expose their provider-owned cgroup-v2 scope read-only. Runtime uses its existing process-tree boundary there only after positively proving the root-worker/dedicated-UID split, root-owned container marker, read-only cgroup lineage and migration controls, finite inherited memory.max and pids.max, full target-UID /proc visibility, and pidfds. Reclaim owns the entire uid-65533 domain, not merely an environment token or process group, so token scrubbing, setsid, double-fork, worker restart, and PID reuse do not leave an old executor alive. A writable but undelegated cgroup-v2 host and an unbounded read-only container still refuse before package import.

Each package environment contains exactly one project wheel and at most 128 dependency wheels. The Runtime bound is 129 total package-owned wheels, counting the project; 129 custom rows plus the project refuses before any grant or ZIP is opened.

Releases may offer multiple project wheels; each environment installs one selected variant. Project and dependency wheels can contain precompiled native code. Runtime checks filename and WHEEL tags against the selected interpreter and host's actual Linux x86-64/glibc compatibility, including CPython-specific and stable abi3 tags. An incompatible interpreter or platform refuses; native bytes labeled with an any platform also refuse. Root-Is-Purelib determines the normal wheel install location and does not assert that every member is Python source.

The same local and remote package installer retains bounded archive expansion, safe paths, exact name/version/tag identity, import ownership, and per-member RECORD SHA-256 validation. Inert source files and headers may accompany prebuilt wheels; installation always passes --no-build and never invokes a compiler or PEP 517 backend. The remote supervisor still owns downloads and storage admission, and receives one exact selected URL/hash for each distribution.

PyTorch, Triton, and the CUDA driver continue to own JIT cache keys and byte validation, but those keys are never treated as a same-UID security boundary. Runtime scopes cache reuse by unpredictable worker boot plus exact package content, then imposes PYTHONPYCACHEPREFIX, TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR, TORCH_EXTENSIONS_DIR, and CUDA_CACHE_PATH before executor import. A respawn within that pod can retain its warm compiler cache; the whole pod scope is deleted after worker shutdown has proven the executor tree gone. Tensorhub's public placement law supplies one public trust domain per pod. Private renter cross-org stacking retains its explicitly documented same-UID risk—Runtime does not relabel worker boot identity as tenant isolation. Cache bytes stay node-local, never enter a package receipt/content identity, and are never published as release artifacts. Every placement consumes this same venv path; OCI provenance belongs to Tensorhub qualification evidence, while the portable receipt names only the exact project/dependency-wheel environment.

On a remote placement, PlacementSet/1 names one exact model_object_set subject. Cozy sends signed logical package/model intent; the worker receives exact digest/length/HTTPS locations directly from Tensorhub over mTLS. The non-control convergence lane starts package and model acquisition together. As soon as the package venv and binding documents are admitted from a standing start, Runtime spawns the executor so interpreter and Torch import overlap continued TensorFS hydration; model preparation/fill still waits for the exact model closure. READY remains the join of package environment, TensorFS closure, binding resolution and device construction. An outgoing placement is never disturbed by this optimization. Verified complete objects survive a re-issue of the same revision; only missing objects move.

PlacementStatus.acquisition exposes that work as observation only. It retains package/model start and end values from one process-monotonic clock plus cumulative downloaded/reused bytes. Partial states are emitted while either leg runs; terminal values remain attached to that placement spec. These values are never read by admission, materialization, convergence, fallback, or readiness code. Cold proof checks interval overlap; warm proof checks zero downloaded package bytes and exact verified reuse, while the real TensorFS cache proof checks non-empty model downloaded/reused accounting.

The fit (model-code-fit §3). TensorRequirements exist only as transient in-memory rows. Runtime executes factory(config) on the meta device in the package interpreter's private bounded child (no network, weight bytes, writes, or CUDA), then discards the rows after use. There is no public derive verb, document, digest, cache, directory, or publication artifact. The serving Loader.construct performs the same census and calls tensorfs.fit directly with requirement objects and the CozyTensors header before reserving a destination. Encoding consent is one boolean policy input, not duplicated per tensor. A non-ok verdict is the typed prepare refusal model_fit_refused. Every PlacementSet writer fills Slot.components from the private construction, never from the header.

A package whose signature names a video or audio input must install cozy-runtime[media]; the executor refuses its import (media_decoder_dependency) if PyAV is absent, since the static reader has no environment to check. Media inputs are handles read through the attempt's one budget; see Media inputs.

An image function declares exactly the inputs its model uses, so a box never offers one that does nothing, and a caller who sends one gets the usual unknown-field warning. ImageRequest holds what every model honours (prompt, aspect_ratio, megapixels, seed); a model that takes pictures subclasses it with prompt: Prompt[Image] and words its references its own way. Steps, Guidance and NegativePrompt type the knobs a function adds.

Image generation uses the shared ImageRequest output-canvas limit: aspect ratios must be between 1:16 and 16:1, inclusive. Explicit ratios outside this range are rejected during request decoding, before admission or model work. A canvas inferred from a reference is checked when target() resolves it, and Sizes.fit() keeps grid rounding within the same envelope. Models can impose narrower bounds. This output-generation rule does not crop or reject source assets merely because they have unusual dimensions; an explicit valid output ratio can use such a reference. Packages that preserve a reference canvas directly must still resolve it through ImageRequest.target() before choosing output dimensions.

A prompt arrives as its text, as typed, and the references it mentions: Prompt(text, references), each reference an Image, Video or Audio of (file, description) under the mention's own spelling ("Amy", "Amy.2", "walk@48-120"), in order of first mention. text.replace(f"<{key}>", ...) is all a package does with them. Prompt[Image] takes pictures only. An Image's file reads upright and scaled down to 16 MP at most, as Picture's did; file.read(max_pixels=) reads it smaller. On the wire the references are a list, each value with its key. Attached files are File, ImageFile, VideoFile and AudioFile; a Latents result field stays on the machine that made it.

A file can carry metadata beside its type: a small JSON map stored with it. An output declares it with file.with_metadata({...}), a copy of the handle; a reader gets it as Image, Video or Audio's metadata. The platform acts on the keys it knows and keeps the rest. An orbit video (a video node turns it instead of playing it) is every frame a keyframe:

clip = out.save_video(frames, fps=24, keyframe_interval=1)
return Turned(video=clip.with_metadata({"orbit": {"degrees": 360, "direction": "cw", "loops": True}}))

Image inputs can separately opt into preparation on a field or an Files argument:

from typing import Annotated
from cozy_runtime.fields import FileLimits, ImagePreparation
from cozy_runtime.media import Files, ImageFile

Pictures = Annotated[
    Files[ImageFile],
    FileLimits(images=3),
    ImagePreparation(max_edge=4096, max_pixels=16_777_216),
]

These example caps are package policy. Still images use Pillow decoding, ImageOps.exif_transpose (including mirrored orientations), RGB conversion and Lanczos resampling. Audio and video retain PyAV. Preparation preserves aspect ratio and never upscales. Creator can run the shared Runtime helper before upload; Runtime applies the same transform to originals sent by other clients. Small inputs keep their original carrier. Oversized inputs use one fast lossless encoding, or the original if it is smaller and admissible. Source files remain unchanged. Byte length, MIME and digest always describe the delivered carrier, so raw and prepared uploads can have different memo keys while decoding to identical pixels.

The helper checks source byte/pixel limits before decoding and uses an isolated 2 GiB address-space limit. That process limit is never applied to a GPU executor. The image-fit/1 declares the geometry policy. Derivative filenames hash freshly produced carrier bytes, so cached images are reused only when the actual output matches. Missing local capability leaves preparation to Runtime. Fidelity remains an independent package-interpreted hint.

A package that prepares, binds, or fills a model selects cozy-runtime[model-execution]. That Runtime capability owns tensorfs>=0.2,<1 from PyPI; package code does not declare TensorFS directly. Every approved base worker image already owns the same distribution, so rental overlay export prunes it and Runtime observes the image-provided facade before execution.

Real-GPU behaviour is verified by hand on a rented card.

Plain transformer components larger than the assigned weight-residence ceiling use their existing _no_split_modules boundaries. Runtime retains common weights and fills one block through the verified TensorFS read plan and bounded pinned ring before its forward. The completion fence must fire before that block returns to meta. During each paged forward, Runtime disables only autocast weight caching and restores the caller's cache setting on every exit. Autocast dtype/enabled state remain unchanged; rematerialized FP32 blocks must not accumulate FP16 cached copies across a multi-tile decode. There is no CPU model copy; the planner reports nominal model bytes separately from the common-plus-largest-block working set. Parameters and persistent buffers remain immutable checkpoint destinations; nonpersistent rotary buffers retain their config-derived initialization. Paged blocks must expose live returned tensors through registered PyTrees. Returned weight views refuse without freeing storage; arbitrary hidden state retaining weight views is not qualified by the no-split declaration or this proof.

An unpublished package runs from captured source without a Tensorhub package release. A local script is the single-file form; editable describes an installation mode. On a rental, an editable checkout is not mounted or watched. Creator builds one exact ephemeral project wheel and dependency-wheel set. TensorD bounds the upload, verifies each wheel's digest and length, then invokes its captured-package installer to build an installed generation. The Python executor receives the installed environment and Runtime grants rather than controller credentials or a watched source checkout.

TensorFS. The fill plane reads through tfs-007's published tensorfs facade. The base package installs and type-checks without model execution, exactly as it does without torch; a fill without the reader is a typed reader_unavailable refusal naming the install, never an ImportError escaping a request.

examples/marco-polo is the weightless reference package — two functions, one CPU-only, no model — and it is what the end-to-end suite builds, installs and runs.

Registration (xs-040). @app.entrypoint registers a public function and @app.entrypoint(private=True) one only its own package's calls run; await call(fn, ...) runs either as its own job (docs/managed-calls.md). Both take weights=, accelerator= (False: CPU only; True or absent: the machine's GPU when it has one), preflight=, defaults=, components=, on_demand= and memoize=. A writer (weights=) is never memoized (memoized_writer); TensorFS derivation replay answers a repeat.

A tool is a struct plus a function (design/tools.md). @app.tool(label=, named=, reserved=, ui=) takes entrypoint's options and registers a public function whose request struct is a box: label names it in the tool list, named a new box (shot makes shot_1), reserved the model's own tags in its text, and ui a component that draws it. On a model's own function it is the shortcut. A box's fields are str, Prompt, bool, a bounded int or float, a Literal or Enum, one top-level AspectRatio, a slot (ImageFile, VideoFile, AudioFile, or Image, Video, Audio and their subclasses, or a list of one), the package's own structs as groups (nested; | None for one, a list for rows), and Seed. Pickup(take, out_frame) is filled only by Pick up from here; a package subclasses it. Markers are msgspec.Meta, so they stack: Range(min, max, step=, unit=), Count(min, max), Total(), Ratios(*allowed, min=, max=), Snap(every, offset, first), Spans(rows), Label(text, options={value: words | (words, description)}), Placeholder(text), Advanced() and When(field, *values): a field that applies only while a sibling field is truthy, or one of values; the box dulls it otherwise, and a run takes its default with a warning. Publish refuses, statically: a required field a box can't draw (an unbounded number, list[str]), two shapes or one in a group, a group holding itself, a result with no file or str for the box to show, and an imported cozy package uv.lock doesn't lock. An optional field a box can't draw keeps its default, off the box, and so does an optional group with a required member a box can't draw. Marker arguments are read from source: literals, arithmetic and module-level constants (STRENGTHS[0] too), never a call, so write Snap(first=124), not Snap(first=frames_for(5)). A keyword request's tuple fields are tuple[int, ...]. The interface carries each tool's tool record and each entry point's calls: the functions it reaches with call(fn, ...), its own and other packages', read at the release it locked, with their models, accelerator and gpus.

Model-producing functions declare WeightsOutput slots and use Context.tensorfs_source, Context.output, and Context.adopt_model. TensorFS owns the typed Derivation, Target, Tensor, Part, and Config declarations and the source/output handles. Inspection keeps the source header's construction order. Native validation locks exact sources, geometry, order and output bounds before accepting values; inherited roles require no payload copy.

Source/config bytes and streamed additions use bounded TensorFS descriptor frames. Runtime exchanges only bounded declaration and receipt metadata; it creates no complete-role spool copy and the executor opens no Store. Added/derived configs are canonicalized into the CozyTensors header. Completed-role inventories and checkpoints come from the native writer.

Runtime's worker assigns the accepted execution's transaction and writer epoch, admits its remaining storage needs, and records native checkpoint/receipt facts in its Workspace before success reaches the executor. ctx.adopt_model(native_receipt) returns the verified ModelArtifact. Source and writer handles are fenced with the attempt, while retained native partial and committed bytes survive process loss for authorized recovery.

Weights.read_values_into provides logical float32 reconstruction over the same TensorFS source handle; its geometry, encoded identities, byte reads and lease lifetime are all TensorFS-owned.

Merging

PRs into master merge through the merge queue: gh pr merge <n> (the queue squashes; never --admin). CI runs there, on the final PR + branch state; nothing runs on a push.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages