Skip to content

About

An end-to-end automated video production pipeline that deliberately does not automate the publish decision

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

10 Commits

Folders and files

Repository files navigation

Gated Video Pipeline

An end-to-end automated video production pipeline for two children's YouTube channels — that deliberately does not automate the publish decision.

CI Python 3.12 License: MIT

Fen and a heron by the stream at dusk — a frame rendered by the pipeline

A real frame from the pipeline. Characters are drawn in code (React/SVG), not stock assets — see remotion/src/characters/.


Why this exists

On 15 July 2025 YouTube renamed its "repetitious content" policy to "inauthentic content". Channels filled with template-stamped, near-identical AI videos started losing monetization on a fixed path: warning → 90-day suspension → permanent removal from the Partner Program. In December 2025 Screen Culture and KH Studio — 2M+ subscribers and 1B+ views between them — were terminated under exactly this rule.

The naive version of this project is a cron job that generates a video and uploads it. That version gets the channel killed, and the failure is not gradual — you find out after the strike.

So this pipeline automates production and stops at publication. Everything up to the finished MP4 runs unattended; a human decides whether it goes out. The interesting engineering is not the generation, it is the set of gates that refuse to let bad output through.

Gates that cannot be bypassed

Gate Module What it refuses
Cadence core/pipeline.py More than 2 long-form / 2 shorts in a single day, in any phase. A hard ceiling no growth phase can raise.
Variation core/variation_guard.py Scripts, titles, asset combinations or voices that repeat the last 50 videos too closely.
Budget core/budget.py Any job that would exceed the per-video or monthly spend ceiling.
Human approval core/approval.py Every video, without exception. Nothing publishes unreviewed.

None of these are configurable off in a way that matters. REQUIRE_HUMAN_APPROVAL=false is accepted by the parser and then reported as a problem by config.validate() and by otomasyon doctor.

The variation gate is the one worth reading. It compares each candidate against the previous 50 videos on four independent axes and rejects on any one of them:

Axis Threshold
Script similarity (TF-IDF cosine) reject above 0.82
Same title pattern in a row max 3
Asset combination overlap reject above 0.60
Same voice in a row max 4

It runs locally — no API call, no per-check cost — and it runs before rendering, because rendering a video that is going to be rejected costs ~23 minutes on this hardware.

Architecture

flowchart TD
    A[ideation.py<br/>premise + combo reservation] --> B{cadence gate}
    B -->|daily cap reached| STOP1([halt])
    B --> C[script.py<br/>Claude, budget-metered]
    C --> D{budget gate}
    D -->|ceiling exceeded| STOP2([halt])
    D --> E[storyboard.py<br/>scenes, assets, transitions]
    E --> F[narration.py + voice.py<br/>Kokoro-ONNX TTS, local, $0]
    F --> G{variation gate<br/>4 axes vs last 50}
    G -->|too similar| STOP3([halt — before render])
    G --> H[render.py → Remotion<br/>1920×1080, 30fps]
    H --> I{{human approval<br/>REQUIRED}}
    I -->|rejected| K[requeue with reason]
    I -->|approved| L{daily cap + MFK<br/>re-checked at publish}
    L -->|cap reached| STOP4([hold — job stays approved])
    L --> M[publish.py → YouTube<br/>resumable upload]
    M --> N[analytics.py<br/>retention, 7+ days later]
    N -.->|bounded axis weights| A
    M -.->|derived, no LLM cost| P[shorts.py 1080×1920<br/>compilation.py ffmpeg concat]
    P -.->|same approval gate| I

    style N fill:#3d5a3d,color:#fff
    style P fill:#3d5a3d,color:#fff
    style B fill:#8b2f2f,color:#fff
    style D fill:#8b2f2f,color:#fff
    style G fill:#8b2f2f,color:#fff
    style L fill:#8b2f2f,color:#fff
    style I fill:#1f5c8b,color:#fff
Loading

Two ordering decisions carry weight:

  • Narration runs before the variation gate. The gate's voice axis needs to know which voice was used, and voice selection happens during narration. More importantly, scene durations are only real once audio exists — the script's estimates are guesses.
  • The variation gate runs before rendering. See above: a rejection after rendering is 23 wasted minutes.
  • The cadence cap is re-checked at publish, not just at creation. Approved jobs accumulate in the queue. Checking the daily ceiling only when a job is created means five approved videos can still go out on the same day — and "how many were published today" is what the policy actually measures.

That second point was not theoretical. In an end-to-end test the script estimated 66 seconds of narration for a scene; the actual audio came out at 32.5 seconds — a 2× error. Without core/narration.py rewriting scene durations from measured audio length, the video would have carried 33 seconds of silence. Over a 10-minute episode that compounds into minutes of drift.

Engineering constraints

Every architectural decision here traces back to one reference machine: i5-9300H (4c/8t), 16 GB RAM, GTX 1650 with 4 GB VRAM, 17 GB free on the system SSD, 1.4 TB on an external exFAT drive.

No Docker. WSL is broken on this machine (REGDB_E_CLASSNOTREG) and Docker Desktop wants ~10 GB that the system drive does not have. n8n is installed via npm — an officially supported path.

Kokoro-ONNX, not PyTorch. 82M parameters, ~300 MB of runtime, comfortable on CPU at roughly 1.0× realtime. PyTorch + CUDA would have wanted ~4 GB of disk, and Chatterbox does not fit in 4 GB of VRAM. Cost per minute of narration: $0.

Long-form is concatenated, never rendered. 8–12 minute episodes render in Remotion at a measured 13.3 fps; 40–60 minute compilations are stitched from them with ffmpeg concat in seconds. Rendering a 60-minute piece in one pass would take ~2.3 hours.

Code on the internal drive, data on the external one — no exceptions. This rule was broken exactly once and the cost was measured: installing n8n's dependency tree (232,320 files) onto the exFAT external drive wrote 31 MB in 45 minutes before being cancelled. The same install on the internal NVMe finished in minutes. Small-file workloads and USB exFAT are incompatible in practice; large render outputs and model weights are exactly what the external drive is for.

Measured numbers live in docs/benchmarks.md.

Channels

Channel A — Bedtime Channel B — Story Time
Audience age 2–5 5–10
Made for Kids Yes Yes
Length 8–12 min episodes → 40–60 min compilations 10–20 min
Pacing Slow, calm (speech_rate=0.82) Normal narration
Status Active Inactive until phase 4

Both channels are marked Made for Kids because both target audiences are under 13, which is the point at which COPPA stops being a preference. Channel B has the better RPM if it is not marked MFK, and that is precisely the trade this codebase refuses to make — core/config.py hard-codes it and config.validate() raises if any channel targeting under-13s is left unmarked. See docs/content-guidelines.md §1.

Channel B stays inactive until Channel A has 30+ videos and real retention data. One channel is faster to learn from and presents a smaller policy surface.

Both produce in English; YouTube's auto-dubbing (27 languages) covers ES / PT-BR / HI / ID / AR.

Quickstart

git clone https://github.com/codcreater1/gated-video-pipeline.git
uv venv --python 3.12 .venv && .venv\Scripts\activate

The core logic — gates, cadence, database, CLI — installs without the heavy runtime dependencies, so the whole gate suite is testable without downloading an ONNX runtime:

uv pip install -e ".[dev]"

For actual production (Kokoro TTS + YouTube client):

uv pip install -e ".[runtime,dev]"

Copy the config and set at minimum OTOMASYON_DATA_ROOT and ANTHROPIC_API_KEY:

copy .env.example .env

Then verify the environment and initialise storage:

otomasyon doctor && otomasyon init

otomasyon doctor checks Python, ffmpeg, Node, the external drive, config consistency and the database, and tells you exactly what is missing. Full setup — including YouTube OAuth and n8n — is in docs/setup.md.

CLI

Command Purpose
otomasyon doctor Environment and configuration audit
otomasyon init Create data directories and database (idempotent)
otomasyon status Queue state, budget spend, active cadence phase
otomasyon queue List videos awaiting approval
otomasyon review <id> Show the review card and value-test checklist
otomasyon approve <id> Approve and move to the upload queue
otomasyon reject <id> "reason" Reject — the job is requeued, never deleted
otomasyon authorize <channel> YouTube OAuth, once per channel
otomasyon uploads Approved videos not yet uploaded
otomasyon publish <id> Upload to YouTube — the only irreversible command
otomasyon short <id> Derive a vertical Short from a published episode
otomasyon compile Stitch published episodes into a 40–60 minute compilation
otomasyon thumbnail <id> Generate a thumbnail from the episode's own frame
otomasyon run [--json] Run the pipeline end to end — what n8n calls
otomasyon analytics [--refresh] Retention per episode and the axis weights it produces
otomasyon setup-node Fetch portable Node 22 for n8n (leaves system PATH alone)

otomasyon status

review is where the human gate actually lives. It prints the script excerpt, the cost, the Made-for-Kids flag, the variation gate's per-axis notes, and a checklist where a single "no" means rejection:

otomasyon review

Both captures are real output from a pipeline run — only the LLM, TTS and renderer were stubbed, exactly as the test suite does it.

Measured performance

Stage Measurement
Render (vector characters, concurrency 3) 13.3 fps → ~23 min for a 10-minute episode
Narration (Kokoro-ONNX, CPU) ~1.0× realtime, $0
End-to-end (33s output) 114.6 s, 0.58 s A/V drift
Output size ~4 MB/min

Full methodology in docs/benchmarks.md.

Publishing

core/publish.py is the pipeline's only irreversible operation, and it is written to behave like it. Everything before it can be undone: a rejected job returns to the queue, a failed render is retried. A video that reaches YouTube becomes part of the channel's policy surface permanently.

So it produces nothing. It moves an already-approved file and re-verifies three things on the way:

  • Approval. approval.assert_publishable() is called — a contract core/approval.py states explicitly.
  • The daily cap, for the reason above.
  • The Made for Kids flag. config.validate() holds this at the configuration level, but the value that reaches the API is built here. The upload body sets selfDeclaredMadeForKids — the writable field; madeForKids is read-only and sending it would leave the flag silently unset.

The variation fingerprint is committed after a successful upload, never at generation time, so unpublished attempts never pollute the comparison window.

Uploads are resumable in 4 MB chunks with exponential backoff on 5xx — a 40 MB episode that drops at 90% resumes instead of restarting. If the upload fails, the job stays approved and can be retried.

snippet.defaultAudioLanguage is set explicitly: YouTube's auto-dubbing does nothing without it, and leaving it empty would silently cancel the entire 27-language plan.

The feedback loop, and why it is deliberately weak

core/analytics.py closes the only loop in the system: it reads retention for published episodes and weights ideation's axis selection accordingly. Settings, companions and narrative shapes that held attention get picked more often.

The interesting constraint is that this feature must not work too well. "Do more of what performed" is the instinct behind every recommendation-driven content operation, and followed honestly it converges on one setting, one companion, one shape — which is the exact pattern the variation gate rejects and the exact pattern the inauthentic-content policy targets. An unbounded optimizer here does not grow the channel; it terminates it.

So the weighting is clamped in three places:

Bound Value What it prevents
Minimum samples before a value's weight moves 3 episodes One lucky video dominating the pool
Weight range 0.5 – 2.0 The best value is at most twice as likely as neutral — never exclusive
Floor above zero no value reaches 0 Feedback can bias the pool, never prune it

Measurement uses averageViewPercentage, not views. Views scale with subscriber count and whatever YouTube's distribution did that day; they say little about whether a given episode held attention. Nothing is measured until an episode is at least 7 days old — the first days are a notification wave, not performance.

A test asserts that with the maximum weight applied, every value in the pool is still reachable. That test exists because the failure it guards against is silent: a pipeline that slowly narrows until the variation gate stops it.

The asset pack, and a gate that was measuring nothing

storyboard.py derives an asset identifier for every scene — bg/kar_altinda_cam_ormani/dusk, char/uzun_bacakli_balikcil/standing — and the variation gate's visual axis compares those identifiers between episodes.

For a while, SceneView.tsx never read the background identifier. Whether the scene was a snowy pine forest, a beach or a bamboo grove, it drew the same treeline and the same stream. The identifiers differed, so the gate reported variation; the screen showed none. The gate was measuring something that did not exist.

remotion/src/backgrounds/ closes that gap. Each of the 12 settings resolves to its own terrain — and the pack is code, not files, because docs/content-guidelines.md §4 requires consistency by construction: the same identifier renders the same place in every episode, with no image model and no per-frame cost.

Snowy pine forest Bamboo grove
snowy pine forest bamboo grove
Shore Tree hollow
shore inside a tree hollow

Water only appears where it belongs — a stream for Fen's home, a moonlit lake, surf at the shore, and nothing at all in a meadow. Drawing the same water strip into every scene was the single biggest reason the settings looked alike.

Keeping two languages in sync

Identifiers are generated in Python and resolved in TypeScript, in separate files that no compiler checks together. That produces a silent failure: add a setting to the pool, forget the renderer, and it quietly falls back to the default terrain — recreating the exact bug above.

So tests/test_backgrounds.py parses the TypeScript keyword tables and asserts against the Python pools: every setting resolves, every companion resolves, and no two settings collapse onto the same terrain. It also pins the ordering that makes kar_altinda_cam_ormani match snow rather than plain forest — it contains both keywords, and the wrong order silently drops the snow.

Three formats, two of them derived

The episode is the only thing the pipeline generates from scratch. The other two formats are built from episodes that have already been published — which means they already cleared every gate.

Format How it is made LLM cost
Episode (8–12 min) Generated: idea → script → storyboard → narration → render metered, capped at $4
Short (20–59 s) shorts.py — one scene of a published episode, narration trimmed at a sentence boundary, re-rendered vertically $0
Compilation (40–60 min) compilation.py — published episodes concatenated with ffmpeg, stream-copied $0

A vertical Short rendered at 1080×1920

Shorts matter more than their length suggests. docs/content-guidelines.md §1 works through what Made for Kids turns off: Super Thanks, channel memberships, end screens, cards, comments, personalized ads. What survives is Shorts revenue sharing — so on an MFK channel, Shorts are the format that actually earns.

Deriving rather than generating is also the cheaper answer to variation. A Short inherits the variety of the episode it came from, and that episode already passed the gate.

Compilations are assembled, never rendered: a 60-minute single-pass render measures ~2.3 hours on this hardware, while ffmpeg concat with stream copy finishes in seconds because every episode shares one codec, resolution and frame rate.


The variation gate needed a format lane

A Short takes its text from its parent episode. Compared against that episode, script similarity saturates and no Short could ever pass the gate — and it would fail quietly, because the gate would look like it was working.

So comparison is now scoped by format: Shorts compete with Shorts, episodes with episodes. This is a lane, not an exemption — a test asserts that two near-identical Shorts are still rejected, and another asserts that removing the lane brings back the original failure. The real question the gate asks is "is this different from the last 50 of its own kind".

Compilations get their own rule instead, because measuring script similarity on an assembly of its own members is meaningless. What matters there is whether two compilations share too many episodes: republishing the same twelve episodes in a different order is repetition in its plainest form, so overlap above 50% is refused.

Scheduling, and the difference between a gate and a fault

The pipeline is driven by n8n, which calls otomasyon run on a schedule. That hand-off is where automating a gated pipeline gets interesting.

A scheduler has to answer one question every night: did this go wrong? For most pipelines a non-zero exit code means yes. Here it usually means the opposite — the variation gate rejected a near-duplicate, or the daily ceiling was already reached. The pipeline worked. Reporting that as a failure would page someone every time a gate did its job, and a few weeks later the alerts get muted and a real fault goes unnoticed.

So otomasyon run separates the two cases in its exit code:

Code Meaning Should it alert?
0 Video produced, waiting for approval No — notify
10 A gate stopped it No — this is normal operation
1 Something broke Yes

The distinction lives on StageResult.halted, not in string-matching the error text, so each stage declares which kind of stop it is at the point it happens. --json emits the same classification as a stable object (outcome, stopped_at, reason, per-stage detail) for the scheduler to branch on.

The workflows

workflows/ holds three importable n8n exports:

Workflow Schedule What it does
production.json daily 09:00 Runs the pipeline, branches three ways on the outcome
publishing.json daily 20:00 Publishes everything a human already approved
analytics.json Mondays 08:00 Refreshes retention, which re-weights ideation

Every Execute Command node sets onError: continueRegularOutput. Without it n8n treats exit 10 as a failed workflow and aborts — which would undo the whole point of having a separate code for it. A test asserts this on every command node, because the symptom is a workflow that looks broken while the pipeline is working perfectly.

The exports are verified rather than assumed: they were imported into n8n 2.34.5 and exported back, confirming the nodes, connections and onError settings survive the round-trip.

Keeping the workflows honest

Workflows call the CLI by name from a JSON file no compiler checks. Rename a command and n8n keeps running while the command fails every night, silently. tests/test_automation.py parses the exports and asserts every command they invoke actually exists in the CLI, that all three outcomes are branched separately, and that the workflows ship inactive so cloning the repo never starts producing on its own.

Thumbnails — the one place an image model is allowed

Everything the viewer watches is drawn in code, on purpose: the same component renders the same character in every episode, so consistency is guaranteed by construction rather than by prompting. core/thumbnail.py is the single exception, and the interesting part is the boundary that keeps it from swallowing the rule.

A thumbnail is not video content. It is the marketing surface — one still per video, and on a Made-for-Kids channel click-through is the strongest lever left after retention, because MFK disables most of the others.

But generation is never free-form. The model is handed an actual rendered frame of that episode as a reference and asked to recompose it, because free-form generation fails twice over:

  • It would draw a different otter every episode. Fen being constant is the only continuity the brand has.
  • A thumbnail that does not represent the video is a misleading-thumbnail violation — not something to experiment with on a children's channel.

The prompt is built from that episode's own premise (setting, companion), carries the channel's visual language, and explicitly excludes the banned topics from content-guidelines §3. Tests assert all of it without calling the API.

Two smaller decisions worth naming:

It runs before approval, not at publish. A thumbnail generated at upload time is a thumbnail nobody looked at. It appears on the review card, so the human approving the video approves the thumbnail with it.

Cost is recorded, not estimated. OpenRouter prices image output per token, so there is no fixed per-image price to hard-code. The budget gate pre-checks with a deliberately conservative figure, then the ledger records the real usage.cost the API returns.

A generated thumbnail: Fen and a fox in a snowy pine forest

Generated live through OpenRouter for $0.039. Fen's design comes straight from the episode's own rendered frame — the model recomposed it rather than inventing an otter.

Two things the live call taught us

The reference frame is cropped before it is sent. Reference-guided generation preserves the reference's layout, so "crop in tight" was only half-obeyed and the characters stayed too small to read at 320px. Fighting that with a stronger prompt did not work. Cropping the source instead turns the model's layout-preservation from a liability into the mechanism — and where to crop is not a guess, because SceneView places the characters at fixed positions.

The output format is read from the bytes, not from the request. The API was asked for jpeg and returned PNG. Saving that as .jpg and uploading it to YouTube as image/jpeg would have had the thumbnail rejected. detect_format() inspects the magic bytes and picks both the extension and the mimetype from what actually arrived.

Neither was visible from the documentation or from the mocked tests. They only appeared on the first real call — which is the argument for making that call before shipping, not after.

The test suite must not be able to spend money

Once a real key was present in .env, the pipeline's thumbnail stage started making live calls inside the test suite: a full run went from 50 seconds to 287, quietly buying a thumbnail for every test that exercised the pipeline.

tests/conftest.py now blanks every paid-API key for the duration of every test, and a test asserts that guard holds. A "forgot to inject the fake" mistake should surface as a failure, not as a bill.

If OPENROUTER_API_KEY is unset the step is skipped and the pipeline continues — a missing thumbnail is not a reason to consider a video unproduced.

Status

Implemented and tested end to end: ideation with performance feedback, scripting, storyboarding, narration, TTS, the background pack, rendering in both aspect ratios, all four gates, the review queue, YouTube publishing, post-publish analytics, the database, and the CLI — 231 tests, run on every push by CI.

Neither the upload nor the analytics path touches a Google API in tests: the uploader and the metrics fetcher are both injectable, so the tests assert what would be sent and what would be done with what comes back.

Next: Channel B is written and configured but stays inactive until Channel A has 30+ episodes and real retention data. One channel is faster to learn from and presents a smaller policy surface.

Project layout

.
├─ core/               # Python production logic (18 modules)
│  ├─ pipeline.py      #   orchestrator — applies gates in order
│  ├─ variation_guard.py, budget.py, approval.py   # the gates
│  ├─ ideation.py, script.py, storyboard.py        # content generation
│  ├─ narration.py, voice.py                       # Kokoro TTS
│  ├─ render.py        #   Remotion driver
│  ├─ publish.py       #   YouTube upload — the irreversible step
│  ├─ thumbnail.py     #   the one image-model call, reference-guided
│  ├─ analytics.py     #   retention → bounded ideation weights
│  ├─ shorts.py        #   vertical Shorts derived from published episodes
│  ├─ compilation.py   #   40–60 min assemblies, concatenated not rendered
│  └─ config.py, db.py, doctor.py, cli.py
├─ remotion/           # video templates (React + TypeScript)
│  └─ src/
│     ├─ characters/   #   Fen and companions, drawn in SVG
│     └─ backgrounds/  #   12 terrains resolved from the asset id
├─ tests/              # 231 tests, no external-drive dependency
├─ workflows/          # n8n exports — the scheduler side
├─ scripts/n8n.ps1     # n8n launcher
└─ docs/

Heavy data (assets, models, render output, cache, database) lives outside the repo under OTOMASYON_DATA_ROOT. When the external drive is unplugged, otomasyon doctor reports one clear error and skips the data checks — n8n and the test suite keep working.

Documentation

License

MIT — see LICENSE.

Note: source comments and CLI output are in Turkish. Documentation is in English.

About

An end-to-end automated video production pipeline that deliberately does not automate the publish decision

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages