Skip to content

About

One-link downloader for the Mac (videos, documents, images, articles, web pages), with optional automatic YouTube monitoring that translates and subtitles new videos.

Topics

Resources

Stars

140 stars

Watchers

0 watching

Forks

Repository files navigation

Universal Content Intake

中文说明

Universal Content Intake is a download tool for macOS and Windows. Give it one link and it downloads what is behind it: a video, a document, images, an article or a whole web page. The result is filed in your Downloads folder.

On top of that it offers an optional companion feature, automatic monitoring. Name a set of YouTube creators and the system finds their new videos that are taking off, downloads them, translates them and burns in Chinese subtitles, then adds the video details and a publish description. You decide whether to use it the first time you run the tool.

Quick start

macOS (Terminal):

git clone https://github.com/YIAN6557/universal-content-intake.git
cd universal-content-intake
bin/uci https://example.com/report.pdf

Windows (Command Prompt or PowerShell; install Python 3.11+ and Git first, for example with winget install Python.Python.3.13 Git.Git):

git clone https://github.com/YIAN6557/universal-content-intake.git
cd universal-content-intake
bin\uci https://example.com/report.pdf

Everything below works the same on both systems. The examples write bin/uci; on Windows type bin\uci (in PowerShell bin/uci works too).

The first run asks two questions:

  1. Turn on automatic monitoring? Yes starts a guided setup. No keeps the system a pure downloader, and you can switch monitoring on later.
  2. When you download a video, translate it and burn in Simplified Chinese subtitles? The answers are on, off, or ask every time. This is independent of the first question. Videos picked by automatic monitoring always get Chinese subtitles.

It then lists the preparation your choices need, such as checking that ffmpeg and yt-dlp are installed. Change your answers any time with bin/uci settings.

Downloading: one link is enough

bin/uci https://example.com/some-page                       # just a link: the type is detected
bin/uci "download the PDF at https://example.com/report"    # say what you want and it does exactly that
bin/uci "下载这个视频 https://www.youtube.com/watch?v=…"
  • Say what you want and it downloads exactly that. "The PDF at this link", "this video" or "save this page" pick the matching downloader without probing. --type video|pdf|document|image|images|article|webpage does the same.
  • Give only a link and it decides. It checks, in order:
    • known platforms: video sites such as YouTube and Bilibili, Google Drive, image sites;
    • the file extension;
    • what the server returns. HTML pages are then classified as an article or a page to keep whole.
  • Videos. Any of the thousands of sites yt-dlp supports, at 1080p by default (--quality 720p|1440p|4k|highest). Low-resolution sources are never upscaled.
    • Subtitles follow your setting: on adds Chinese subtitles, off downloads the original only, and ask asks you on each download.
    • --zh or --no-zh overrides the setting for one download.
  • Documents. Direct links to PDF, Word, Excel, PowerPoint, archives and more; Google Drive shares; configured rclone remotes.
  • Images. A single picture, or a whole album or gallery.
  • Web pages. Articles are extracted as Markdown. Whole pages are saved as offline HTML, Markdown and PDF.

Downloads land in your Downloads folder under their original title. A short description file sits next to them with the source, author and original link. Multi-file results, such as galleries, get their own folder. --to <folder> saves them elsewhere.

Keeping the download engines current

The downloads rely on open-source engines from GitHub. Setup installs two scheduled background jobs, whether or not monitoring is on:

  • yt-dlp updates itself weekly. Each new version is tried on a real lookup and rolled back if it fails.
  • The other engines are checked every two weeks: gallery-dl, gdown, trafilatura, SingleFile, rclone, aria2, deno, and whisper.cpp for subtitles. Each is compared with its latest GitHub release. When one is newer you get a desktop notification. Nothing is installed automatically. Each update is one command; a whisper.cpp upgrade is checked on a test recording first, and the old version stays if the check fails.

On macOS these are LaunchAgents; on Windows they are tasks in the "Universal Content Intake" folder of Task Scheduler.

Chinese subtitles: who translates

Speech recognition (whisper.cpp) always runs on your computer. Translation:

  • macOS: Apple Translation, on the device and free, by default. You can switch to an online model instead.
  • Windows: an online model, since Windows has no built-in translation. Two are offered: DeepSeek (the default, very cheap) and Qwen (Alibaba Cloud Bailian, with a free quota for new users).

When you turn subtitles on, the tool asks whether to connect a model, explains step by step how to get an API key, tries the key on one sentence and keeps it in the Keychain or Windows Credential Manager. Any time later:

bin/uci setup translation use deepseek   # or qwen; shows how to get a key
bin/uci setup translation key            # paste the key (not shown on screen); it is tested before it is saved
bin/uci setup translation test           # translate one sentence with the saved key
bin/uci setup translation apple          # macOS: back to Apple Translation
bin/uci engines    # check now: each engine's version and how to update it
bin/uci status     # the result of the last check

Companion feature: automatic monitoring (optional)

 Creators ─► periodic discovery ─► early view tracking ─► rules + optional AI editorial check ─► daily pick
                                                                                                   │
 Delivery folder ◄─ burn-in ◄─ translation ◄─ subtitles / on-device speech recognition ◄─ local worker
  • In the cloud (Google Apps Script + one Google Sheet).
    • During your discovery window it checks the creators every 10 minutes and records views at 30, 60 and 120 minutes after publishing.
    • It compares each video with that creator's own history to find the ones taking off.
    • It applies your length and title rules. With Gemini connected, it also checks each video against your editorial direction.
    • Once a day it picks one or two videos for your computer.
  • On your computer (a background worker). It downloads the picked videos, translates them, burns in Chinese subtitles and delivers each one with a publish sheet: title, copy, hashtags and source.

Turning it on and off:

bin/uci settings --monitoring on     # on; if it has never been set up, run bin/uci setup as prompted
bin/uci settings --monitoring off    # off: stops the local worker and pauses the cloud; creators and settings are kept
bin/uci status                       # how automatic monitoring is running, and whether the engines are current

Setup needs a Google account, and a wizard walks you through it in about 30–60 minutes. No YouTube API key is needed. The cloud part uses your own Google authorization, with a free quota of 10,000 units a day, of which a typical setup uses one to two thousand. A free Gemini API key is recommended, which takes about two minutes to create. It filters videos by your editorial direction and writes the publish copy.

Requirements

Feature Needs
Downloading (everyone) macOS 15 or newer (Apple silicon recommended) or Windows 10 (1809+) / 11; Python 3.11+, ffmpeg, yt-dlp, deno; gallery-dl for images, SingleFile and Chrome for whole pages, rclone for cloud drives
Subtitles on macOS (when set to on or ask) Xcode Command Line Tools; the wizard downloads the ~490 MB Whisper speech model and builds whisper.cpp and the translation helper; Apple Translation language pack (English → Simplified Chinese), or a DeepSeek / Qwen API key
Subtitles on Windows (when set to on or ask) the wizard downloads the ~490 MB Whisper speech model and whisper.cpp's ready-made Windows build; a DeepSeek or Qwen API key
Automatic monitoring (when on) All of the above, plus a Google account and Node.js (for clasp); a free Gemini API key is recommended

bin/uci setup doctor lists anything missing with the command to install it (Homebrew on macOS, winget on Windows). Tools for features you have switched off are marked as not needed.

Settings and setup

bin/uci settings          # show or change the two choices
bin/uci setup             # guided setup: only the steps your choices need, and what to do next
bin/uci setup verify      # acceptance check: a trial download

Every wizard step says whether an agent can do it or you must do it yourself, for example signing in to Google, approving access or pasting a secret. SETUP.md walks through every step. If you use a coding agent such as Claude Code, point it at SKILL.md. It asks you the two questions first, then guides you through setup.

The older bin/uci-get, bin/uci-setup and bin/uci-status still work as aliases of bin/uci get, bin/uci setup and bin/uci status.

Where settings live

Where What
~/.config/universal-content-intake/config.yaml the two choices (features.monitoring, features.video_subtitles); downloads, delivery and work folders; video quality (1080p by default); Chrome sign-in fallback (off by default); cloud endpoint
Google Sheet → Config (monitoring; edit with bin/uci setup config set key=value) timezone, discovery window, final sweep, selection time, daily count, latest start time for the second pick, view thresholds, maximum length, title rules, Gemini check and your editorial direction
Google Sheet → Creators (monitoring; edit with bin/uci setup creators …) the creator whitelist
macOS Keychain / Windows Credential Manager the shared secret between your computer and the cloud; the DeepSeek / Qwen key for subtitles; optional Gemini / Anthropic keys for publish copy (bin/uci setup apikey gemini)
Apps Script → Script Properties the same shared secret; optional UCI_GEMINI_API_KEY

Your creator list and editorial direction stay in your own Google Sheet and local config; none of it is stored in this repository.

Privacy and security

  • Downloads are anonymous. Chrome's sign-in state is used for sign-in-gated videos only when you enable video.allow_browser_cookies.
  • Speech recognition runs on your computer, and so does Apple Translation on macOS. With an online model (always on Windows) the subtitle text is sent to DeepSeek or Qwen for translation. The optional Gemini check sends video titles, descriptions and statistics to Google. The optional publish writer also sends translated subtitle excerpts.
  • With automatic monitoring, every request between your computer and the cloud is signed with HMAC-SHA256. The secret lives only in your Keychain or Credential Manager and in Script Properties, and the wizard never displays it.

Responsible use

Download, process and publish content only where you have the rights or permission to do so, and follow each platform's terms of service and the laws that apply to you. The software is provided as is, and its authors accept no liability for how it is used.

Development

python3 -m unittest discover -s tests   # Python
node --test tests/*.js                  # Apps Script (run in a Node VM)

GitHub Actions runs the Python tests on macOS and Windows, plus real downloads, a full subtitle run and the background tasks on Windows (.github/workflows/tests.yml). What differs between the two systems lives in src/core/compat.py.

The per-stage command-line tools and module ownership are described in docs/REFERENCE.md.

License

Universal Content Intake is dual-licensed:

Use by companies, organizations, studios, teams, employees, contractors or for client projects counts as commercial use. So does use for paid services, commercial content, marketing, advertising, brand communication, or integration into products and workflows. Internal use, trials, not yet being profitable, or not charging a client separately do not make a use non-commercial.

Third-party components are not relicensed by this repository; see THIRD_PARTY_NOTICES.md.

Maintainer: @YIAN6557

About

One-link downloader for the Mac (videos, documents, images, articles, web pages), with optional automatic YouTube monitoring that translates and subtitles new videos.

Topics

Resources

Stars

140 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages