Skip to content

Feeds v4: one Animals feed, pictures by tags, random music from three sources, Modrinth in one feed - #343

Merged
liamt8d merged 6 commits into
mainfrom
feeds-v4
Oct 27, 2026
Merged

liamt8d merged 6 commits into
mainfrom
feeds-v4

Conversation

@liamt8d

@liamt8d liamt8d commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

  • One Animals feed. animals takes the kinds you want (cat, dog, fox, bird, duck, capybara, bunny; English, Spanish and Portuguese words work). None means any. The pace and the kinds live in the account (every:30:cat+dog), so the content is shared by every server with the same pace and choice. The old one-animal feeds keep working.
  • Pictures by tags. New pictures auto feed: one to three tags (pink, cats aesthetic) and Petto posts pictures from the public tags of Flickr. Pinterest has no public search, so it cannot be used. Some words are refused (badTags), and the content is not checked picture by picture.
  • Random music from three sources. Deezer, Apple Music and Spotify (Spotify only when its keys are set). When Spotify fails the reason is logged once every 10 minutes (wrong keys, restricted app…), so it can be told apart from a bug.
  • Modrinth in one feed. new:mod+shader, new:all… follow the newest projects of several kinds in one feed, so one place of the limit is enough. New project notice type (“New project”) next to update.
  • YouTube player option. A feed option player posts the link instead of the card so Discord plays the video (videos and Shorts only).
  • YouTube push, a bit more careful. After a push the channel is looked at again at 1.5, 5, 10 and 20 minutes; a channel that pushes is looked at every 10 minutes (every 30 with more than 1500 channels); a video that reaches the notice more than 8 minutes late says so in the log, with whether a push arrived.
  • Cheaper YouTube search. A one-word name is tried as an @name first (1 unit) before the 100-unit search; answers kept 24 hours; 10 searches a day for each server.
  • Texts in English, Spanish and Portuguese.

Database (it was almost full: 449 MB, of which profile_history_images was 212 MB and malicious_links 57 MB)

  • Avatar copies weigh about half. New copies are WebP at quality 72 instead of 90 (same 256 px). The older ones are rewritten in the background (40 a minute) and only replaced when the new copy is at most 85% of the old one. The copies stop growing at 300 MB (AVATAR_COPIES_MAX_MB, it was 1 GB).
  • Public threat feeds forget what they stopped listing. After a refresh that brought 100 entries or more, the entries of that feed not listed for 14 days are deleted. What a person reported, and what the team was alerted about, stays.
  • Servers Petto left. The date Petto left a server is written down (guild_departures) and, 3 days later, what is stored for it is deleted (settings, everything tied to them, code data). A server that adds Petto back in time keeps everything; one with a Premium slot in use or backups kept in this database waits; nothing is decided while the bot is not ready or has fewer servers loaded than Discord says it is in. GUILD_PURGE=false, GUILD_PURGE_DAYS and GUILD_PURGE_DRY_RUN=true control it.
  • Second database (PETTO_DATA2_DATABASE_URL, optional). The avatar copies and the threat lists can live in a second PostgreSQL database; without the variable nothing changes. With it, the tables are made there, what the main database has is moved over in the background (written in the second database first, deleted from the main one after), reads fall back to the main database while the move is not done, and the dashboard's REST route reads and writes those two tables there. The triggers that deleted a copy with its entry cannot cross databases, so the bot does it: when the history is cleared or turned off (also from the dashboard), when a person has more avatars than are kept, and an hourly pass for the rest. While the threat links move (a few minutes) a known link not moved yet is not seen.
  • Deleting or rewriting rows makes the space reusable, but the size the host shows only goes down after a VACUUM FULL of the table.

How it was checked

npm run check passes (feeds, i18n, types, the full suite, and three new checks against a real PostgreSQL: the servers cleanup, the storage trims, and the second database: the move without losses, reads, new copies, the hourly cleanup and the threat lists; the second database is a schema of the test database there).

Not tested in a real Discord server, with Spotify's real keys, or with a real second Discloud database. The server cleanup deletes data for good: with GUILD_PURGE_DRY_RUN=true it only writes in the log what it would delete.

@liamt8d
liamt8d requested review from kiraio1e and removed request for kiraio1e October 27, 2026 16:54
@liamt8d
liamt8d merged commit 40b1aaa into main Oct 27, 2026
4 checks passed
@liamt8d
liamt8d deleted the feeds-v4 branch October 27, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant