CaddyShortener

This cog makes and removes short URLs with the Caddy admin API. Caddy does all the redirects. The cog does not need a database or a web app.

How it works

1. Caddy serves the short URL domain with a map handler. The map handler changes a request path to a redirect target.
2. The cog sends requests to the Caddy admin API. The requests change the mappings array in the map handler.
3. The cog also keeps its own copy of each short URL in the RedBot config. The cog puts these copies back into Caddy at start-up.
4. Caddy saves the running config to autosave.json. The --resume option loads this file after a restart.

Discord              Caddy admin API            Caddy (go.example.com)
  │                       │                          │
  ├─ [p]shorten ─────────► write /config/.../mappings│
  │                       ├── add redirect rule ─────►
  │                       ◄──── 200 OK ──────────────┤
  │◄── go.example.com/abc ─┤                          │
  │                       │                          │
  │    A user opens go.example.com/abc ──────────────►
  │                       │     ─── 302 → target ────┤

Prerequisites

⚠️ The Caddy admin API has no authentication. Any client that reaches the port controls Caddy fully. It can replace the full config. Therefore, isolate the admin API. Give access only to RedBot. Do not put it on a network that other apps use. Do not map it to a host port.

Set up Caddy

Step 1. Make a private admin network. Only Caddy and RedBot use this network.

docker network create --subnet 172.30.0.0/24 caddy-admin

Step 2. Attach Caddy to the network with a fixed IP. Keep the data. Turn on --resume. Edit the caddy service in docker-compose.yml.

services:
  caddy:
    command: caddy run --config /etc/caddy/Caddyfile --adapter caddyfile --resume
    volumes:
      - /opt/caddy_data:/data       # certificates
      - /opt/caddy_config:/config   # autosave.json (necessary for --resume)
      # ...your Caddyfile and other mounts...
    networks:
      your-proxy-net:               # the network that serves your apps
      caddy-admin:
        ipv4_address: 172.30.0.2

networks:
  your-proxy-net:
    external: true
  caddy-admin:
    external: true

Keep both volumes. The /data volume holds the certificates. The /config volume holds autosave.json. Without the /config volume, --resume has no file to load after you rebuild the container.

Step 3. Bind the admin API to the fixed IP. Add this global block to the Caddyfile.

{
    admin 172.30.0.2:2019
}

Use the private IP. Do not use :2019 or 0.0.0.0. These addresses listen on all interfaces, which includes the network that your apps use.

Step 4. Add the shortener block to the Caddyfile.

go.example.com {
    map {path} {dst} {
        default not_found
    }
    @found expression `{dst} != "not_found"`
    redir @found {dst} 302
    respond "Unknown short URL" 404
}

Put the map directive first in the block. Reload Caddy to load the new block. Use caddy reload --config /etc/caddy/Caddyfile --address 172.30.0.2:2019, or restart the container.

Set up RedBot

Attach RedBot to the caddy-admin network. Keep the other networks.

services:
  redbot:
    networks:
      your-existing-net:
      caddy-admin:

networks:
  caddy-admin:
    external: true

Set the admin URL to the IP, not the caddy hostname. When two containers share more than one network, DNS can give the wrong interface. Use http://172.30.0.2:2019.

Installation

1. Copy the caddyshortener/ folder to ~/Docker/redbot/.

2. Add these lines to docker/Dockerfile.

   RUN mkdir -p /cogs/caddyshortener
   COPY caddyshortener/ /cogs/caddyshortener/

3. Rebuild RedBot.

   docker compose -f docker/docker-compose.yml up -d --build redbot

4. Register the cog path in Discord. Do this one time.

   [p]addpath /cogs
   

5. Load the cog in Discord.

   [p]load caddyshortener
   

First-time setup

Run these commands one time. Only the owner can run them.

[p]caddyset adminurl http://172.30.0.2:2019   # your caddy-admin IP
[p]caddyset domain go.example.com             # must match the Caddyfile block
[p]caddyset allowedrole add @Admins           # who can manage URLs
[p]caddyset channel add #links                # optional: limit to channels

Test the cog.

[p]shorten https://example.com/some/long/path test

The cog replies with https://go.example.com/test → https://example.com/some/long/path. Use [p]caddyset to see the settings. Use [p]shortenhelp to see the cheat sheet. The cog has no default domain. The commands do not work until you set the domain with [p]caddyset domain.

Commands

Configuration (owner only)

Command	Description
[p]caddyset	Show the current settings.
[p]caddyset adminurl <url>	Set the Caddy admin API URL.
[p]caddyset domain <domain>	Set the short URL domain. There is no default.
[p]caddyset allowedrole add <role>	Add a role to the allowlist. The cog stores the role ID.
[p]caddyset allowedrole remove <role>	Remove a role from the allowlist.
[p]caddyset channel add|remove <#channel>	Limit the user commands to specific channels.
[p]caddyset channel clear	Remove all channel limits.
[p]caddyset resync	Send the stored short URLs to Caddy again. Use this after a Caddyfile reload.
[p]caddyset import	Copy the short URLs that are already in Caddy into the store.
[p]caddyset export	Download all stored short URLs as a JSON file.
[p]caddyset restore	Load short URLs from an attached JSON file.
[p]caddyset purge	Remove all short URLs. The cog asks you to confirm.

URL management (allowed roles)

Command	Description
[p]shorten <url> [shortcode]	Make a short URL. The cog makes a random 6-character code if you give none.
[p]unshorten <shortcode>	Remove one short URL.
[p]shorturls	Show all short URLs.
[p]shortenbulk	Make many short URLs. Put one <url> [shortcode] on each line.
[p]unshortenbulk <code...>	Remove more than one short URL in one command.
[p]shortenhelp	Show the cheat sheet.

Security

• No DMs. A cog check stops every command in direct messages. This includes caddyset and shortenhelp. It applies to all users, and to the owner.
• Roles control access. The commands shorten, unshorten, and shorturls need an allowed role. Set the roles with [p]caddyset allowedrole. If you set no roles, only the bot owner can use the commands.
• Channel limit. Use [p]caddyset channel to limit the user commands to specific channels.
• Role match by ID. The allowlist stores role IDs, not names. A user with Manage Roles can copy a role name, so names never give access.
• Owner-only config. Only the owner can use the caddyset group. This includes the admin URL, so the internal address does not reach other users.
• Isolated admin API. The Caddy admin API has no authentication. Bind it to the private caddy-admin network only. See Prerequisites.
• Placeholder guard. The cog rejects URLs that contain {, }, or $. A target such as https://evil.com/{env.SECRET} cannot put a Caddy placeholder into a redirect.
• Input checks. A URL must start with http:// or https://. A shortcode can have up to 20 characters: letters, numbers, -, and _.
• No mentions. Every reply turns off mentions. A crafted URL, shortcode, or role name cannot ping a user.

Troubleshooting

"Could not find go.example.com in Caddy config"

The cog cannot find the map handler. Check these items.

• The go block is in the running config, not only in the Caddyfile. With --resume, an old autosave.json can hide a new Caddyfile. Reload Caddy, or delete /config/caddy/autosave.json, to load the block.
• The admin API is reachable. Make sure [p]caddyset adminurl uses the correct caddy-admin IP.
• The domain in the config is the same as the domain in the Caddyfile. Set it with [p]caddyset domain.
• The map directive is first in the go block. The cog checks for a map handler and writes a warning to the log if it does not find one.

Short URLs disappear after a Caddy restart

The cog keeps its own copy of each short URL in the RedBot config. The cog sends them to Caddy again when the bot starts. Therefore a Caddy restart alone does not lose data after the bot reconnects.

Two cases still need action.

• You reloaded the Caddyfile (caddy reload). This empties the live map. Run [p]caddyset resync to send the stored URLs again. A bot restart does the same automatically.
• A short URL is not in the store. This can happen for URLs from before this feature, or for URLs that you added directly with the admin API. Run [p]caddyset import one time to copy the live URLs into the store.

Keep --resume and the /config volume so that Caddy also keeps its state between restarts. The cog no longer needs them to prevent data loss.

"Failed to read existing short URLs from Caddy"

• Caddy does not run, or the admin API is not reachable.

• Test the connection from RedBot. Change the IP to your caddy-admin bind.

  docker exec redbot python3 -c "import urllib.request; print(urllib.request.urlopen('http://172.30.0.2:2019/config/').read()[:80])"

• If the test fails, attach both Caddy and RedBot to the caddy-admin network, and bind the admin API to that IP.

Change the short domain

Set a new domain. This clears the stored path and finds it again.

[p]caddyset domain go.example.com

File structure

caddyshortener/
├── __init__.py           # setup() entry point
├── caddyshortener.py     # cog class, commands, and Caddy API logic
├── info.json             # RedBot cog metadata
└── README.md             # this file

Under the hood

This section shows how the cog changes short URLs in Caddy.

Step 1. The Caddyfile block becomes JSON

Caddy changes the go block to JSON. The JSON looks like this (made simpler):

{
  "match": [{"host": ["go.example.com"]}],
  "handle": [{
    "handler": "subroute",
    "routes": [
      {
        "handle": [{
          "handler": "map",
          "source": "{http.request.uri.path}",
          "destinations": ["{dst}"],
          "defaults": ["not_found"],
          "mappings": []
        }]
      }
    ]
  }]
}

The mappings array is the important part. It starts empty. Caddy can leave the mappings key out while the map has no entries. Each entry that the cog adds becomes one short URL. The cog manages this one array.

Step 2. Find the map handler

The go block can go into any Caddy server (srv0, srv1, and so on) and at any index in the routes of that server. The server names are not stable. Therefore the cog reads every server. The cog checks that a map handler is at the path before it saves the path.

async def _discover_mappings_path(self):
    # 1. GET /config/ returns the full running config.
    status, config = await self._caddy_request("GET", "/config/")

    # 2. Read every http server. The names are not stable.
    servers = config["apps"]["http"]["servers"]
    for srv_name, srv in servers.items():
        for idx, route in enumerate(srv.get("routes", [])):
            if domain in route["match"][0]["host"]:
                # The map handler is at handle/0/routes/0/handle/0.
                handler_path = (f"/config/apps/http/servers/{srv_name}"
                                f"/routes/{idx}/handle/0/routes/0/handle/0")

                # 3. Confirm the handler is a map before you save the path.
                _, handler = await self._caddy_request("GET", handler_path)
                if handler.get("handler") != "map":
                    return None  # writes a warning to the log

                await self.config.mappings_path.set(handler_path + "/mappings")
                return handler_path + "/mappings"

The cog checks the handler object, not the mappings array. An empty map has no mappings key, so a check of the array would fail. The cog saves the path in the RedBot config. The next command uses the saved path.

Step 3. Read the current mappings

Every command reads the map handler first. The cog reads the handler object, and takes the mappings value. If the key is absent, the cog uses an empty list.

status, handler = await self._caddy_request("GET", handler_path)
mappings = handler.get("mappings", [])

Step 4. Write the full array

The cog writes the whole array back. The Caddy admin API uses two verbs for this. PUT makes a new array. PATCH replaces an array that exists. Therefore the cog uses PUT the first time, and PATCH after that.

# Create the array the first time (empty map, or after a reload):
PUT   /config/.../mappings   [ {"input": "/test", "outputs": ["https://..."]} ]

# Replace the array after it exists:
PATCH /config/.../mappings   [ ...the full new list... ]

[p]shorten reads the array, adds one entry, and writes the array. [p]unshorten reads the array, removes one entry, and writes the array. The cog holds a lock during read and write, so two commands cannot damage the array.

Step 5. Keep the short URLs

The cog stores each short URL in the RedBot config as {shortcode: url}. This store is the source of truth.

• At start-up, the cog sends any missing URLs to Caddy.
• [p]caddyset resync does the same on demand, after a Caddyfile reload.
• [p]caddyset export and [p]caddyset restore move the store to and from a JSON file.

Note - the sync only adds. The sync matches a short URL by its shortcode, not by its target URL. It sends only the shortcodes that Caddy does not have. It does not change or remove a shortcode that is already in Caddy. Therefore, if a shortcode in Caddy points to a different URL than the store, the sync does not correct it. This can happen only if you edit the map directly with the admin API. The commands [p]shorten and [p]unshorten keep the store and Caddy the same. To force Caddy to match the store for one shortcode, remove it and add it again, or run [p]unshorten then [p]shorten.

Caddy also saves its running config to autosave.json. The --resume option loads this file after a restart. If the file is absent, Caddy reads the Caddyfile.

Step 6. Serve a redirect

A user opens go.example.com/abc. The map handler reads the path /abc. The map sets {dst} to the target URL. The @found matcher is true, so Caddy sends a 302 redirect to the target. If the path has no entry, {dst} is not_found, and Caddy sends a 404.

Why this design

Design	Good	Bad
Separate app and DB (YOURLS, and others)	Rich reports and a UI.	One more service to run and back up.
File based (write the Caddyfile, reload)	No API.	Needs file access between containers, and is fragile.
Caddy admin API (this cog)	No extra service. Live changes.	No click reports.