Agent Mail Proxy

A Go-based intermediary that proxies SMTP and IMAP protocols for AI agents, keeping upstream credentials secure and enforcing content-based email filtering rules.

Overview

Agent Mail Proxy sits between an AI agent (or any local client) and one or more real email providers (e.g., Gmail, Outlook). It accepts unencrypted local connections, filters every email by sender, receiver, subject, and body content using configurable regex rules, and forwards only authorized traffic upstream. Blocked emails are silently dropped — the sender sees a success response while the blocked content never reaches the upstream provider.

Features

• Dual-protocol proxy — SMTP for sending, IMAP for reading
• Multi-upstream support — Route different authenticated users to different email providers
• Content filtering — Regex-based rules on sender, receiver, subject, and body
• First-match-wins rule engine — Ordered rule evaluation with AND logic per rule
• Silent denial — Blocked SMTP messages return 250 OK; blocked IMAP messages are stripped from FETCH responses
• Structured JSON logging — All actions logged via zerolog with configurable output (stdout, file, or both)
• External rule sources — Load filter rules from local files or remote URLs
• Per-upstream filter rules — Different filter rules per upstream provider, loaded from separate files or directories
• Environment variable overrides — Per-upstream credentials can be set via environment variables for CI/CD and production
• Virtual sequence numbering — IMAP state manager remaps sequence numbers so clients see a contiguous range
• Log rotation — Built-in file rotation via lumberjack

Prerequisites

• Mise — development toolchain manager

Quick Start

# Install Go and tools, then initialize dependencies
mise i
mise run init

# Build the binary
mise run build

# Run tests
mise run test

Configuration

All configuration is in YAML files under the src/ directory.

src/config.yaml

Controls proxy listeners, authentication, logging, global filter rules, upstream definitions (or references), and user-to-upstream mappings:

# Optional: load upstreams from a separate file instead of inline
# The UPSTREAMS_FILE environment variable overrides this field
upstreams_file: "./upstreams.yaml"

smtp:
  listen: "127.0.0.1:1025"  # Local SMTP listener

imap:
  listen: "127.0.0.1:1143"  # Local IMAP listener

# Proxy-level auth. REQUIRED when multiple upstreams are configured.
auth:
  enabled: true
  users:
    - username: alice
      password: secret1

# Upstream definitions (can also be loaded via upstreams_file)
upstreams:
  # Global filters directory: place files named after each upstream here
  filters_dir: "./filters/"

  entries:
    - name: gmail
      username: user@gmail.com
      password: app_password
      smtp_upstream: "smtp.gmail.com:587"
      imap_upstream: "imap.gmail.com:993"
      # Optional per-upstream filter file (highest precedence)
      filters_path: "./gmail-specific.yaml"
      # Optional per-upstream filters directory
      filters_dir: "./gmail-filters/"

    - name: outlook
      username: user@outlook.com
      password: app_password
      smtp_upstream: "smtp.office365.com:587"
      imap_upstream: "outlook.office365.com:993"

# Map authenticated proxy users to upstream names
mappings:
  - auth_user: alice
    upstream: gmail

filters:
  default_action: allow           # What to do when no rules match
  rules:                          # Ordered list — first match wins
    - direction: outbound
      action: deny
      sender_regex: ".*@spamdomain\\.com"

src/upstreams.yaml (Optional)

Upstreams can be defined inline in config.yaml or loaded from a separate file. The file uses the same upstreams structure:

filters_dir: "./filters/"

entries:
  - name: gmail
    username: user@gmail.com
    password: app_password
    smtp_upstream: "smtp.gmail.com:587"
    imap_upstream: "imap.gmail.com:993"

Environment Variable Overrides

Environment variables override file values per upstream. The upstream name is normalized to uppercase with non-alphanumeric characters replaced by underscores.

Variable	Overrides
MAILPROXY_<NAME>_USER	Username for upstream <name>
MAILPROXY_<NAME>_PASS	Password for upstream <name>
UPSTREAMS_FILE	Path to the separate upstreams file

For example, for an upstream named gmail:

• MAILPROXY_GMAIL_USER
• MAILPROXY_GMAIL_PASS

For an upstream named office-365:

• MAILPROXY_OFFICE_365_USER
• MAILPROXY_OFFICE_365_PASS

Filter Rules

Rules are an ordered array evaluated top to bottom. The first matching rule wins.

Within a single rule, all specified conditions must match (AND logic). Unspecified fields act as wildcards.

Field	Description
direction	inbound or outbound
action	allow or deny
sender_regex	Regex against the sender email address
receiver_regex	Regex against recipient email addresses
subject_regex	Regex against the email subject
body_regex	Regex against the decoded plain-text body

Per-Upstream Filter Precedence

Each upstream can have its own filter rules with higher precedence than global rules. The order is (most specific → least specific):

1. upstream.filters_path — explicit file for this upstream
2. Global upstreams.filters_dir/<upstream-name>.yaml — named file in the global filters directory
3. upstream.filters_dir/*.yaml — all .yaml files in the upstream's own filters directory (sorted alphabetically)
4. Global config.yaml filters.rules + filters.rules_source

External Rule Sources

In addition to inline rules, you can load rules from external sources:

filters:
  rules_source:
    - type: file
      path: "/path/to/extra-filters.yaml"
    - type: url
      url: "https://example.com/filter-list.yaml"
      refresh_interval_minutes: 60

External rules are fetched at startup and appended after inline rules.

Usage

Run the proxy

mise run debug     # Run with Delve debugger (breakpoints supported)

Or build and run directly:

cd src
go build ./cmd/agent-mail-proxy
./agent-mail-proxy

Available commands

Command	Description
mise run init	Run go mod tidy to install dependencies
mise run build	Compile the binary
mise run test	Run all unit tests with verbose output
mise run debug	Build and run with dlv debug (Delve)

Project Structure

agent-mail-proxy/
├── cmd/agent-mail-proxy/main.go    # Entry point
├── internal/
│   ├── config/config.go            # YAML config & upstreams parsing
│   ├── log/log.go                  # zerolog + lumberjack initialization
│   ├── mail/parser.go              # RFC5322 email parsing (net/mail + enmime)
│   ├── filter/engine.go            # Rule evaluation engine
│   ├── state/disk.go               # IMAP sequence number mapping (disk-backed JSON)
│   ├── state/manager.go            # IMAPStateManager interface
│   ├── upstream/smtp.go            # SMTP upstream forwarding
│   ├── upstream/imap.go            # IMAP upstream connection
│   ├── proxy/smtp.go               # go-smtp backend implementation
│   └── proxy/imap.go               # go-imap backend implementation
├── pkg/models/email.go             # Core data types
├── src/config.yaml                 # Main configuration file
├── src/upstreams.yaml              # Optional separate upstreams file
├── mise.toml                       # Mise tool & task definitions
└── go.mod

Architecture

Single Upstream

┌──────────────┐     unencrypted      ┌───────────────────┐     TLS      ┌──────────────┐
│   AI Agent   │ ──────────────────▶  │  Agent Mail Proxy │ ──────────▶  │  Upstream    │
│  (or client) │ ◀──────────────────  │  (localhost)      │ ◀──────────  │  (Gmail etc) │
└──────────────┘                      └───────────────────┘              └──────────────┘
                                              │
                                              ▼
                                     ┌───────────────────┐
                                     │   Filter Engine    │
                                     │  (regex matching)  │
                                     └───────────────────┘

Multiple Upstreams (with Auth)

                                    ┌───────────────────┐
                                    │  Agent Mail Proxy │
                                    │   (localhost)      │
                                    └─────────┬─────────┘
                                              │
                     ┌────────────────────────┼────────────────────────┐
                     │                        │                        │
                     ▼                        ▼                        ▼
            ┌──────────────┐         ┌──────────────┐         ┌──────────────┐
            │   Upstream A   │         │   Upstream B   │         │   Upstream C   │
            │  (Gmail)       │         │  (Outlook)     │         │  (Custom)      │
            └──────────────┘         └──────────────┘         └──────────────┘

• SMTP flow: Client connects → (authenticates if enabled) → sends email → proxy resolves user → upstream filter engine evaluates → allowed: forwarded to mapped upstream / denied: 250 OK (silent drop)
• IMAP flow: Client connects → (authenticates if enabled) → proxy resolves user → connects to mapped upstream → commands forwarded transparently → FETCH responses intercepted → each message filtered by per-upstream engine → denied messages stripped → sequence numbers remapped

Development

Running tests

mise run test
# or directly:
cd src && go test ./... -v

Building

mise run build
# or directly:
cd src && go build ./cmd/agent-mail-proxy

Debugging with Delve

mise run debug

This launches the proxy under dlv debug, allowing you to set breakpoints and step through code.

Adding a dependency

cd src
go get <module-path>
go mod tidy

Key Design Decisions

• First-match-wins — Rules are an ordered array; no precedence integers
• Silent denial — Blocked emails are dropped without alerting the sender
• Multi-upstream via auth mapping — Authenticated users are routed to different upstreams by name
• Per-upstream filter engines — Each upstream gets its own compiled rule set with highest-precedence rules first
• Separate upstreams file — Keeps credentials out of the main config and enables easier secret management
• Env var overrides per upstream — Enables credential injection in CI/CD without file modification
• Extensible state backend — IMAPStateManager interface allows swapping disk for Redis later
• External rule sources — Supports file and URL-based rule loading at startup

Future Enhancements

• Redis-backed IMAP state management
• API-based upstream providers (SendGrid, Mailgun, Microsoft Graph)
• Dynamic rule hot-reloading with refresh intervals
• HTML body content stripping for filtering
• Prometheus metrics / observability export