High-performance, mode-switching NNTP server host targeting sustained throughput exceeding 40 Gbps.
Current phase: Phase 0.3 — remove obsolete MEL console formatter configuration (Serilog remains exclusive). This repository provides the generic-host composition root, explicit application lifecycle, application-service orchestration, configuration, Serilog-exclusive structured logging, Windows Service support, and production-oriented systemd notify/watchdog integration. NNTP protocol handling and networking remain deferred.
- .NET SDK 10.0 or later (latest stable non-preview)
- Windows, Linux, or macOS for interactive development
- Windows (for Windows Service) or Linux with systemd (for service installation)
VectorNNTP.NNTPD.sln
Directory.Build.props
src/
VectorNNTP.NNTPD/
Program.cs # Composition root
Configuration/ # NntpdOptions + SystemdOptions + validation
Cloudflare/ # DNS API client + reconciliation
Networking/ # Bind-address resolution / IP eligibility
Core/ # Lifecycle, service manager, abstractions
Hosting/ # Generic Host / Windows Service / systemd
Systemd/ # Notify bridge, readiness, watchdog, health
Logging/ # Serilog host integration + log helpers
tests/
VectorNNTP.NNTPD.Tests/
deploy/
systemd/vectornntpd.service # Example unit file
docs/
architecture.md
configuration.md
postfilter.md # POST PostFilter operator guide (NntpDB cluster policy)
schema/postfilter.sql # PostFilter NntpDB DDL
schema/nntpusers-account-art-type.sql # Additive nntpusers.account_art_type ALTER
logging.md
systemd.md
systemd-integration-verification.md
standards/rfcs/ # RFC reference library
standards/spamd/ # Apache SpamAssassin SPAMC/SPAMD protocol
dotnet restore VectorNNTP.NNTPD.sln
dotnet build VectorNNTP.NNTPD.sln -c Release
dotnet test VectorNNTP.NNTPD.sln -c Release --no-build
dotnet format VectorNNTP.NNTPD.sln --verify-no-changesdotnet run --project src/VectorNNTP.NNTPDStop with Ctrl+C (SIGINT) or SIGTERM. The host initiates a single graceful shutdown of application services. systemd notify/watchdog remain inactive outside a systemd notify environment.
tools/Publish-Release.ps1 publishes VectorNNTP.NNTPD, VectorNNTP.StorageServer, VectorNNTP.BackFiller, and NNTPCancelMessage as self-contained x64 Native AOT binaries. One platform directory holds every application, its {entry assembly name}.json file, and a single shared RabbitMq.json.
powershell -File tools/Publish-Release.ps1 -Platform win-64
powershell -File tools/Publish-Release.ps1 -Platform linux-64
powershell -File tools/Publish-Release.ps1 -Platform allArtifacts:
release/win-64/ win-x64 Native AOT
release/linux-64/ linux-x64 Native AOT
The script restores and publishes from the repository root regardless of the current directory. win-64 and linux-64 are the only release platforms. The four applications reference the StuDev.AotAnywhere 1.0.4 MSBuild SDK, so both releases can be produced on this Windows machine. Linux linking uses the Zig toolchain that NuGet restores with that SDK. No Docker, WSL, or global Zig install is required. A globalization-enabled Linux host still needs libicu.
Publish with tools/Publish-Release.ps1 -Platform win-64, then install (elevated PowerShell):
New-Service -Name "VectorNNTP.NNTPD" `
-BinaryPathName "C:\path\to\VectorNNTP\release\win-64\VectorNNTP.NNTPD.exe" `
-DisplayName "VectorNNTP NNTPD" `
-StartupType Automatic
Start-Service VectorNNTP.NNTPDThe process uses Microsoft.Extensions.Hosting.WindowsServices. Interactive console execution continues to work on developer machines.
See docs/systemd.md for full install, operations, and troubleshooting.
Quick path:
pwsh -File tools/Publish-Release.ps1 -Platform linux-64
sudo mkdir -p /opt/vectornntp/nntpd
sudo cp -a release/linux-64/. /opt/vectornntp/nntpd/
sudo cp deploy/systemd/vectornntpd.service /etc/systemd/system/vectornntpd.service
sudo systemctl daemon-reload
sudo systemctl enable --now vectornntpd
sudo systemctl status vectornntpd
sudo journalctl -u vectornntpd -fThe example unit uses Type=notify. Readiness is reported after application lifecycle reaches Running, not merely because the Generic Host constructed. Optional WatchdogSec= is documented but left disabled in the example until operators verify readiness on target hosts.
Real systemd integration was not executed in the Windows development environment used for this phase. Follow docs/systemd-integration-verification.md on a Linux host.
Created → Starting → Running → Stopping → Stopped
↘___________↗
- State transitions are explicit, validated, and observable via
StateChanged. - Invalid transitions throw
InvalidOperationException. - Startup failure or cancellation moves
Starting → Stopping → Stoppedafter rolling back partially started services. - Repeated stop requests are idempotent.
- Lifecycle operations are serialized; concurrent unsafe execution fails predictably.
| Scenario | Behavior |
|---|---|
| Successful startup | Options validated; DNS reconciliation verifies A/AAAA for {Fqdn}; services start in registration order; state becomes Running; systemd READY=1 (when notify enabled). |
| Startup failure | Started services stop in reverse order (including exact-FQDN DNS cleanup when DNS ownership was active); state becomes Stopped; exception is rethrown (host start fails); no readiness. |
| Startup cancellation | Same rollback path; OperationCanceledException propagates. |
| Graceful shutdown | STOPPING=1 (when enabled); reverse-order stop under one overall GracefulShutdownTimeout budget; DNS service removes all records for the exact {Fqdn} after other app services stop (API-verified; recursive caches not waited out). |
| Shutdown timeout | Timeout is logged and surfaced as TimeoutException; DNS cleanup may be incomplete and is not claimed successful; state still ends in Stopped. |
| Unexpected service termination | Logged as critical; watchdog keep-alives stop; host stop is requested when configured. |
| Ctrl+C / SIGTERM | Host lifetime initiates a single shutdown request (no custom competing signal handlers). |
Section: Nntpd (VectorNNTP.NNTPD.json / environment variables / command line). See docs/configuration.md for bind addresses, Cloudflare secrets, and FQDN generation.
| Option | Default | Description |
|---|---|---|
ApplicationName |
Entry assembly name | Optional lifecycle display name. Log files, the Serilog Application property, the Windows service name, and the RabbitMQ connection-name prefix use the entry assembly name directly. |
GracefulShutdownTimeout |
00:00:30 |
Overall wall-clock bound for application-service shutdown (also applied to host shutdown when configured) |
StartupTimeout |
null |
Optional bound for startup; null means host cancellation only |
StopHostOnUnexpectedServiceTermination |
true |
Request host stop when a service execution faults/completes while Running |
Systemd:EnableWatchdog |
true |
Allow watchdog keep-alives when systemd configured a deadline |
Systemd:ReportLifecycleStatus |
true |
Send STATUS= on lifecycle changes |
Systemd:NotifyReadyOnApplicationRunning |
true |
Send READY=1 at lifecycle Running |
Systemd:NotifyStoppingOnApplicationShutdown |
true |
Send STOPPING=1 when shutdown begins |
Systemd:WatchdogIntervalFraction |
0.5 |
Heartbeat interval as a fraction of systemd’s deadline |
Nntpd:BindAddress |
["*"] when omitted |
Listen addresses / wildcards; explicit IPs must be local NIC addresses |
Nntpd:BindPort |
119 |
Cleartext TCP port (1–65535) |
Nntpd:BindPortTls |
0 |
TLS TCP port; 0/unset disables TLS; 1–65535 enables |
nntpsharedconfig.cloudflareapikey |
(MySQL) | Required secret for NNTPD; not nntpd__cloudflareapikey or VECTOR__CLOUDFLAREAPIKEY; never commit or log |
CloudFlareZoneId |
(configured) | Required; nntpd__CloudFlareZoneId may supply it — missing/blank fails startup |
DnsSuffix |
usenet.ninja |
DNS suffix for generated FQDN |
ServerId |
(required; no default) | Required integer 1–99 (nntpd__ServerId); no silent default |
Fqdn |
generated | nntpd{ServerId:00}.{DnsSuffix} — not independently configurable. BackFiller uses the distinct identity backfiller{ServerId:00}.{DnsSuffix}. The nntpd / backfiller prefixes are fixed application identity, not configuration. |
Options are validated at startup via IValidateOptions<NntpdOptions> and data annotations (ValidateOnStart) before the application enters Running.
Serilog is the exclusive logging implementation. Application code uses ILogger<T>; the Generic Host registers SerilogLoggerFactory only (Microsoft Console/Debug/EventLog providers are cleared). Microsoft console formatter options that AddSystemd() may register are removed; Serilog owns console formatting.
See docs/logging.md for bootstrap logging, configuration, journald filtering, and shutdown flushing.
Serilog__MinimumLevel__Default=Debug
sudo journalctl -u vectornntpd -fIn scope (Phase 0 / 0.1 / 0.2 / 0.3)
- .NET Generic Host composition root
- Explicit application lifecycle
- Application service manager
- Options pattern + validation
- Serilog-exclusive structured logging (no Microsoft console formatter configuration)
- Windows Service hosting
- systemd detect / notify / readiness / optional watchdog
- Example unit file + deployment docs
- Automated offline tests for lifecycle, systemd, logging, bind-address resolution, and Cloudflare DNS reconciliation/cleanup (mocked API; no live DNS)
Explicitly deferred
- NNTP protocol parsing and command handling
- Network listeners, sockets, and TLS
- Article storage, indexing, and peering
- Throughput optimizations and the 40+ Gbps data plane
- Metrics servers / telemetry exporters
- NNTP-specific health checks for watchdog
See docs/architecture.md for component responsibilities and the byte-oriented protocol data-plane rule.