Skip to content

About

VectorNNTP — A next-generation, high-performance NNTP platform built with modern .NET and C#, designed for scalable Usenet infrastructure, distributed storage, and high-throughput article processing.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

VectorNNTP.NNTPD

High-performance, mode-switching NNTP server host targeting sustained throughput exceeding 40 Gbps.

Current phase: Phase 0.3 — remove obsolete MEL console formatter configuration (Serilog remains exclusive). This repository provides the generic-host composition root, explicit application lifecycle, application-service orchestration, configuration, Serilog-exclusive structured logging, Windows Service support, and production-oriented systemd notify/watchdog integration. NNTP protocol handling and networking remain deferred.

Prerequisites

  • .NET SDK 10.0 or later (latest stable non-preview)
  • Windows, Linux, or macOS for interactive development
  • Windows (for Windows Service) or Linux with systemd (for service installation)

Solution structure

VectorNNTP.NNTPD.sln
Directory.Build.props
src/
  VectorNNTP.NNTPD/
    Program.cs                 # Composition root
    Configuration/             # NntpdOptions + SystemdOptions + validation
    Cloudflare/                # DNS API client + reconciliation
    Networking/                # Bind-address resolution / IP eligibility
    Core/                      # Lifecycle, service manager, abstractions
    Hosting/                   # Generic Host / Windows Service / systemd
      Systemd/                 # Notify bridge, readiness, watchdog, health
    Logging/                   # Serilog host integration + log helpers
tests/
  VectorNNTP.NNTPD.Tests/
deploy/
  systemd/vectornntpd.service  # Example unit file
docs/
  architecture.md
  configuration.md
  postfilter.md                 # POST PostFilter operator guide (NntpDB cluster policy)
  schema/postfilter.sql         # PostFilter NntpDB DDL
  schema/nntpusers-account-art-type.sql # Additive nntpusers.account_art_type ALTER
  logging.md
  systemd.md
  systemd-integration-verification.md
  standards/rfcs/               # RFC reference library
  standards/spamd/              # Apache SpamAssassin SPAMC/SPAMD protocol

Build and test

dotnet restore VectorNNTP.NNTPD.sln
dotnet build VectorNNTP.NNTPD.sln -c Release
dotnet test VectorNNTP.NNTPD.sln -c Release --no-build
dotnet format VectorNNTP.NNTPD.sln --verify-no-changes

Running interactively

dotnet run --project src/VectorNNTP.NNTPD

Stop with Ctrl+C (SIGINT) or SIGTERM. The host initiates a single graceful shutdown of application services. systemd notify/watchdog remain inactive outside a systemd notify environment.

Release publish

tools/Publish-Release.ps1 publishes VectorNNTP.NNTPD, VectorNNTP.StorageServer, VectorNNTP.BackFiller, and NNTPCancelMessage as self-contained x64 Native AOT binaries. One platform directory holds every application, its {entry assembly name}.json file, and a single shared RabbitMq.json.

powershell -File tools/Publish-Release.ps1 -Platform win-64
powershell -File tools/Publish-Release.ps1 -Platform linux-64
powershell -File tools/Publish-Release.ps1 -Platform all

Artifacts:

release/win-64/      win-x64 Native AOT
release/linux-64/    linux-x64 Native AOT

The script restores and publishes from the repository root regardless of the current directory. win-64 and linux-64 are the only release platforms. The four applications reference the StuDev.AotAnywhere 1.0.4 MSBuild SDK, so both releases can be produced on this Windows machine. Linux linking uses the Zig toolchain that NuGet restores with that SDK. No Docker, WSL, or global Zig install is required. A globalization-enabled Linux host still needs libicu.

Running as a Windows Service

Publish with tools/Publish-Release.ps1 -Platform win-64, then install (elevated PowerShell):

New-Service -Name "VectorNNTP.NNTPD" `
  -BinaryPathName "C:\path\to\VectorNNTP\release\win-64\VectorNNTP.NNTPD.exe" `
  -DisplayName "VectorNNTP NNTPD" `
  -StartupType Automatic
Start-Service VectorNNTP.NNTPD

The process uses Microsoft.Extensions.Hosting.WindowsServices. Interactive console execution continues to work on developer machines.

Running under systemd

See docs/systemd.md for full install, operations, and troubleshooting.

Quick path:

pwsh -File tools/Publish-Release.ps1 -Platform linux-64
sudo mkdir -p /opt/vectornntp/nntpd
sudo cp -a release/linux-64/. /opt/vectornntp/nntpd/
sudo cp deploy/systemd/vectornntpd.service /etc/systemd/system/vectornntpd.service
sudo systemctl daemon-reload
sudo systemctl enable --now vectornntpd
sudo systemctl status vectornntpd
sudo journalctl -u vectornntpd -f

The example unit uses Type=notify. Readiness is reported after application lifecycle reaches Running, not merely because the Generic Host constructed. Optional WatchdogSec= is documented but left disabled in the example until operators verify readiness on target hosts.

Real systemd integration was not executed in the Windows development environment used for this phase. Follow docs/systemd-integration-verification.md on a Linux host.

Lifecycle state machine

Created → Starting → Running → Stopping → Stopped
                ↘___________↗
  • State transitions are explicit, validated, and observable via StateChanged.
  • Invalid transitions throw InvalidOperationException.
  • Startup failure or cancellation moves Starting → Stopping → Stopped after rolling back partially started services.
  • Repeated stop requests are idempotent.
  • Lifecycle operations are serialized; concurrent unsafe execution fails predictably.

Startup and shutdown behavior

Scenario Behavior
Successful startup Options validated; DNS reconciliation verifies A/AAAA for {Fqdn}; services start in registration order; state becomes Running; systemd READY=1 (when notify enabled).
Startup failure Started services stop in reverse order (including exact-FQDN DNS cleanup when DNS ownership was active); state becomes Stopped; exception is rethrown (host start fails); no readiness.
Startup cancellation Same rollback path; OperationCanceledException propagates.
Graceful shutdown STOPPING=1 (when enabled); reverse-order stop under one overall GracefulShutdownTimeout budget; DNS service removes all records for the exact {Fqdn} after other app services stop (API-verified; recursive caches not waited out).
Shutdown timeout Timeout is logged and surfaced as TimeoutException; DNS cleanup may be incomplete and is not claimed successful; state still ends in Stopped.
Unexpected service termination Logged as critical; watchdog keep-alives stop; host stop is requested when configured.
Ctrl+C / SIGTERM Host lifetime initiates a single shutdown request (no custom competing signal handlers).

Configuration options

Section: Nntpd (VectorNNTP.NNTPD.json / environment variables / command line). See docs/configuration.md for bind addresses, Cloudflare secrets, and FQDN generation.

Option Default Description
ApplicationName Entry assembly name Optional lifecycle display name. Log files, the Serilog Application property, the Windows service name, and the RabbitMQ connection-name prefix use the entry assembly name directly.
GracefulShutdownTimeout 00:00:30 Overall wall-clock bound for application-service shutdown (also applied to host shutdown when configured)
StartupTimeout null Optional bound for startup; null means host cancellation only
StopHostOnUnexpectedServiceTermination true Request host stop when a service execution faults/completes while Running
Systemd:EnableWatchdog true Allow watchdog keep-alives when systemd configured a deadline
Systemd:ReportLifecycleStatus true Send STATUS= on lifecycle changes
Systemd:NotifyReadyOnApplicationRunning true Send READY=1 at lifecycle Running
Systemd:NotifyStoppingOnApplicationShutdown true Send STOPPING=1 when shutdown begins
Systemd:WatchdogIntervalFraction 0.5 Heartbeat interval as a fraction of systemd’s deadline
Nntpd:BindAddress ["*"] when omitted Listen addresses / wildcards; explicit IPs must be local NIC addresses
Nntpd:BindPort 119 Cleartext TCP port (1–65535)
Nntpd:BindPortTls 0 TLS TCP port; 0/unset disables TLS; 1–65535 enables
nntpsharedconfig.cloudflareapikey (MySQL) Required secret for NNTPD; not nntpd__cloudflareapikey or VECTOR__CLOUDFLAREAPIKEY; never commit or log
CloudFlareZoneId (configured) Required; nntpd__CloudFlareZoneId may supply it — missing/blank fails startup
DnsSuffix usenet.ninja DNS suffix for generated FQDN
ServerId (required; no default) Required integer 1–99 (nntpd__ServerId); no silent default
Fqdn generated nntpd{ServerId:00}.{DnsSuffix} — not independently configurable. BackFiller uses the distinct identity backfiller{ServerId:00}.{DnsSuffix}. The nntpd / backfiller prefixes are fixed application identity, not configuration.

Options are validated at startup via IValidateOptions<NntpdOptions> and data annotations (ValidateOnStart) before the application enters Running.

Logging

Serilog is the exclusive logging implementation. Application code uses ILogger<T>; the Generic Host registers SerilogLoggerFactory only (Microsoft Console/Debug/EventLog providers are cleared). Microsoft console formatter options that AddSystemd() may register are removed; Serilog owns console formatting.

See docs/logging.md for bootstrap logging, configuration, journald filtering, and shutdown flushing.

Serilog__MinimumLevel__Default=Debug
sudo journalctl -u vectornntpd -f

Current scope

In scope (Phase 0 / 0.1 / 0.2 / 0.3)

  • .NET Generic Host composition root
  • Explicit application lifecycle
  • Application service manager
  • Options pattern + validation
  • Serilog-exclusive structured logging (no Microsoft console formatter configuration)
  • Windows Service hosting
  • systemd detect / notify / readiness / optional watchdog
  • Example unit file + deployment docs
  • Automated offline tests for lifecycle, systemd, logging, bind-address resolution, and Cloudflare DNS reconciliation/cleanup (mocked API; no live DNS)

Explicitly deferred

  • NNTP protocol parsing and command handling
  • Network listeners, sockets, and TLS
  • Article storage, indexing, and peering
  • Throughput optimizations and the 40+ Gbps data plane
  • Metrics servers / telemetry exporters
  • NNTP-specific health checks for watchdog

See docs/architecture.md for component responsibilities and the byte-oriented protocol data-plane rule.

About

VectorNNTP — A next-generation, high-performance NNTP platform built with modern .NET and C#, designed for scalable Usenet infrastructure, distributed storage, and high-throughput article processing.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages