This document is the configuration contract for the standalone BackFiller host. Property names below match the current options types. Defaults and ranges below are the values those types and their validators enforce today.
Startup registers options with ValidateOnStart in BackFillerServiceCollectionExtensions. Invalid mandatory configuration fails startup. Do not log a complete options object: ACME, Cloudflare, RabbitMQ, and NntpDB values include secrets.
Minimum supported physical RAM: 8 GiB (8,589,934,592 bytes). Intended production hosts are 32–64 GiB. BackFillerOptionsValidator probes total physical memory through IPhysicalMemoryProvider at ValidateOnStart. Detection failure or a total below the minimum fails startup with a message that states the requirement and, when known, the detected byte count. The check is not skipped for any BackFiller hosting path that uses AddBackFillerHosting.
BackFiller:ArticleRetention:MaximumRetainedPayloadGigabytes is an integer gibibyte count and must stay within 80% of detected physical memory, truncated down to whole gibibytes with no artificial floor. On an 8 GiB host that ceiling is 6 GiB, so the default of 4 GiB remains valid. Host eligibility and retention-ceiling failures are reported as separate messages.
VectorEnvironment loads environment variables with the prefix VECTOR__. After the prefix is removed, VECTOR__RABBITMQ__USERNAME binds to RabbitMQ:Username. The prefixes nntpd__ and backfiller__ are not registered and are not aliases.
The host content root is the application binary directory. Common ApplicationJsonConfiguration removes Generic Host appsettings.json, appsettings.{environment}.json, and *.settings.json sources and loads these files instead:
| File | Contents |
|---|---|
VectorNNTPD.BackFillerService.json |
Primary application configuration. The name is {entry assembly name}.json. |
VectorNNTPD.BackFillerService.{environment}.json |
Optional environment overlay. |
VectorNNTPD.RabbitMQ.json |
RabbitMQ connectivity. It is a separate file. Its section name remains RabbitMQ. |
The replacement keeps the position of the earliest removed appsettings source. When the host registered none, the entry-assembly files are inserted at the start of the source list. Optional and reload-on-change are copied from the matching removed source. When that source was absent, the added file is optional and reloads on change. A second call does not add another copy of the same path. The helper does not change the builder base path and does not create the files.
VectorNNTPD.RabbitMQ.json is the only RabbitMQ JSON file BackFiller registers. It is inserted immediately after the entry-assembly JSON sources. Its optional and reload-on-change flags are copied from the last of those sources. A second call with the same filename does nothing. A present RabbitMq.json is not read. A missing VectorNNTPD.RabbitMQ.json follows the copied optional flag and does not select another filename. Environment variables and command-line arguments remain after the JSON files and override the same keys. A missing optional file does not fail configuration loading. A required primary file that is missing fails with FileNotFoundException when the configuration is built. Invalid JSON in a present file fails when that source is read (InvalidDataException). Mandatory option validation still runs at startup.
Section name: BackFiller (BackFillerOptions.SectionName).
| Key | Required | Notes |
|---|---|---|
ServerId |
Yes | int?, range 1–255. Missing and 0 both fail. There is no default. Old key BackFiller:Id is not the current key. |
DnsSuffix |
No | Binder default usenet.ninja. A successful nntpsharedconfig projection replaces it. Whitespace fails when the runtime snapshot canonicalizes the suffix. |
CloudFlareZoneId |
Shared config | The property remarks say it binds from BackFiller:CloudFlareZoneId only. BackFillerOptionsValidator does not itself reject an empty value. NntpSharedConfigurationReader rejects a blank cloudflarezoneid column, and AcmeCloudflareOptionsValidator rejects a blank zone id when shared database fields are validated. ProviderAccountConfigurationService copies the shared zone id onto BackFillerOptions. |
BindAddress |
No | Omitted or empty means all interfaces. Explicit addresses must be locally assigned. Wildcard tokens include *, 0.0.0.0, and ::. |
BindPortTls |
Yes | int?, range 1–65535. BackFiller is TLS-only. There is no BackFiller:BindPort. |
AcmeDirectoryUrl |
No | Binder default is the Let's Encrypt staging directory. ACME validation requires an absolute HTTPS URL. A successful nntpsharedconfig projection replaces the bound value. |
AcmeRenewalThresholdDays |
No | Binder default 30. ACME validation accepts 1–90. A successful nntpsharedconfig projection replaces the bound value before renewal uses it. |
AcmeStateDir |
No | Default certs/. ApplicationLocalPath in Common resolves a relative path against the application binary directory and rejects it when that canonical path leaves the directory. Rooted paths stay rooted. |
CertificateDirectory |
No | Default certs. Old key DirCerts. |
BackFillerAccountRefreshIntervalSeconds |
No | Default 60. Range 5–3600. This is the NntpDB account poll. It is not the per-account NNTP DATE keepalive. |
Generated Fqdn is backfiller{ServerId:00}.{DnsSuffix}. It is not a configuration key. It is empty when ServerId is missing or out of range, or when DnsSuffix is empty.
BackFillerRuntimeOptions is built from the bindable options at DI construction time, so its FQDN starts as that config-file provisional name. When ProviderAccountConfigurationService publishes nntpsharedconfig, Project sets BackFillerOptions.DnsSuffix, AcmeCloudflareOptions.Fqdn / DnsSuffix, and calls BackFillerRuntimeOptions.ApplySharedDnsIdentity so the singleton runtime FQDN, certificate domain names, retention fallback, DI article-parser accessor, and RabbitMQ connection name all use the database-derived identity. Startup smoke that only builds the host (without starting the account service) still reports the provisional FQDN.
BackFillerAcmeCloudflareOptionsAdapter copies BackFiller bind and ACME fields onto AcmeCloudflareOptions and sets IncludeNewsHostnameInCertificate to false. The shared AcmeCloudflareOptions type still defaults that flag to true and documents BindPort 119 as an NNTPD cleartext port. BackFiller validation does not use BindPort.
Nested groups:
BackFiller:LoggingBackFiller:ArticleRetention—MaximumRetainedPayloadGigabytesdefault 4 (must be > 0 and within the truncated 80% physical-memory ceiling; see Host physical memory),RetentionTtlSecondsdefault 60 and range 1–60,MaxOpenableRequestIdsPerArticledefault 16 and range 1–256,SweepIntervalSecondsdefault 1 and range 1–60BackFiller:Shutdown—GracePeriodSecondsdefault 30, range 5–600;DrainQueuedWorkdefault true;FinishActiveArticlesdefault trueBackFiller:Listener—TlsHandshakeTimeoutSecondsdefault 30, range 1–300, copied into the runtime snapshot and used for the handshake.IoProgressTimeoutSecondsdefault 60, range 1–600, copied and used as the no-progress deadline for one listener read or one non-empty listener write. A completed call starts a new deadline for the next call. The deadline is not restarted for each 32 KiB of one buffer.MaxActiveConnectionsdefault 1024, minimum 1, and the listener rejects further accepts at that count.SendBufferBytesdefault 2097152 (2 MiB) andReceiveBufferBytesdefault 1048576 (1 MiB) are copied and applied to each accepted socket before TLS. They are independent. The range for each is 16384 through 16777216. Nagle is disabled on those sockets and is not a setting. TCP keepalive for accepted sockets comes fromBackFiller:TcpKeepAlive, not this section. The listening socket does not carry buffer or keepalive options. At the default connection cap the default buffers are a 3 GiB kernel-buffer ceiling (1024 × 3 MiB), not memory reserved at bind, and not a measured TCP window. The operating system may round, clamp, or double the stored limit.MaxQueuedFoundPayloadBytesdefault 67108864, minimum 1, is copied into the snapshot; the cache listener does not read it.ParserAccumulationMaxBytesdefault 262144, minimum 32768, andAwaitingReceiptAckTimeoutSecondsdefault 30, range 1–300, are validated and are not copied intoBackFillerListenerRuntimeOptions.BackFiller:Nntp—MaxConcurrentSessionEstablishmentsdefault 8, range 1–1024. Zero is invalid. This does not replace per-providerMaxSessionsfrom MySQL.SendBufferBytesdefault 1048576 (1 MiB) andReceiveBufferBytesdefault 2097152 (2 MiB) are the upstream TCP buffer limits. Each is validated from 16384 through 16777216. They are independent of each other and of the 64 KiB managed reader buffer. Nagle is disabled in code. One upstream session at the defaults requests 3 MiB of kernel buffering. Steady-state session count remains the sum of per-providerMaxSessions. These buffers do not raise the establishment gate. The effective window still depends on the operating system, autotuning, TLS, and the path.BackFiller:TcpKeepAlive— TCPSO_KEEPALIVEfor every upstream NNTP socket and every accepted cache-listener socket.TimeSecondsdefault 10 (range 1–7200) is idle time before the first probe.IntervalSecondsdefault 5 (range 1–300) is the gap between probes.RetryCountdefault 3 (range 1–20) is unanswered probes before the stack drops the connection. At the defaults, dead-path detection completes in about 25 s, before the default NNTP command timeout (30 s) and well before the listener I/O-progress timeout (60 s). Independent of per-provider MySQLkeepalive(NNTPDATE). Nagle and buffer settings are unchanged. Platforms that cannot set a probe option still enableSO_KEEPALIVEand keep OS defaults for that option; enabling keepalive itself must succeed or the socket is rejected.BackFiller:Systemd— notify and watchdog enablementBackFiller:SilentConsumerMonitor— optional. OnlyEnabled(default true) is public. Do not place VirtualHost, Hosts, Management URL, or queue lists here. AMQP hosts/vhost/port and Management BaseUrl bind from theRabbitMQsection; username/password are projected fromnntpsharedconfigonto liveRabbitMqOptionson account refresh (Management uses them on the next HTTP call; AMQP uses them on the next reconnect dial). Article Work queues come from the MySQL provider catalog asbackfiller.{backbone}. Detection thresholds are internal code defaults (ProcessingStall/DispatchStall/ConsumerLoss). Management poll failures yieldUnknownand do not resolve open shared alerts; instance-local monitoring continues. Seedocs/architecture.md§ Silent-consumer monitoring (including Management outages andUnknown) for the traced data flow and multi-instance semantics. Monitor failures never disable consumption.BackFiller:LabStageProfiling— lab-only Article Work stage latency histograms. DefaultEnabledfalse (no ambient profiler installed; production path unchanged). WhenEnabledis true, installs bounded per-stage histograms (pipeline, handler, session acquire, busy wait, ARTICLE command/status, payload receive, parse, retain, reply publish, plus reply serialize / publish-gate wait / gate-held confirm) and optionally writesarticle-work-stage-profile.jsonunderExportDirectoryeveryExportIntervalSeconds(default 2). Nested stages overlap — do not sum totals as exclusive wall time. Env:BackFiller__LabStageProfiling__Enabled,BackFiller__LabStageProfiling__ExportDirectory,BackFiller__LabStageProfiling__ExportIntervalSeconds.BackFiller:LabUpstreamIsolation— lab-only. DefaultEnabledfalse (ordinary deployments unchanged). WhenEnabledis true, every upstream NNTP dial throughINntpTransportFactorymust targetAllowedHost:AllowedPortbefore a socket opens; otherwise connect fails closed with no unguarded fallback.AllowedHostmust be exactly127.0.0.1(nolocalhost, DNS names, IPv6, or non-loopback).AllowedPortis required (1–65535) when enabled.RejectTlsdefault true rejects providerUseTls=true.RewritePublishedDestinationsdefault false; when true (requiresEnabledandRejectTls), each account refresh rewrites published provider dial fields toAllowedHost:AllowedPortwithUseTls=falseand clears username/password so AUTHINFO is not sent — MySQL rows are not updated, and production dial behaviour is unchanged when rewrite is off. Incomplete or inconsistent enabled/rewrite configuration failsValidateOnStart. Env examples:BackFiller__LabUpstreamIsolation__Enabled,BackFiller__LabUpstreamIsolation__AllowedPort,BackFiller__LabUpstreamIsolation__RewritePublishedDestinations.
Section name: RabbitMQ (RabbitMqOptions.SectionName). It is not nested under BackFiller.
Username and password are published from nntpsharedconfig (rabbitmqusername, rabbitmqpassword). Bound username and password values are discarded. Environment variables:
VECTOR__RABBITMQ__USERNAME
VECTOR__RABBITMQ__PASSWORD
Never commit or log those values. BackFiller discards values bound from those variables before validation. They are not required in the Linux environment file. Other connectivity settings, including Hosts, bind from the RabbitMQ section. See RabbitMqOptions for the current defaults (WorkRequestMaxPayloadBytes 1024, RpcTimeoutSeconds 30, ConnectionBlockedTimeoutSeconds 30, ChannelLeaseTimeoutSeconds 60).
These are not BackFiller-section keys. NNTPD, BackFiller, and StorageServer publish them from nntpsharedconfig. Environment variables:
VECTOR__CLOUDFLAREAPIKEY
VECTOR__CLOUDFLAREZONEID
VECTOR__ACMEACCOUNT
VECTOR__ACMECERTIFICATEPASSWORD
CloudFlareApiKey is a secret. BackFiller clears the API key, ACME account, and certificate password bound at startup, and it does not copy the root VECTOR__CLOUDFLAREZONEID value onto BackFiller options. Those names are not required in the Linux environment file. Ordinary options validation must not call Cloudflare or mutate DNS. Tests inject NIC data through ILocalIpAddressAssignee instead of reading the live host.
BackFiller:CloudFlareZoneId is the application key. VECTOR__CLOUDFLAREZONEID is the shared environment variable documented on AcmeCloudflareOptions. Do not invent an nntpd__ name for either.
NntpDbOptions.SectionName is NntpDb. The connection string name is ConnectionStrings:NntpDB. The environment variable is:
ConnectionStrings__NntpDB
That name has no VECTOR__ prefix. Do not log the connection string. The same key is what NNTPD uses. That is shared-domain configuration, not an instruction to build the NNTPD solution.
For the shipped TLS configuration, this is the only environment setting startup validation requires. TlsOnlyAcmeCloudflareOptionsValidator requires BindPortTls in the range 1–65535, and the shipped JSON already sets that port. ACME and Cloudflare credential validation is not enabled at startup (validateSharedCredentials: false).
deploy/systemd/vectornntp-backfiller.service contains:
EnvironmentFile=-/etc/vectornntp/backfiller.env
systemd reads that file as root before it executes the service. The leading - suppresses errors reading the environment file. Current systemd ignores a missing, unreadable, or invalid file. Older systemd ignored only a missing file. Suppressing that error does not insert a connection string. If ConnectionStrings__NntpDB is absent, NntpDbOptionsValidator fails with ConnectionStrings:NntpDB must be configured. and the process exits 1. When that message appears, check that the file exists, is owned by root:root, is mode 0600, and that its syntax is valid. Do not print the file.
The file is systemd KEY=VALUE syntax. A line whose first non-space character is # is a comment. systemd parses the file itself. It does not expand variables, command substitutions, or backticks, and source does not reproduce that parse. In an unquoted value, a backslash escapes the next character, so \\ is one backslash, and a backslash at the end of a line joins the next line. Interior spaces remain. Leading and trailing spaces are removed. On systemd 259, single quotes around the whole value keep the enclosed characters verbatim and do not treat a backslash as an escape; a single quote cannot appear inside that pair. Double quotes use a different escape set. Those quote rules are not the same on every systemd version. After editing a value that contains a backslash, a quote, or leading or trailing whitespace, confirm the loaded value on that host without writing it to a log.
Install it as owner root:root and mode 0600. The service account does not need to read the file. Do not commit the installed file, do not pass the connection string on a command line, and do not make the file world-readable. deploy/systemd/backfiller.env.example is the placeholder template. Replace REPLACE_WITH_MYSQL_HOST, REPLACE_WITH_MYSQL_USER, REPLACE_WITH_MYSQL_PASSWORD, and REPLACE_WITH_DATABASE_NAME with a root editor before the first start.
These names are not required in /etc/vectornntp/backfiller.env:
VECTOR__RABBITMQ__USERNAME
VECTOR__RABBITMQ__PASSWORD
VECTOR__CLOUDFLAREAPIKEY
VECTOR__CLOUDFLAREZONEID
VECTOR__ACMEACCOUNT
VECTOR__ACMECERTIFICATEPASSWORD
After the connection string works, ProviderAccountConfigurationService copies the ACME account, ACME certificate password, Cloudflare API key, Cloudflare zone id, DNS suffix, and RabbitMQ username and password from nntpsharedconfig. Assigning the names above in the environment file does not override that row.
Environment=DOTNET_ENVIRONMENT=Production in the unit selects the optional VectorNNTPD.BackFillerService.Production.json overlay. A missing overlay does not fail startup. daemon-reload is required after a unit-file change. An edit to the environment file is read on the next service start; daemon-reload does not reload the environment of a running process. WatchdogSec stays commented out.
- Missing
ServerIdis not the same as0. Both fail. The property is nullable so those cases stay distinct. BindPortTlsmissing,0, or outside 1–65535 fails. TLS disabled is not a valid BackFiller mode.nntpd__cloudflareapikeyandnntpd__CloudFlareZoneIdare not current environment variables.- Do not log
BackFillerOptions,AcmeCloudflareOptions, orRabbitMqOptionsas a whole object. AcmeStateDirandBackFiller:Logging:File:LogDiruse CommonApplicationLocalPath. A relative value, including one that contains.., is rejected when its canonical path leaves the application binary directory. A rooted value is an explicit location and is not forced back under that directory. The check normalizes.and..and does not follow symlinks or junctions.