Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,9 @@ Nested groups:
- `BackFiller:Logging`
- `BackFiller:ArticleRetention` — `MaximumRetainedPayloadGigabytes` default 4 (must be > 0 and within the truncated 80% physical-memory ceiling; see Host physical memory), `RetentionTtlSeconds` default 60 and range 1–60, `MaxOpenableRequestIdsPerArticle` default 16 and range 1–256, `SweepIntervalSeconds` default 1 and range 1–60
- `BackFiller:Shutdown` — `GracePeriodSeconds` default 30, range 5–600; `DrainQueuedWork` default true; `FinishActiveArticles` default true
- `BackFiller:Listener` — `TlsHandshakeTimeoutSeconds` default 30, range 1–300, copied into the runtime snapshot and used for the handshake. `IoProgressTimeoutSeconds` default 60, range 1–600, copied and used as the no-progress deadline for one listener read or one non-empty listener write. A completed call starts a new deadline for the next call. The deadline is not restarted for each 32 KiB of one buffer. `MaxActiveConnections` default 1024, minimum 1, and the listener rejects further accepts at that count. `SendBufferBytes` default 2097152 (2 MiB) and `ReceiveBufferBytes` default 1048576 (1 MiB) are copied and applied to each accepted socket before TLS. They are independent. The range for each is 16384 through 16777216. Nagle is disabled on those sockets and is not a setting. The listening socket does not carry these options. At the default connection cap the default buffers are a 3 GiB kernel-buffer ceiling (1024 × 3 MiB), not memory reserved at bind, and not a measured TCP window. The operating system may round, clamp, or double the stored limit. `MaxQueuedFoundPayloadBytes` default 67108864, minimum 1, is copied into the snapshot; the cache listener does not read it. `ParserAccumulationMaxBytes` default 262144, minimum 32768, and `AwaitingReceiptAckTimeoutSeconds` default 30, range 1–300, are validated and are not copied into `BackFillerListenerRuntimeOptions`.
- `BackFiller:Listener` — `TlsHandshakeTimeoutSeconds` default 30, range 1–300, copied into the runtime snapshot and used for the handshake. `IoProgressTimeoutSeconds` default 60, range 1–600, copied and used as the no-progress deadline for one listener read or one non-empty listener write. A completed call starts a new deadline for the next call. The deadline is not restarted for each 32 KiB of one buffer. `MaxActiveConnections` default 1024, minimum 1, and the listener rejects further accepts at that count. `SendBufferBytes` default 2097152 (2 MiB) and `ReceiveBufferBytes` default 1048576 (1 MiB) are copied and applied to each accepted socket before TLS. They are independent. The range for each is 16384 through 16777216. Nagle is disabled on those sockets and is not a setting. TCP keepalive for accepted sockets comes from `BackFiller:TcpKeepAlive`, not this section. The listening socket does not carry buffer or keepalive options. At the default connection cap the default buffers are a 3 GiB kernel-buffer ceiling (1024 × 3 MiB), not memory reserved at bind, and not a measured TCP window. The operating system may round, clamp, or double the stored limit. `MaxQueuedFoundPayloadBytes` default 67108864, minimum 1, is copied into the snapshot; the cache listener does not read it. `ParserAccumulationMaxBytes` default 262144, minimum 32768, and `AwaitingReceiptAckTimeoutSeconds` default 30, range 1–300, are validated and are not copied into `BackFillerListenerRuntimeOptions`.
- `BackFiller:Nntp` — `MaxConcurrentSessionEstablishments` default 8, range 1–1024. Zero is invalid. This does not replace per-provider `MaxSessions` from MySQL. `SendBufferBytes` default 1048576 (1 MiB) and `ReceiveBufferBytes` default 2097152 (2 MiB) are the upstream TCP buffer limits. Each is validated from 16384 through 16777216. They are independent of each other and of the 64 KiB managed reader buffer. Nagle is disabled in code. One upstream session at the defaults requests 3 MiB of kernel buffering. Steady-state session count remains the sum of per-provider `MaxSessions`. These buffers do not raise the establishment gate. The effective window still depends on the operating system, autotuning, TLS, and the path.
- `BackFiller:TcpKeepAlive` — TCP `SO_KEEPALIVE` for every upstream NNTP socket and every accepted cache-listener socket. `TimeSeconds` default 10 (range 1–7200) is idle time before the first probe. `IntervalSeconds` default 5 (range 1–300) is the gap between probes. `RetryCount` default 3 (range 1–20) is unanswered probes before the stack drops the connection. At the defaults, dead-path detection completes in about 25 s, before the default NNTP command timeout (30 s) and well before the listener I/O-progress timeout (60 s). Independent of per-provider MySQL `keepalive` (NNTP `DATE`). Nagle and buffer settings are unchanged. Platforms that cannot set a probe option still enable `SO_KEEPALIVE` and keep OS defaults for that option; enabling keepalive itself must succeed or the socket is rejected.
- `BackFiller:Systemd` — notify and watchdog enablement

## RabbitMQ section
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,15 @@ ServerId is { } serverId
/// <remarks>Required. The runtime snapshot rejects a null section.</remarks>
public BackFillerNntpOptions Nntp { get; set; } = new();

/// <summary>
/// Gets or sets TCP keepalive probe timings for upstream NNTP sockets and accepted listener sockets.
/// </summary>
/// <remarks>
/// Required. Defaults and ranges live on <see cref="BackFillerTcpKeepAliveOptions"/>.
/// Independent of per-provider MySQL <c>keepalive</c> (NNTP <c>DATE</c>).
/// </remarks>
public BackFillerTcpKeepAliveOptions TcpKeepAlive { get; set; } = new();

/// <summary>
/// Gets or sets Linux systemd notify/watchdog options.
/// </summary>
Expand Down Expand Up @@ -480,4 +489,46 @@ internal sealed class BackFillerArticleRetentionOptions
/// <remarks>Default 1. Validation accepts 1–60. The snapshot stores a <see cref="TimeSpan"/>.</remarks>
public int SweepIntervalSeconds { get; set; } = 1;
}

/// <summary>
/// TCP keepalive timings for upstream provider sockets and accepted cache-listener sockets.
/// </summary>
/// <remarks>
/// Applied by <see cref="Networking.TcpSocketOptions"/> before NNTP or TLS traffic.
/// Does not replace application-level NNTP <c>DATE</c> keepalive from MySQL <c>keepalive</c>.
/// Defaults match <see cref="Networking.TcpKeepAliveSettings.Default"/>.
/// </remarks>
internal sealed class BackFillerTcpKeepAliveOptions
{
/// <summary>
/// Idle seconds before the first TCP keepalive probe.
/// </summary>
/// <remarks>
/// Default <see cref="Networking.TcpKeepAliveSettings.DefaultTimeSeconds"/>.
/// Validation accepts <see cref="Networking.TcpKeepAliveSettings.MinimumPositive"/>–
/// <see cref="Networking.TcpKeepAliveSettings.MaximumTimeSeconds"/>.
/// </remarks>
public int TimeSeconds { get; set; } = Networking.TcpKeepAliveSettings.DefaultTimeSeconds;

/// <summary>
/// Seconds between consecutive TCP keepalive probes.
/// </summary>
/// <remarks>
/// Default <see cref="Networking.TcpKeepAliveSettings.DefaultIntervalSeconds"/>.
/// Validation accepts <see cref="Networking.TcpKeepAliveSettings.MinimumPositive"/>–
/// <see cref="Networking.TcpKeepAliveSettings.MaximumIntervalSeconds"/>.
/// </remarks>
public int IntervalSeconds { get; set; } = Networking.TcpKeepAliveSettings.DefaultIntervalSeconds;

/// <summary>
/// Unanswered probe count before the stack drops the connection.
/// </summary>
/// <remarks>
/// Default <see cref="Networking.TcpKeepAliveSettings.DefaultRetryCount"/>.
/// Validation accepts <see cref="Networking.TcpKeepAliveSettings.MinimumPositive"/>–
/// <see cref="Networking.TcpKeepAliveSettings.MaximumRetryCount"/>.
/// Platforms that cannot set retry count keep OS defaults after enabling <c>SO_KEEPALIVE</c>.
/// </remarks>
public int RetryCount { get; set; } = Networking.TcpKeepAliveSettings.DefaultRetryCount;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ public ValidateOptionsResult Validate(string? name, BackFillerOptions options)
ValidateShutdown(options, failures);
ValidateListener(options, failures);
ValidateNntp(options, failures);
ValidateTcpKeepAlive(options, failures);
ValidateAccountRefresh(options, failures);
ValidateArticleRetention(options, failures);
ValidateSystemd(options, failures);
Expand Down Expand Up @@ -308,6 +309,43 @@ private static void ValidateNntp(BackFillerOptions options, List<string> failure
ValidateBufferSize("BackFiller:Nntp:ReceiveBufferBytes", nntp.ReceiveBufferBytes, failures);
}

/// <summary>
/// Appends failures when TCP keepalive timings are outside the ranges shared with
/// <see cref="TcpKeepAliveSettings"/>.
/// </summary>
/// <param name="options">Options whose <see cref="BackFillerOptions.TcpKeepAlive"/> section is checked.</param>
/// <param name="failures">Failure list to append. Not cleared.</param>
private static void ValidateTcpKeepAlive(BackFillerOptions options, List<string> failures)
{
var keepAlive = options.TcpKeepAlive;
if (keepAlive.TimeSeconds
is < TcpKeepAliveSettings.MinimumPositive
or > TcpKeepAliveSettings.MaximumTimeSeconds)
{
failures.Add(
"BackFiller:TcpKeepAlive:TimeSeconds must be between "
+ $"{TcpKeepAliveSettings.MinimumPositive} and {TcpKeepAliveSettings.MaximumTimeSeconds}.");
}

if (keepAlive.IntervalSeconds
is < TcpKeepAliveSettings.MinimumPositive
or > TcpKeepAliveSettings.MaximumIntervalSeconds)
{
failures.Add(
"BackFiller:TcpKeepAlive:IntervalSeconds must be between "
+ $"{TcpKeepAliveSettings.MinimumPositive} and {TcpKeepAliveSettings.MaximumIntervalSeconds}.");
}

if (keepAlive.RetryCount
is < TcpKeepAliveSettings.MinimumPositive
or > TcpKeepAliveSettings.MaximumRetryCount)
{
failures.Add(
"BackFiller:TcpKeepAlive:RetryCount must be between "
+ $"{TcpKeepAliveSettings.MinimumPositive} and {TcpKeepAliveSettings.MaximumRetryCount}.");
}
}

/// <summary>Appends a failure when a TCP buffer limit is outside 16 KiB through 16 MiB.</summary>
/// <param name="key">Configuration key included in the failure text.</param>
/// <param name="bytes">Requested buffer size.</param>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ namespace VectorNNTPD.BackFillerService.Configuration
/// <param name="NntpDb">NntpDB connection projection. <see cref="NntpDbRuntimeOptions.ConnectionString"/> is secret.</param>
/// <param name="AccountRefreshInterval">MySQL account-refresh poll interval.</param>
/// <param name="Nntp">Validated outbound NNTP provider session bounds.</param>
/// <param name="TcpKeepAlive">Validated TCP keepalive timings for upstream and accepted listener sockets.</param>
internal sealed record BackFillerRuntimeOptions(
int ServerId,
BackFillerRuntimeDnsIdentity DnsIdentity,
Expand All @@ -48,7 +49,8 @@ internal sealed record BackFillerRuntimeOptions(
BackFillerRabbitMqRuntimeOptions RabbitMq,
NntpDbRuntimeOptions NntpDb,
TimeSpan AccountRefreshInterval,
BackFillerNntpRuntimeOptions Nntp)
BackFillerNntpRuntimeOptions Nntp,
Networking.TcpKeepAliveSettings TcpKeepAlive)
{
/// <summary>Canonical DNS suffix. Updated by <see cref="ApplySharedDnsIdentity"/>.</summary>
internal string DnsSuffix => DnsIdentity.DnsSuffix;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using MySqlConnector;
using VectorNNTPD.BackFillerService.Acme;
using VectorNNTPD.BackFillerService.Networking;
using VectorNNTPD.Common.Configuration;
using VectorNNTPD.Common.Messaging.RabbitMq;
using VectorNNTPD.Common.Acme;
Expand Down Expand Up @@ -154,6 +155,7 @@ internal static BackFillerRuntimeOptions Create(
var listener = options.Listener ?? throw new InvalidOperationException("BackFiller:Listener is required.");
var shutdown = options.Shutdown ?? throw new InvalidOperationException("BackFiller:Shutdown is required.");
var nntp = options.Nntp ?? throw new InvalidOperationException("BackFiller:Nntp is required.");
var tcpKeepAlive = options.TcpKeepAlive ?? throw new InvalidOperationException("BackFiller:TcpKeepAlive is required.");

return new BackFillerRuntimeOptions(
ServerId: serverId,
Expand Down Expand Up @@ -193,7 +195,11 @@ internal static BackFillerRuntimeOptions Create(
Nntp: new BackFillerNntpRuntimeOptions(
nntp.MaxConcurrentSessionEstablishments,
nntp.SendBufferBytes,
nntp.ReceiveBufferBytes));
nntp.ReceiveBufferBytes),
TcpKeepAlive: new TcpKeepAliveSettings(
tcpKeepAlive.TimeSeconds,
tcpKeepAlive.IntervalSeconds,
tcpKeepAlive.RetryCount));
}

/// <summary>Resolves the file-log directory, or returns empty when that directory is blank.</summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -852,7 +852,8 @@ private bool TryConfigureAcceptedSocket(Socket accepted, string remote)
TcpSocketOptions.Apply(
accepted,
_runtime.Listener.SendBufferBytes,
_runtime.Listener.ReceiveBufferBytes);
_runtime.Listener.ReceiveBufferBytes,
_runtime.TcpKeepAlive);
}
catch (Exception ex) when (ex is SocketException or ObjectDisposedException or ArgumentOutOfRangeException)
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
namespace VectorNNTPD.BackFillerService.Networking
{
/// <summary>
/// TCP keepalive probe timings applied with <c>SO_KEEPALIVE</c>.
/// </summary>
/// <remarks>
/// Defaults are chosen so dead-path detection begins and progresses before the default
/// NNTP <c>CommandTimeout</c> (30 s) and listener <c>IoProgressTimeout</c> (60 s):
/// idle <see cref="DefaultTimeSeconds"/>, then up to <see cref="DefaultRetryCount"/> probes
/// spaced by <see cref="DefaultIntervalSeconds"/> (about 25 s worst case to drop).
/// Values are seconds, matching the .NET <c>TcpKeepAliveTime</c>, <c>TcpKeepAliveInterval</c>,
/// and <c>TcpKeepAliveRetryCount</c> socket options.
/// </remarks>
/// <param name="TimeSeconds">Idle seconds before the first keepalive probe.</param>
/// <param name="IntervalSeconds">Seconds between consecutive probes.</param>
/// <param name="RetryCount">Number of unanswered probes before the stack drops the connection.</param>
internal readonly record struct TcpKeepAliveSettings(int TimeSeconds, int IntervalSeconds, int RetryCount)
{
/// <summary>Default idle seconds before the first probe.</summary>
internal const int DefaultTimeSeconds = 10;

/// <summary>Default seconds between probes.</summary>
internal const int DefaultIntervalSeconds = 5;

/// <summary>Default unanswered probe count before the connection is dropped.</summary>
internal const int DefaultRetryCount = 3;

/// <summary>Smallest accepted idle, interval, or retry value.</summary>
internal const int MinimumPositive = 1;

/// <summary>Largest accepted idle time in seconds (two hours).</summary>
internal const int MaximumTimeSeconds = 7200;

/// <summary>Largest accepted probe interval in seconds.</summary>
internal const int MaximumIntervalSeconds = 300;

/// <summary>Largest accepted retry count.</summary>
internal const int MaximumRetryCount = 20;

/// <summary>Production and configuration default timings.</summary>
internal static TcpKeepAliveSettings Default { get; } = new(
DefaultTimeSeconds,
DefaultIntervalSeconds,
DefaultRetryCount);

/// <summary>
/// Throws when any field is outside the accepted ranges used by configuration validation.
/// </summary>
/// <exception cref="ArgumentOutOfRangeException">A field is outside its documented range.</exception>
internal void Validate()
{
ArgumentOutOfRangeException.ThrowIfLessThan(TimeSeconds, MinimumPositive);
ArgumentOutOfRangeException.ThrowIfGreaterThan(TimeSeconds, MaximumTimeSeconds);
ArgumentOutOfRangeException.ThrowIfLessThan(IntervalSeconds, MinimumPositive);
ArgumentOutOfRangeException.ThrowIfGreaterThan(IntervalSeconds, MaximumIntervalSeconds);
ArgumentOutOfRangeException.ThrowIfLessThan(RetryCount, MinimumPositive);
ArgumentOutOfRangeException.ThrowIfGreaterThan(RetryCount, MaximumRetryCount);
}
}
}
Loading
Loading