Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Standalone VectorNNTPD hosts extracted from VectorNNTP.NNTPD. This repository cu
## Structure

- `src/VectorNNTPD.BackFillerService/VectorNNTPD.BackFillerService.csproj` - BackFiller worker app (net10.0)
- `src/VectorNNTPD.StorageService/VectorNNTPD.StorageService.csproj` - StorageService / VATP cache host (net10.0)
- `src/VectorNNTPD.StorageService/VectorNNTPD.StorageService.csproj` - StorageService / NNTP TAKETHIS feed host (net10.0)
- `src/VectorNNTPD.Common/VectorNNTPD.Common.csproj` - shared library
- `tests/VectorNNTPD.BackFillerService.Tests/` / `tests/VectorNNTPD.StorageService.Tests/` / `tests/VectorNNTPD.Common.Tests/` - xUnit suites
- `VectorNNTPD.slnx` - solution including apps + tests
Expand Down
10 changes: 4 additions & 6 deletions docs/common/vatp.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,17 +193,15 @@ and requires removal after terminalization before reuse.

NNTPD retrieval already opens many OPEN streams on one pooled TLS connection, up to
`MaxStreamsPerConnection` (64). BackFiller schedules their DATA frames one at a time on
a ready ring. StorageServer accepts concurrent STORE streams on one connection the same
way, and writes RESULT after the article is enqueued. `ArticlePlacementClient` keeps one TLS connection per target and sends at most
`Nntpd:ArticleIngestion:MaxOutstandingStoresPerConnection` STOREs on it (default 2, allowed 1 or 2).
One writer emits complete frames. A lost RESULT after END is acknowledgement-not-observed and is not
retried. The spool writer still does not call placement. Retrieval OPEN multiplexing is unchanged.
a ready ring. StorageService article *ingestion* is NNTP RFC 4644 `TAKETHIS` (not VATP STORE);
the Common advertisement field `vatpPort` still names the TLS feed listen port for placement.
Retrieval OPEN multiplexing is unchanged. VATP STORE multiplexing language that formerly
described StorageServer ingestion no longer applies to this host.

## Not implemented yet (later phases)

- Numeric ARTICLE/HEAD/BODY/STAT (article-number / current-article forms)
- OverviewDB article-number resolution
- Storage VATP consumers
- Outbound VATP server-to-server transfers

## NNTP message-id retrieval (Phase 3B)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ namespace VectorNNTPD.StorageService.Configuration
/// </summary>
/// <remarks>
/// <para>
/// VATP RESULT is still sent when an article enters <c>AcceptedArticleQueue</c>.
/// TAKETHIS 239 is still sent when an article enters <c>AcceptedArticleQueue</c>.
/// These settings bound later disk work. They do not make that RESULT durable.
/// </para>
/// <para>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ ServerId is { } serverId
public int BindPort { get; set; }

/// <summary>
/// Gets or sets the TLS TCP port used by the VATP listener.
/// Gets or sets the TLS TCP port used by the NNTP feed listener.
/// </summary>
/// <remarks>
/// Required. Range 1–65535. StorageServer is TLS-only; there is no cleartext fallback.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ public static class StorageServerServiceCollectionExtensions
/// </summary>
/// <remarks>
/// Application service start order:
/// NntpDB → nntpsharedconfig → Cloudflare DNS → RabbitMQ (connectivity only) → ACME → segment-pool validation → spool shard initialization → article index → block-device monitor → accepted-article worker → VATP listener → AdvertisementPublisher.
/// NntpDB → nntpsharedconfig → Cloudflare DNS → RabbitMQ (connectivity only) → ACME → segment-pool validation → spool shard initialization → article index → block-device monitor → accepted-article worker → NNTP feed listener → AdvertisementPublisher.
/// Shutdown is that order reversed, so advertisement stops before the listener, and the listener completes the queue before the worker stops.
/// Topology, queues, and consumers are not registered here. NntpDB does not migrate schema.
/// </remarks>
Expand Down Expand Up @@ -138,8 +138,6 @@ public static HostApplicationBuilder AddStorageServerHosting(this HostApplicatio
builder.Services.TryAddSingleton<SpoolArticlePublisher>();
builder.Services.TryAddSingleton<IAcceptedArticleProcessor, SpoolArticlePersistenceProcessor>();
builder.Services.TryAddSingleton<AcceptedArticleWorker>();
builder.Services.TryAddSingleton<IStorageArticleOpenBoundary, RejectingStorageArticleOpenBoundary>();

builder.Services.TryAddSingleton<IRabbitMqConnectionNameProvider>(static sp =>
new DelegateRabbitMqConnectionNameProvider(() =>
RabbitMqRuntimeOptions.GetDefaultConnectionName(
Expand Down Expand Up @@ -204,19 +202,33 @@ public static HostApplicationBuilder AddStorageServerHosting(this HostApplicatio
{
var listener = provider.GetRequiredService<IOptions<StorageServerOptions>>().Value.Listener;
var dns = provider.GetRequiredService<CloudflareDnsReconciliationService>();
return new StorageVatpListenerService(
var index = provider.GetService<IArticleIndex>();
Func<Common.Articles.ArticleId, bool>? articlePresent = index is null
? null
: id =>
{
try
{
return index.IsReady && index.Contains(id);
}
catch (InvalidOperationException)
{
return false;
}
};
return new StorageNntpListenerService(
provider.GetRequiredService<StorageServerRuntimeOptionsSource>(),
provider.GetRequiredService<Common.Networking.Certificates.ITlsCertificateContextProvider>(),
provider.GetRequiredService<IStorageArticleOpenBoundary>(),
provider.GetRequiredService<IAcmeCertificateReadiness>(),
provider.GetRequiredService<ILogger<StorageVatpListenerService>>(),
provider.GetRequiredService<ILogger<StorageNntpListenerService>>(),
provider.GetRequiredService<IAcceptedArticleQueue>(),
new StoreAssemblyAdmission(listener.MaxActiveConnections),
articlePresent: articlePresent,
publishAfterBind: dns.PublishAfterListenerBoundAsync);
});
builder.Services.TryAddEnumerable(
ServiceDescriptor.Singleton<IApplicationService, StorageVatpListenerService>(static provider =>
provider.GetRequiredService<StorageVatpListenerService>()));
ServiceDescriptor.Singleton<IApplicationService, StorageNntpListenerService>(static provider =>
provider.GetRequiredService<StorageNntpListenerService>()));
builder.Services.TryAddEnumerable(
ServiceDescriptor.Singleton<IApplicationService, StorageServerAdvertisementPublisherService>(static sp =>
sp.GetRequiredService<StorageServerAdvertisementPublisherService>()));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ namespace VectorNNTPD.StorageService.Ingress
{

/// <summary>
/// One accepted VATP article handed to in-memory consumers before Path rewrite.
/// One accepted feed article handed to in-memory consumers before Path rewrite.
/// </summary>
/// <param name="ArtId">Article identity from the accepted inbound bytes.</param>
/// <param name="ArtHash">Hash of <paramref name="ArtData"/>, matching the inbound META hash.</param>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ namespace VectorNNTPD.StorageService.Ingress
/// </para>
/// <para>
/// The default budget is one gibibyte. At 768,000-byte articles that is about 1,398 articles.
/// VATP RESULT is sent only after a true enqueue. A false enqueue is the overload signal
/// TAKETHIS 239 is sent only after a true enqueue. A false enqueue is the overload signal
/// and is not a durable-storage acknowledgement.
/// </para>
/// </remarks>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ namespace VectorNNTPD.StorageService.Ingress
/// Each worker holds at most one dequeued article. A waiting-for-capacity result keeps that
/// article off the queue and retries after <see cref="SpoolPersistenceOptions.IneligibleRetryMilliseconds"/>.
/// Shutdown cancels in-flight publishes. An article already dequeued and not yet published is
/// logged and dropped. VATP RESULT was sent at enqueue and is not repeated here.
/// logged and dropped. TAKETHIS 239 was sent at enqueue and is not repeated here.
/// </para>
/// </remarks>
internal sealed class AcceptedArticleWorker : IApplicationService
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ namespace VectorNNTPD.StorageService.Ingress
public readonly record struct AcceptedArticleDelivery(bool HasArticle, AcceptedArticle Article);

/// <summary>
/// In-memory handoff of accepted VATP articles. Multiple sessions may enqueue,
/// In-memory handoff of accepted feed articles. Multiple sessions may enqueue,
/// and multiple consumers may dequeue. The queue does not persist articles.
/// </summary>
public interface IAcceptedArticleQueue
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
namespace VectorNNTPD.StorageService.Listener
{
/// <summary>One established TLS transport owned by a StorageServer NNTP feed session.</summary>
public interface IStorageListenerTransport : IAsyncDisposable
{
/// <summary>Reads bytes. Zero means the peer closed.</summary>
ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken);

/// <summary>Writes bytes and returns how many were accepted.</summary>
ValueTask<int> WriteAsync(ReadOnlyMemory<byte> buffer, CancellationToken cancellationToken);

/// <summary>
/// Suspends or resumes the read no-progress timeout.
/// A TAKETHIS wait for queue capacity suspends it until that wait ends.
/// </summary>
/// <param name="suspended"><see langword="true"/> to disarm the outstanding read timer.</param>
void SetReadProgressSuspended(bool suspended) { }
}
}
Loading
Loading