feat(logging): add managed file output and worker log aggregation

.github/workflows/master.yml

@@ -414,6 +414,9 @@ jobs:
       - name: Verify spawned worker lifecycle on Windows
         run: uv run --locked --no-sync pytest -q tests/builtin/runtime/test_artifact_processing.py
 
+      - name: Verify spawned worker logging on Windows
+        run: uv run --locked --no-sync pytest -q tests/e2e/test_worker_logging.py
+
       - name: Verify bootstrap receipt persistence and failure recovery on Windows
         run: >-
           uv run --locked --no-sync pytest -q

docs/en/development/observability.md

@@ -85,14 +85,11 @@ full database URLs, or sensitive filesystem paths. An allowed field still requir
 a redaction engine. Do not interpolate those values into the message or include them in a raw provider exception.
 Use `exc_info` only after reviewing the error's values and attached context for sensitive data.
 
-The Server configures handlers and format. It injects the picklable `WorkerLogging` bootstrap into automatically
-composed spawn workers through `worker_initializer`. Custom bindings own their launcher initialization.
-Importing or running the Runtime as an SDK must not configure global logging. Keep Runtime observations independent
-of Server settings, Prometheus, and OTel implementation types.
-
-Operational logs are best-effort. The Server handler isolates filter/formatter/write failures and emits one
-content-free failure notification per handler. Do not use logs or sampled traces as a durable audit, billing,
-or commit ledger. Request/trace/scope/source/worker identities do not belong in metric labels.
+The Server configures handlers and format. It supplies `worker_initializer` for automatically composed spawn workers; file output prepares a picklable `WorkerLogging` bootstrap for each launch. Custom bindings own their launcher initialization. Importing or running the Runtime as an SDK must not configure global logging. Keep Runtime observations independent of Server settings, Prometheus, and OTel implementation types.
+
+Operational logs are best-effort. The Server handler isolates filter/formatter/write failures and emits one content-free failure notification per handler. File output uses `RotatingFileHandler` for rotation and retention, with `namer`/`rotator` and `gzip` for compression. The `worker_logging` context reuses the configured Server receiver or owns a worker-only sink for an embedded Server; it must outlive Runtime worker cleanup. Workers apply the field allowlist, identity binding, and formatting before sending. Only rendered UTF-8 output crosses the process boundary; the receiver must not rebind parent identity or serialize original extras. Before each spawn, the bootstrap's `prepare()` context creates a dedicated socketpair and yields the picklable child initializer. The launcher closes its sending end after spawn succeeds or cancellation cleanup completes; the receiver keeps reading until EOF. JSON-line envelopes preserve multiline console output. Malformed or incomplete records retire only their stream and are never echoed. Send waits are bounded. The receiver multiplexes streams into a bounded in-process queue feeding the native handlers. A full queue drops the record and reports a content-free failure. The Server stops Runtime and tracing before draining streams and calling the public `QueueListener.stop()` to finish queued output. The drain deadline bounds Worker-channel cleanup; file writes, compression, and close follow the native logging lifecycle without a separate deadline. Confirmed receiver discards increase the process-local drop counter independently of the once-only notification; clean EOF/reset does not. Logging starts no Manager or other helper process.
+
+Do not use logs or sampled traces as a durable audit, billing, or commit ledger. Request/trace/scope/source/worker identities do not belong in metric labels.
 
 ## Verify the behavior
 

docs/en/docs/operate/configuration.md

@@ -109,6 +109,11 @@ Server settings use the `POWERCONTEXT_SERVER_` prefix.
 | `POWERCONTEXT_SERVER_LOGGING_LEVEL` | `INFO` | Operational log level |
 | `POWERCONTEXT_SERVER_LOGGING_FORMAT` | `console` | `console` or structured `json` output |
 | `POWERCONTEXT_SERVER_LOGGING_ACCESS` | `true` | Log external HTTP and logical MCP request completion |
+| `POWERCONTEXT_SERVER_LOGGING_FILE` | `false` | Write managed files instead of stdout; enabled by default for macOS/Windows personal services |
+| `POWERCONTEXT_SERVER_LOGGING_DIRECTORY` | User data directory + `logs/` | Managed log directory |
+| `POWERCONTEXT_SERVER_LOGGING_MAX_BYTES` | `20971520` | Positive size threshold for rotation |
+| `POWERCONTEXT_SERVER_LOGGING_BACKUP_COUNT` | `10` | Positive maximum archive count per active file |
+| `POWERCONTEXT_SERVER_LOGGING_COMPRESSION` | `gzip` | Archive compression: `gzip` or `none` |
 | `POWERCONTEXT_SERVER_METRICS_ENABLED` | `true` | Expose Prometheus metrics at `/metrics` |
 | `POWERCONTEXT_SERVER_METRICS_BACKGROUND_PORT` | `8001` | Background-role infrastructure listener port; uses `HTTP_HOST` and is disabled with metrics |
 | `POWERCONTEXT_SERVER_TRACING_ENABLED` | `false` | Enable span recording and OTLP export |

docs/en/docs/operate/observability.md

@@ -15,6 +15,34 @@ The foreground Server writes operational logs to its process output. Configure
 and `POWERCONTEXT_SERVER_LOGGING_ACCESS` (default `true`). For a native personal service, run
 `powercontext service status` to find the journal selector or log paths. Docker users can inspect container logs.
 
+### File output, rotation, and compression
+
+Enable file output with `POWERCONTEXT_SERVER_LOGGING_FILE=true`. The Server creates the directory and writes operational logs to files instead of stdout. Python's `RotatingFileHandler` defaults to rotation near 20 MiB and retains up to ten gzip archives per active file. Adjust the threshold, retention, and compression to fit the available storage and CPU budget. Restarting appends to the active file; no external rotation tool is required.
+
+| Environment variable | Default | Meaning |
+| --- | --- | --- |
+| `POWERCONTEXT_SERVER_LOGGING_FILE` | `false` | Write managed files instead of stdout |
+| `POWERCONTEXT_SERVER_LOGGING_DIRECTORY` | User data directory + `logs/` | Directory for log files; respects `POWERCONTEXT_HOME` |
+| `POWERCONTEXT_SERVER_LOGGING_MAX_BYTES` | `20971520` | Positive size threshold for rotation |
+| `POWERCONTEXT_SERVER_LOGGING_BACKUP_COUNT` | `10` | Maximum archives per active file; must be at least one |
+| `POWERCONTEXT_SERVER_LOGGING_COMPRESSION` | `gzip` | Archive compression: `gzip` or `none` |
+
+```bash
+export POWERCONTEXT_SERVER_LOGGING_FILE=true
+export POWERCONTEXT_SERVER_LOGGING_DIRECTORY=/var/lib/powercontext/logs
+powercontext server run
+```
+
+Files are named by the existing Server process role: `all.log`, `api.log`, or `background.log`; explicit configuration without a role uses `server.log`. Workers share their owning Server's file, with execution identity recorded in each entry. Archives are named `all.log.1.gz`, `all.log.2.gz`, and so on, with `.1` being the newest archive. Without compression they use `.1`, `.2`, and so on. Changing compression does not remove archives from the previous policy. The active file remains uncompressed. The size threshold is approximate; one large record can exceed it. Retention applies per active file, not to unrelated files in the directory. `POWERCONTEXT_SERVER_LOGGING_FORMAT` selects readable text or JSON for either stdout or file output.
+
+Workers send already formatted, allowlisted records over dedicated socketpairs to their owning Server. Each channel is created before spawn, and only its sending end is passed to that Worker. The receiver does not listen on a network port or deserialize Python objects. JSON-line envelopes preserve multiline output. Worker identity and request/trace correlation are captured before sending. One receiver thread multiplexes the streams and one writer thread consumes a bounded in-process queue; logging starts no additional processes. Stream-only output keeps the standard stream handlers.
+
+Each file has an exclusive writer lock; a second Server with the same directory and role is rejected. Use separate directories for independent replicas. Stop workers before closing the logging owner. Shutdown drains Worker streams for at most one second, then waits for queued output through `QueueListener.stop()` before closing the handlers. File writes, compression, and close use native logging cleanup without a separate timeout. A stalled file sink can therefore delay Server process exit. Incomplete records are discarded without printing their contents. A failed Worker stream does not stop other streams. Send waits are bounded; a failed send retires that Worker's stream. The receiver handles up to 256 concurrent streams, and JSON-line envelopes are limited to 256 KiB after UTF-8 encoding. Oversized records, full queues, transport failures, and forced process exits can lose logs. Logs remain best-effort.
+
+macOS and Windows personal services enable managed file output by default. An explicit `POWERCONTEXT_SERVER_LOGGING_FILE=false` keeps stream output. Linux personal services keep journal output by default. Host-owned stdout/stderr files still capture startup diagnostics and direct stream writes; those files are outside the managed handler's rotation policy. `powercontext service status` reports the host's diagnostic directory or journal selector; a custom managed directory is selected through `POWERCONTEXT_SERVER_LOGGING_DIRECTORY`.
+
+An unavailable directory or an existing writer prevents file logging configuration. Once running, formatting, transport, queue, write, and compression failures use the same content-free `observability.log_failed` notification as stream failures. Completed gzip archives are published before the original file is removed. SDK Runtime use does not configure logging; embedding applications retain ownership. An embedded Server with file output can own a worker-only sink without reconfiguring the embedding application's loggers.
+
 Use the response's `X-PowerContext-Request-ID` to correlate a failed request with diagnostics. The host guides also
 explain content-free recall and capture diagnostics; a missing context result can be valid rather than an outage.
 
@@ -125,7 +153,7 @@ Runtime/Supervisor instance and reset when that owner is recreated; scrapes do n
 | `dependency_probe_duration_seconds` | `dependency`, `outcome`: actual probes, excluding cache hits |
 | `telemetry_collection_timestamp_seconds` | Last successful passive snapshot collection; `0` before collection |
 | `telemetry_collection_failures_total` | Failed passive collection attempts; the previous observation is retained |
-| `telemetry_log_records_dropped_total` | Records that the process-owned log handler failed to deliver |
+| `telemetry_log_records_dropped_total` | Observed records that the process-owned handler or Worker receiver failed to deliver; clean EOF/reset is not a loss, and unobserved losses are not counted |
 
 Detailed probe observations cover `database`, `inference.generation`, `inference.rerank`, `inference.decision`, and
 `inference.embedding`. The same `dependency_state` family also includes bounded aggregate checks for `runtime`,

docs/en/rfcs/0046_observability_foundations.md

@@ -149,6 +149,8 @@ operation and its outcome.
 
 ## Logging boundary
 
+> Implementation note: The Server supports optional file output with size-based rotation and gzip archives through standard-library logging. macOS/Windows personal services enable it by default. Shutdown bounds Worker-channel cleanup; native file I/O cleanup has no separate timeout. See [file logging](../docs/operate/observability.md#file-output-rotation-and-compression).
+
 Server logging covers:
 
 - startup, readiness changes, and shutdown;

docs/zh/development/observability.md

@@ -86,13 +86,10 @@ Error classification 来自 `powercontext._logging.error_category`；`blocked` 
 这些值也不能拼入 message 或原样包含在 provider exception 中。
 只有确认 error values 和关联上下文不含敏感数据后，才使用 `exc_info`。
 
-Server 配置 handlers 和 format，通过 `worker_initializer` 向自动构建的 spawn workers 注入可序列化的
-`WorkerLogging` bootstrap。Custom bindings 自己管理 launcher initialization。
-通过 SDK 导入或运行 Runtime 时，不得配置 global logging。
-Runtime observations 不依赖 Server settings、Prometheus 或 OTel implementation types。
+Server 配置 handlers 和 format，通过 `worker_initializer` 初始化自动构建的 spawn workers。文件输出为每次启动准备可序列化的 `WorkerLogging` bootstrap。Custom bindings 自己管理 launcher initialization。通过 SDK 导入或运行 Runtime 时，不得配置 global logging。Runtime observations 不依赖 Server settings、Prometheus 或 OTel implementation types。
+
+Operational logs 保持 best-effort。Server handler 隔离 filter、formatter 和 write failures，每个 handler 最多输出一次无内容的 failure notification。文件输出由标准库 `RotatingFileHandler` 管理轮替和保留，通过 `namer`/`rotator` 与 `gzip` 压缩归档。`worker_logging` context 复用已配置的 Server receiver，或为嵌入式 Server 创建仅接收 Worker 日志的 sink；其退出必须晚于 Runtime 的 Worker 清理。Worker 在发送端完成字段过滤、identity 绑定和格式化，跨进程只发送格式化后的 UTF-8 内容；接收端不得重新绑定父进程 identity 或序列化原始 extras。每次 spawn 前，bootstrap 的 `prepare()` context 创建专用 socketpair，返回可序列化的子进程 initializer。launcher 在 spawn 完成或取消清理结束后关闭父进程持有的发送端，接收器持续读取到 EOF。日志通过 JSON 行封装，保留 console 输出中的换行。畸形或残缺记录只影响所属通道，不得回显原始内容。发送等待有上限。接收线程复用多个通道，通过有界的进程内队列交给原生 handler；队列满时丢弃该记录并报告无内容的失败通知。Server 先停止 Runtime 和 tracing，再排空日志通道，通过公开的 `QueueListener.stop()` 完成队列中的输出。排空期限约束 Worker 通道清理；文件写入、压缩和关闭使用原生 logging 生命周期，不另设超时。接收端确认丢弃的记录独立计入进程丢失计数，不受一次性通知限制；干净的 EOF/reset 不计数。日志处理不启动 Manager 或其他辅助进程。
 
-Operational logs 保持 best-effort。Server handler 隔离 filter、formatter 和 write failures，
-每个 handler 最多输出一次无内容的 failure notification。
 不要把 logs 或 sampled traces 当作 durable audit、billing 或 commit ledger。
 Request、trace、scope、source 和 worker identities 不作为 metric labels。
 

docs/zh/docs/operate/configuration.md

@@ -100,6 +100,11 @@ Server 配置使用 `POWERCONTEXT_SERVER_` 前缀。
 | `POWERCONTEXT_SERVER_LOGGING_LEVEL` | `INFO` | operational log 级别 |
 | `POWERCONTEXT_SERVER_LOGGING_FORMAT` | `console` | `console` 或结构化 `json` 输出 |
 | `POWERCONTEXT_SERVER_LOGGING_ACCESS` | `true` | 记录外部 HTTP 和逻辑 MCP request completion |
+| `POWERCONTEXT_SERVER_LOGGING_FILE` | `false` | 写入托管文件以替代 stdout；macOS/Windows 个人服务默认开启 |
+| `POWERCONTEXT_SERVER_LOGGING_DIRECTORY` | 用户数据目录下的 `logs/` | 托管日志目录 |
+| `POWERCONTEXT_SERVER_LOGGING_MAX_BYTES` | `20971520` | 按大小轮替的阈值，必须为正数 |
+| `POWERCONTEXT_SERVER_LOGGING_BACKUP_COUNT` | `10` | 每个活动文件最多保留的归档数，至少为一 |
+| `POWERCONTEXT_SERVER_LOGGING_COMPRESSION` | `gzip` | 归档压缩方式：`gzip` 或 `none` |
 | `POWERCONTEXT_SERVER_METRICS_ENABLED` | `true` | 在 `/metrics` 暴露 Prometheus metrics |
 | `POWERCONTEXT_SERVER_METRICS_BACKGROUND_PORT` | `8001` | Background role 的 infrastructure listener 端口；使用 `HTTP_HOST`，随 metrics 一起关闭 |
 | `POWERCONTEXT_SERVER_TRACING_ENABLED` | `false` | 启用 span recording 和 OTLP export |