diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index e26f4b6..50f3bcb 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -25,9 +25,17 @@ order they are loaded (later files win on conflicts): Windows uses the same XDG logic for the global config, so the default is `%USERPROFILE%\.config\phosphor\phosphor.json` unless `XDG_CONFIG_HOME` is set. +All files are **deep-merged per JSON key**, not replaced wholesale — a project +file that sets only `models.large` leaves everything else in your global file +intact. The files with the higher numbers above override the lower ones for +any key they define. + The **global data config** is where Phosphor persists things it learns at runtime (recently used models, per-provider settings saved from the TUI). The **project config** is safe to commit to a repository to share settings with your team. +If the TUI ever writes something to the global data config that you disagree +with (e.g. a `models.small` selection), edit or delete that key there — it +overrides the same key in your global config. The workspace also gets a **data directory** (default `.phosphor/` at the project root, configurable via `options.data_directory` or `--data-dir`). It holds the @@ -256,7 +264,9 @@ The minimum viable cloud setup: ``` `large` is the primary coding/reasoning model; `small` handles title generation -and lightweight sub-tasks. Model entries also accept sampling overrides +and lightweight sub-tasks (summarization, `agentic_fetch`). If `small` is +omitted it falls back to `large` — Phosphor never auto-selects a small model +from another provider. Model entries also accept sampling overrides (`temperature`, `top_p`, `top_k`, `max_tokens`, `seed`, `min_p`, `repetition_penalty`, `stop`, …) plus `reasoning_effort` (OpenAI-style), `think` (Anthropic), and `max_thinking_tokens`. diff --git a/docs/MODELS_AND_PROVIDERS_CONFIG.md b/docs/MODELS_AND_PROVIDERS_CONFIG.md index e9eb005..5e3046b 100644 --- a/docs/MODELS_AND_PROVIDERS_CONFIG.md +++ b/docs/MODELS_AND_PROVIDERS_CONFIG.md @@ -199,6 +199,22 @@ The `models` section selects which model to use for different tasks. Phosphor us - **`large`** — Primary model for coding, reasoning, and complex tasks - **`small`** — Fallback model for title generation, quick responses, and sub-agents +If `small` is omitted, it falls back to the `large` model rather than +auto-selecting one from the provider catalog. Every consumer of the small +slot (title generation, summarization, `agentic_fetch`) uses the large model +until you pin `models.small` explicitly. A `models.small` whose provider or +model no longer resolves also falls back to `large`. + +The list of providers and models offered in the model dialog is **not** +stored in your config: it comes from the [Catwalk](https://catwalk.charm.land) +catalog, fetched at runtime and cached alongside the global data config +(`%LOCALAPPDATA%\phosphor\providers.json` on Windows, +`~/.local/share/phosphor/providers.json` on Linux/macOS; override the source +with `CATWALK_URL`). A known provider appears in the dialog as soon as its +API key is available in your environment — that does **not** mean it is +selected. `phosphor.json` only pins or overrides choices; anything you don't +pin resolves against the catalog. + ### Model Fields | Field | Type | Required | Description | diff --git a/docs/security/NETWORK_EGRESS_HARDENING.md b/docs/security/NETWORK_EGRESS_HARDENING.md index 787b8f9..a53842a 100644 --- a/docs/security/NETWORK_EGRESS_HARDENING.md +++ b/docs/security/NETWORK_EGRESS_HARDENING.md @@ -31,7 +31,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* ## Components -### `internal/agent/tools/web_fetch.go` +### `pkg/agent/tools/web_fetch.go` - `securityTransport` — custom `http.RoundTripper` that intercepts each request - `allowList` — in-memory session allow cache (cleared on session restart) @@ -42,7 +42,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* - `FetchURLAndConvertWithTrust` — wraps `FetchURLAndConvert` with the trust check - `NewWebFetchTool` — constructs tool with `allowFn` callback (TUI permission prompt) -### `internal/agent/tools/web_search.go` +### `pkg/agent/tools/web_search.go` - `NewWebSearchTool` — same security contract as `web_fetch`; panics if `client` is nil - `sanitizeSearchResults` — regex scrubber redacts: @@ -51,7 +51,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* - `randomDelay` — adds 100–150ms random delay per round-trip to stay under DDG's request-budget threshold - OTel span tagged with `attribute.Int("gen_ai.tool.web_search.results", len(results))` -### `internal/agent/coordinator.go` +### `pkg/agent/coordinator.go` - `tuiAllowPrompt` field added to `coordinator` struct - Wired in `NewCoordinator` via `c.permissions.Request` with synthetic tool-call ID (`tui:allow:`) @@ -101,8 +101,8 @@ Phosphor's `web_fetch` and `web_search` tools consult the user's `phosphor.json` | Field | Type | Default | Description | |---|---|---|---| -| `IPAllowList` | `[]string` | `[]` | CIDR or literal IPs allowed to be reached | -| `AllowRawIPs` | `bool` | `false` | Opt-in for raw IP access (default: FQDN required) | +| `IPAllowList` | `[]string` | `[]` | CIDR or literal IPs allowed to be reached (consulted only when `allow_raw_ips` is true) | +| `AllowRawIPs` | `bool` | `false` | Gate for raw IP access; IPs are allowed only if they also match `IPAllowList` (default: FQDN required) | ### Example `phosphor.json` @@ -126,9 +126,8 @@ Phosphor's `web_fetch` and `web_search` tools consult the user's `phosphor.json` When `securityTransport` intercepts a request to a raw IP: 1. **127.0.0.1 / ::1** (localhost) → always allowed, no config needed -2. **`IPAllowList`** → literal match or CIDR match via `matchCIDR` → allowed -3. **`AllowRawIPs = true`** → allowed (if no CIDR match) -4. **No match** → TUI prompt shows "WebFetch is requesting access to [host]. Allow this domain?" +2. **`AllowRawIPs = true` + `IPAllowList`** → literal match or CIDR match via `matchCIDR` → allowed +3. **Anything else** → the request fails closed with an error; raw IPs never reach the TUI prompt (prompts apply to hostnames only) ## Bash Network Egress Policy @@ -172,30 +171,50 @@ guarantees. ## Security Properties - **Fail-closed**: no network requests succeed until the agent confirms the domain is trusted -- **IP-block policy**: raw IP addresses (e.g., `10.x.x.x`, `192.168.x.x`, public IPs) are rejected — FQDNs required +- **IP-block policy**: raw IP addresses are rejected for both IPv4 and IPv6 literal forms (e.g., `10.x.x.x`, `192.168.x.x`, `[::1]`, `[fd00::…]`) — FQDNs required +- **Resolved-address validation**: the same IP policy is enforced at dial time on the *resolved* socket address, so an approved hostname whose DNS answer rebinds into a private, link-local, or cloud-metadata range is refused at the socket (DNS-rebinding defense). Loopback stays allowed; `IPAllowList`/`AllowRawIPs` exempt like raw IPs; when an environment proxy is configured the operator's proxy is the egress decision and the strict path remains the egress broker tier - **Opt-in overrides**: `AllowRawIPs` + `IPAllowList` (CIDR) allow raw IPs for local/network development - **Layered trust**: workspace + global allow lists + session allow list + user prompt - **OTel observability**: every allow/deny/event is captured as an OTel span with structured attributes - **Cacheable**: user approvals persist across requests (in-memory cache) but are session-local - **Shared transport**: both `web_fetch` and `web_search` share the same `*http.Client` → same `securityTransport` -- **Fail-closed**: no network requests succeed until the agent confirms the domain is trusted -- **IP-block policy**: raw IP addresses (e.g., `10.x.x.x`, `192.168.x.x`, public IPs) are rejected — FQDNs required -- **Layered trust**: workspace + global allow lists + session allow list + user prompt -- **OTel observability**: every allow/deny/event is captured as an OTel span with structured attributes -- **Cacheable**: user approvals persist across requests (in-memory cache) but are session-local -- **Shared transport**: both `web_fetch` and `web_search` share the same `*http.Client` → same `securityTransport` +## TLS Trust and CA Compromise + +The IP-block policy and trust hierarchy govern *where* the agent may connect; +certificate validation governs *whether the peer is who it claims to be*. The +two layers are independent, and an AI agent depends on the second more than +ordinary software does: the agent carries provider API keys in every request, +fetches attacker-chosen URLs by design, and can download and execute code from +hosts it has never seen. A compromised certificate authority — one whose +intermediate keys leak and can sign certs for *any* domain — silently defeats +every hostname-level defense at once (allow lists, IP blocks, FQDN +requirements). + +Phosphor performs no custom certificate handling. TLS verification is delegated +to the runtime trust store: + +- **Windows / macOS**: the platform certificate store, so OS security updates + and browser-grade revocation (CRLSets, OneCRL) cover CA compromise. +- **Linux**: Phosphor builds with `CGO_ENABLED=0`, so Go uses its compiled-in + copy of the Mozilla root program. Keeping the Go toolchain patched — + `go.mod` pins the minimum version — is therefore part of the security + posture, not just a build detail. + +Concretely: when an intermediate CA in a trusted root program is compromised, +the remedy is a toolchain/OS update, and users should update promptly the same +way they would for a browser. ## Files Modified -- `internal/agent/tools/web_fetch.go` — added security transport, allow lists, TUI prompt wiring -- `internal/agent/tools/web_search.go` — added sanitization, OTel tagging, random delay, nil client panic -- `internal/agent/coordinator.go` — added `tuiAllowPrompt` field and wiring -- `internal/agent/agentic_fetch_tool.go` — updated call site to pass `c.tuiAllowPrompt` +- `pkg/agent/tools/web_fetch.go` — added security transport, allow lists, TUI prompt wiring +- `pkg/agent/tools/web_search.go` — added sanitization, OTel tagging, random delay, nil client panic +- `pkg/agent/coordinator.go` — added `tuiAllowPrompt` field and wiring +- `pkg/agent/agentic_fetch_tool.go` — updated call site to pass `c.tuiAllowPrompt` ## Notes -- `go.mod` requires Go 1.26.4+ (running 1.25.0 locally — LSP errors may appear for tooling checks) +- `go.mod` requires Go 1.27.3+ (patched trust store; see TLS Trust and CA Compromise above) - `fmt`, `net`, `strings`, `html/template`, `encoding/json`, `time`, `embed` imports are all used in body - `maybeDelaySearch` removed from `web_search.go` (already present in `search.go`) - `randomDelay` added for DDG request-budget threshold diff --git a/docs/security/SECURITY_VALIDATION_TEST_SUITE.md b/docs/security/SECURITY_VALIDATION_TEST_SUITE.md index 591f4a1..7ad0cc0 100644 --- a/docs/security/SECURITY_VALIDATION_TEST_SUITE.md +++ b/docs/security/SECURITY_VALIDATION_TEST_SUITE.md @@ -120,7 +120,7 @@ Validates that IsInside and SmartJoin handle complex path resolutions robustly. | `TestPathObfuscation_SymlinkTraversal` | Symlinks pointing outside workspace resolved by EvalSymlinks and blocked | | `TestPathObfuscation_AbsolutePathVariants` | Absolute paths on all platforms blocked: Windows (`C:\Windows\...`), Linux (`/etc/passwd`), macOS (`/Users/...`) | -### `internal/agent/tools/web_fetch_test.go` +### `pkg/agent/tools/web_fetch_test.go` Validates network egress hardening: the `securityTransport` interceptor, IP-block policy, CIDR allow list, localhost exceptions, and TUI permission prompt. @@ -129,6 +129,11 @@ Validates network egress hardening: the `securityTransport` interceptor, IP-bloc | `TestWebFetchHardeningIPBlock` | Raw IP addresses (e.g. `192.168.1.1`) rejected — FQDN required | | `TestWebFetchHardeningCIDRAllow` | CIDR allow list (`192.168.1.0/24`) respected — IP falls in range allowed | | `TestWebFetchHardeningLocalhostAlwaysAllowed` | `127.0.0.1` / `::1` always allowed — no config needed | +| `TestWebFetchHardeningIPv6LiteralBlocked` | Bracketed IPv6 literals (`[::1]:6379`, `[fd00::…]`) rejected as raw IPs — never reach the trust hierarchy | +| `TestWebFetchHardeningIPv6LocalhostAllowed` | `[::1]` treated as localhost — passes the gate without a prompt | +| `TestWebFetchDialGateResolvedAddress` | DNS-rebinding defense: hostnames resolving to private/link-local/metadata ranges refused at the socket; loopback and public addresses pass | +| `TestWebFetchDialGateHonorsConfigOverrides` | `allow_raw_ips` and `ip_allow_list` CIDRs exempt the dial gate, matching the raw-IP policy | +| `TestWebFetchDialGateLocalhostHostnameAllowed` | Local dev flows survive the default gate — `127.0.0.1` httptest server reachable through the guarded dialer | | `TestWebFetchHardeningUserPromptVerified` | Unknown host triggers TUI prompt — user granted access | | `TestWebFetchHardeningUserPromptDenyVerified` | Unknown host triggers TUI prompt — user denied access → request blocked | diff --git a/go.mod b/go.mod index 1fb8120..03f2b55 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/hackafterdark/phosphor -go 1.27.0 +go 1.27.3 require ( charm.land/bubbles/v2 v2.2.1 diff --git a/pkg/agent/agent.go b/pkg/agent/agent.go index 4b5d176..d25fdfd 100644 --- a/pkg/agent/agent.go +++ b/pkg/agent/agent.go @@ -2196,6 +2196,14 @@ func (a *sessionAgent) Summarize(ctx context.Context, sessionID string, opts fan if systemPromptPrefix != "" { prepared.Messages = append([]fantasy.Message{fantasy.NewSystemMessage(systemPromptPrefix)}, prepared.Messages...) } + // Route the summarizer request through the same wire defenses as + // the main agent loop: the full history (including raw stored + // reasoning) rides this call, so it must meet the defang and the + // last-resort secret/PII mask too. + prepared.Messages = defangOutgoingMessages(prepared.Messages) + if a.outgoingRedactionEnabled() { + prepared.Messages = a.redactOutgoingMessages(prepared.Messages) + } return callContext, prepared, nil }, OnReasoningDelta: func(id string, text string) error { @@ -2914,7 +2922,9 @@ func (a *sessionAgent) GenerateTitle(ctx context.Context, sessionID string, user } streamCall := fantasy.AgentStreamCall{ - Prompt: userPrompt, + // The raw user prompt rides this call verbatim; give it the same + // defang and last-resort mask the main loop applies to history. + Prompt: a.redactOutgoingText(tools.DefangSpecialTokens(userPrompt)), } var resp *fantasy.AgentResult diff --git a/pkg/agent/agentic_fetch_tool.go b/pkg/agent/agentic_fetch_tool.go index 5c075dc..e2b0a30 100644 --- a/pkg/agent/agentic_fetch_tool.go +++ b/pkg/agent/agentic_fetch_tool.go @@ -172,7 +172,7 @@ func (c *coordinator) agenticFetchTool(_ context.Context, client *http.Client) ( return fantasy.ToolResponse{}, errors.New("small model provider not configured") } - webFetchTool := tools.NewWebFetchTool(tmpDir, client, c.tuiAllowPrompt) + webFetchTool := tools.NewWebFetchTool(tmpDir, client, c.cfg.Config(), c.tuiAllowPrompt) webSearchTool := tools.NewWebSearchTool(client) fetchTools := []fantasy.AgentTool{ webFetchTool, diff --git a/pkg/agent/outgoing_redact.go b/pkg/agent/outgoing_redact.go index 10eaaea..7236e4c 100644 --- a/pkg/agent/outgoing_redact.go +++ b/pkg/agent/outgoing_redact.go @@ -1,6 +1,7 @@ package agent import ( + "errors" "log/slog" "regexp" "strings" @@ -149,17 +150,61 @@ func (a *sessionAgent) redactOutgoingMessages(msgs []fantasy.Message) []fantasy. parts := make([]fantasy.MessagePart, len(msg.Content)) changed := false for j, part := range msg.Content { - tp, ok := part.(fantasy.TextPart) - if !ok { + switch p := part.(type) { + case fantasy.TextPart: + next := a.redactOutgoingText(p.Text) + if next != p.Text { + p.Text = next + changed = true + } + parts[j] = p + case fantasy.ToolCallPart: + // Tool-call inputs (bash command lines, MCP arguments, edit + // bodies) are replayed verbatim on every subsequent request; + // a credential typed into an argument would otherwise ride + // the wire in plaintext forever. + next := a.redactOutgoingText(p.Input) + if next != p.Input { + p.Input = next + changed = true + } + parts[j] = p + case fantasy.ToolResultPart: + // Text results are normally masked per-tool, but anything + // that reaches a result outside the redacted surfaces must + // still meet the last-resort mask at the wire. Error text + // and media-attached text ride the wire verbatim too. + switch out := p.Output.(type) { + case fantasy.ToolResultOutputContentText: + next := a.redactOutgoingText(out.Text) + if next != out.Text { + p.Output = fantasy.ToolResultOutputContentText{Text: next} + changed = true + } + case fantasy.ToolResultOutputContentError: + if out.Error == nil { + break + } + next := a.redactOutgoingText(out.Error.Error()) + if next != out.Error.Error() { + p.Output = fantasy.ToolResultOutputContentError{Error: errors.New(next)} + changed = true + } + case fantasy.ToolResultOutputContentMedia: + if out.Text == "" { + break + } + next := a.redactOutgoingText(out.Text) + if next != out.Text { + out.Text = next + p.Output = out + changed = true + } + } + parts[j] = p + default: parts[j] = part - continue - } - next := a.redactOutgoingText(tp.Text) - if next != tp.Text { - tp.Text = next - changed = true } - parts[j] = tp } if changed { out[i].Content = parts diff --git a/pkg/agent/outgoing_redact_test.go b/pkg/agent/outgoing_redact_test.go index b686606..3fcc47b 100644 --- a/pkg/agent/outgoing_redact_test.go +++ b/pkg/agent/outgoing_redact_test.go @@ -335,3 +335,89 @@ func TestSessionAgent_OutgoingRedaction_ForceOnlyMasksThroughCallSite(t *testing require.NotContains(t, textOf(out[0]), secret) require.Contains(t, textOf(out[0]), " 0 { small.MaxTokens = smallModelSelected.MaxTokens @@ -975,22 +964,12 @@ func configureSelectedModels(store *ConfigStore, knownProviders []catwalk.Provid } } - // When small isn't explicitly configured and the provider isn't a - // known built-in, use the large model as the small model. This - // prevents two different models from being requested concurrently - // for local/openai-compat providers. - if !smallModelConfigured { - isKnownProvider := false - for _, kp := range knownProviders { - if string(kp.ID) == small.Provider { - isKnownProvider = true - break - } - } - if !isKnownProvider { - slog.Warn("Using large model as small model for unknown provider", "provider", large.Provider, "model", large.Model) - small = large - } + // When small isn't configured it falls back to the large model + // rather than auto-selecting one from the provider catalog — the + // auto-pick landed on arbitrary providers users never chose. + if _, ok := c.Models[SelectedModelTypeSmall]; !ok { + slog.Info("Small model not configured, falling back to large model", "provider", large.Provider, "model", large.Model) + small = large } c.Models[SelectedModelTypeLarge] = large diff --git a/pkg/config/load_test.go b/pkg/config/load_test.go index 0b3126a..24c2bf6 100644 --- a/pkg/config/load_test.go +++ b/pkg/config/load_test.go @@ -1277,9 +1277,10 @@ func TestConfig_defaultModelSelection(t *testing.T) { require.Equal(t, "large-model", large.Model) require.Equal(t, "openai", large.Provider) require.Equal(t, int64(1000), large.MaxTokens) - require.Equal(t, "small-model", small.Model) - require.Equal(t, "openai", small.Provider) - require.Equal(t, int64(500), small.MaxTokens) + // The small slot is intentionally left unset so consumers fall + // back to the large model instead of guessing one from the catalog. + require.Empty(t, small.Model) + require.Empty(t, small.Provider) }) t.Run("should error if no providers configured", func(t *testing.T) { knownProviders := []catwalk.Provider{ @@ -1471,9 +1472,10 @@ func TestConfig_defaultModelSelection(t *testing.T) { require.Equal(t, "large-model", large.Model) require.Equal(t, "openai", large.Provider) require.Equal(t, int64(1000), large.MaxTokens) - require.Equal(t, "small-model", small.Model) - require.Equal(t, "openai", small.Provider) - require.Equal(t, int64(500), small.MaxTokens) + // The small slot is intentionally left unset so consumers fall + // back to the large model instead of guessing one from the catalog. + require.Empty(t, small.Model) + require.Empty(t, small.Provider) }) } @@ -1776,9 +1778,10 @@ func TestConfig_configureSelectedModels(t *testing.T) { require.Equal(t, "larger-model", large.Model) require.Equal(t, "openai", large.Provider) require.Equal(t, int64(2000), large.MaxTokens) - require.Equal(t, "small-model", small.Model) + // Small is unset, so it falls back to the configured large model. + require.Equal(t, "larger-model", small.Model) require.Equal(t, "openai", small.Provider) - require.Equal(t, int64(500), small.MaxTokens) + require.Equal(t, int64(2000), small.MaxTokens) }) t.Run("should be possible to use multiple providers", func(t *testing.T) { knownProviders := []catwalk.Provider{ @@ -1843,6 +1846,51 @@ func TestConfig_configureSelectedModels(t *testing.T) { require.Equal(t, int64(300), small.MaxTokens) }) + t.Run("should fall back to the large model when the small model is invalid", func(t *testing.T) { + knownProviders := []catwalk.Provider{ + { + ID: "openai", + APIKey: "abc", + DefaultLargeModelID: "large-model", + DefaultSmallModelID: "small-model", + Models: []catwalk.Model{ + { + ID: "large-model", + DefaultMaxTokens: 1000, + }, + { + ID: "small-model", + DefaultMaxTokens: 500, + }, + }, + }, + } + + cfg := &Config{ + Models: map[SelectedModelType]SelectedModel{ + "large": { + Model: "large-model", + Provider: "openai", + }, + "small": { + Model: "ghost-model", + Provider: "ghost-provider", + }, + }, + } + cfg.setDefaults("/tmp", "") + env := env.NewFromMap(map[string]string{}) + resolver := NewShellVariableResolver(env) + err := cfg.configureProviders(context.Background(), testStore(cfg), env, resolver, knownProviders) + require.NoError(t, err) + + err = configureSelectedModels(testStore(cfg), knownProviders, false) + require.NoError(t, err) + small := cfg.Models[SelectedModelTypeSmall] + require.Equal(t, "large-model", small.Model) + require.Equal(t, "openai", small.Provider) + }) + t.Run("should override the max tokens only", func(t *testing.T) { knownProviders := []catwalk.Provider{ { diff --git a/pkg/egress/transport.go b/pkg/egress/transport.go index c215e3f..ee37879 100644 --- a/pkg/egress/transport.go +++ b/pkg/egress/transport.go @@ -8,6 +8,7 @@ import ( "net" "net/http" "net/url" + "syscall" "time" ) @@ -58,32 +59,72 @@ func envProxyWhenBrokerInactive(r *http.Request) (*url.URL, error) { return http.ProxyFromEnvironment(r) } +// TransportOption customizes the RoundTripper returned by [Transport]. +type TransportOption func(*RoundTripperPolicy) + +// WithDialGuard composes guard into the resolved-address checks of the direct +// transport used while the broker is active. Without it, wrapping a guarded +// base transport would silently drop the caller's dial gate on the active +// path, because the active path replaces the base transport. +func WithDialGuard(guard func(ctx context.Context, network, address string) error) TransportOption { + return func(rt *RoundTripperPolicy) { + rt.dialGuard = guard + } +} + // Transport wraps base with the broker-aware policy and sentinel resolver. A nil // base uses a broker-aware clone of http.DefaultTransport. The wrapper consults // the frozen broker state on each request, so a client constructed before the // coordinator arms the tier still honors it after [Start]. -func Transport(base http.RoundTripper) http.RoundTripper { +func Transport(base http.RoundTripper, opts ...TransportOption) http.RoundTripper { if base == nil { base = NewHTTPTransport() } - return &RoundTripperPolicy{ + rt := &RoundTripperPolicy{ store: Store(), inner: base, direct: newDirectTransport(), policyFunc: brokerPolicy, } + for _, opt := range opts { + opt(rt) + } + if rt.dialGuard != nil { + if tr, ok := rt.direct.(*http.Transport); ok { + tr.DialContext = dialPolicyAwareGuarded(rt.dialGuard) + } + } + return rt } func dialPolicyAware(ctx context.Context, network, address string) (net.Conn, error) { - policy, active, failed := brokerPolicy() - if failed { - return nil, ErrBrokerFailedToStart - } - dialer := policy.Dialer() - if !active { - dialer = &net.Dialer{KeepAlive: DefaultKeepAlive} + return dialPolicyAwareGuarded(nil)(ctx, network, address) +} + +func dialPolicyAwareGuarded(guard func(ctx context.Context, network, address string) error) func(context.Context, string, string) (net.Conn, error) { + return func(ctx context.Context, network, address string) (net.Conn, error) { + policy, active, failed := brokerPolicy() + if failed { + return nil, ErrBrokerFailedToStart + } + dialer := &net.Dialer{KeepAlive: DefaultKeepAlive} + if active { + dialer.ControlContext = func(ctx context.Context, network, address string, c syscall.RawConn) error { + if err := policy.ControlContext(ctx, network, address, c); err != nil { + return err + } + if guard != nil { + return guard(ctx, network, address) + } + return nil + } + } else if guard != nil { + dialer.ControlContext = func(ctx context.Context, network, address string, _ syscall.RawConn) error { + return guard(ctx, network, address) + } + } + return dialer.DialContext(ctx, network, address) } - return dialer.DialContext(ctx, network, address) } // RoundTripperPolicy is an [http.RoundTripper] that enforces the net-policy and @@ -95,6 +136,7 @@ type RoundTripperPolicy struct { store TokenResolver inner http.RoundTripper direct http.RoundTripper + dialGuard func(ctx context.Context, network, address string) error } // NewRoundTripper builds a broker transport over inner (nil means a broker-aware diff --git a/pkg/phosphordocs/content/CONFIGURATION.md b/pkg/phosphordocs/content/CONFIGURATION.md index e26f4b6..50f3bcb 100644 --- a/pkg/phosphordocs/content/CONFIGURATION.md +++ b/pkg/phosphordocs/content/CONFIGURATION.md @@ -25,9 +25,17 @@ order they are loaded (later files win on conflicts): Windows uses the same XDG logic for the global config, so the default is `%USERPROFILE%\.config\phosphor\phosphor.json` unless `XDG_CONFIG_HOME` is set. +All files are **deep-merged per JSON key**, not replaced wholesale — a project +file that sets only `models.large` leaves everything else in your global file +intact. The files with the higher numbers above override the lower ones for +any key they define. + The **global data config** is where Phosphor persists things it learns at runtime (recently used models, per-provider settings saved from the TUI). The **project config** is safe to commit to a repository to share settings with your team. +If the TUI ever writes something to the global data config that you disagree +with (e.g. a `models.small` selection), edit or delete that key there — it +overrides the same key in your global config. The workspace also gets a **data directory** (default `.phosphor/` at the project root, configurable via `options.data_directory` or `--data-dir`). It holds the @@ -256,7 +264,9 @@ The minimum viable cloud setup: ``` `large` is the primary coding/reasoning model; `small` handles title generation -and lightweight sub-tasks. Model entries also accept sampling overrides +and lightweight sub-tasks (summarization, `agentic_fetch`). If `small` is +omitted it falls back to `large` — Phosphor never auto-selects a small model +from another provider. Model entries also accept sampling overrides (`temperature`, `top_p`, `top_k`, `max_tokens`, `seed`, `min_p`, `repetition_penalty`, `stop`, …) plus `reasoning_effort` (OpenAI-style), `think` (Anthropic), and `max_thinking_tokens`. diff --git a/pkg/phosphordocs/content/MODELS_AND_PROVIDERS_CONFIG.md b/pkg/phosphordocs/content/MODELS_AND_PROVIDERS_CONFIG.md index e9eb005..5e3046b 100644 --- a/pkg/phosphordocs/content/MODELS_AND_PROVIDERS_CONFIG.md +++ b/pkg/phosphordocs/content/MODELS_AND_PROVIDERS_CONFIG.md @@ -199,6 +199,22 @@ The `models` section selects which model to use for different tasks. Phosphor us - **`large`** — Primary model for coding, reasoning, and complex tasks - **`small`** — Fallback model for title generation, quick responses, and sub-agents +If `small` is omitted, it falls back to the `large` model rather than +auto-selecting one from the provider catalog. Every consumer of the small +slot (title generation, summarization, `agentic_fetch`) uses the large model +until you pin `models.small` explicitly. A `models.small` whose provider or +model no longer resolves also falls back to `large`. + +The list of providers and models offered in the model dialog is **not** +stored in your config: it comes from the [Catwalk](https://catwalk.charm.land) +catalog, fetched at runtime and cached alongside the global data config +(`%LOCALAPPDATA%\phosphor\providers.json` on Windows, +`~/.local/share/phosphor/providers.json` on Linux/macOS; override the source +with `CATWALK_URL`). A known provider appears in the dialog as soon as its +API key is available in your environment — that does **not** mean it is +selected. `phosphor.json` only pins or overrides choices; anything you don't +pin resolves against the catalog. + ### Model Fields | Field | Type | Required | Description | diff --git a/pkg/phosphordocs/content/security/NETWORK_EGRESS_HARDENING.md b/pkg/phosphordocs/content/security/NETWORK_EGRESS_HARDENING.md index 787b8f9..a53842a 100644 --- a/pkg/phosphordocs/content/security/NETWORK_EGRESS_HARDENING.md +++ b/pkg/phosphordocs/content/security/NETWORK_EGRESS_HARDENING.md @@ -31,7 +31,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* ## Components -### `internal/agent/tools/web_fetch.go` +### `pkg/agent/tools/web_fetch.go` - `securityTransport` — custom `http.RoundTripper` that intercepts each request - `allowList` — in-memory session allow cache (cleared on session restart) @@ -42,7 +42,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* - `FetchURLAndConvertWithTrust` — wraps `FetchURLAndConvert` with the trust check - `NewWebFetchTool` — constructs tool with `allowFn` callback (TUI permission prompt) -### `internal/agent/tools/web_search.go` +### `pkg/agent/tools/web_search.go` - `NewWebSearchTool` — same security contract as `web_fetch`; panics if `client` is nil - `sanitizeSearchResults` — regex scrubber redacts: @@ -51,7 +51,7 @@ Phosphor's web-fetch and web-search tools now operate under an **Adaptive Trust* - `randomDelay` — adds 100–150ms random delay per round-trip to stay under DDG's request-budget threshold - OTel span tagged with `attribute.Int("gen_ai.tool.web_search.results", len(results))` -### `internal/agent/coordinator.go` +### `pkg/agent/coordinator.go` - `tuiAllowPrompt` field added to `coordinator` struct - Wired in `NewCoordinator` via `c.permissions.Request` with synthetic tool-call ID (`tui:allow:`) @@ -101,8 +101,8 @@ Phosphor's `web_fetch` and `web_search` tools consult the user's `phosphor.json` | Field | Type | Default | Description | |---|---|---|---| -| `IPAllowList` | `[]string` | `[]` | CIDR or literal IPs allowed to be reached | -| `AllowRawIPs` | `bool` | `false` | Opt-in for raw IP access (default: FQDN required) | +| `IPAllowList` | `[]string` | `[]` | CIDR or literal IPs allowed to be reached (consulted only when `allow_raw_ips` is true) | +| `AllowRawIPs` | `bool` | `false` | Gate for raw IP access; IPs are allowed only if they also match `IPAllowList` (default: FQDN required) | ### Example `phosphor.json` @@ -126,9 +126,8 @@ Phosphor's `web_fetch` and `web_search` tools consult the user's `phosphor.json` When `securityTransport` intercepts a request to a raw IP: 1. **127.0.0.1 / ::1** (localhost) → always allowed, no config needed -2. **`IPAllowList`** → literal match or CIDR match via `matchCIDR` → allowed -3. **`AllowRawIPs = true`** → allowed (if no CIDR match) -4. **No match** → TUI prompt shows "WebFetch is requesting access to [host]. Allow this domain?" +2. **`AllowRawIPs = true` + `IPAllowList`** → literal match or CIDR match via `matchCIDR` → allowed +3. **Anything else** → the request fails closed with an error; raw IPs never reach the TUI prompt (prompts apply to hostnames only) ## Bash Network Egress Policy @@ -172,30 +171,50 @@ guarantees. ## Security Properties - **Fail-closed**: no network requests succeed until the agent confirms the domain is trusted -- **IP-block policy**: raw IP addresses (e.g., `10.x.x.x`, `192.168.x.x`, public IPs) are rejected — FQDNs required +- **IP-block policy**: raw IP addresses are rejected for both IPv4 and IPv6 literal forms (e.g., `10.x.x.x`, `192.168.x.x`, `[::1]`, `[fd00::…]`) — FQDNs required +- **Resolved-address validation**: the same IP policy is enforced at dial time on the *resolved* socket address, so an approved hostname whose DNS answer rebinds into a private, link-local, or cloud-metadata range is refused at the socket (DNS-rebinding defense). Loopback stays allowed; `IPAllowList`/`AllowRawIPs` exempt like raw IPs; when an environment proxy is configured the operator's proxy is the egress decision and the strict path remains the egress broker tier - **Opt-in overrides**: `AllowRawIPs` + `IPAllowList` (CIDR) allow raw IPs for local/network development - **Layered trust**: workspace + global allow lists + session allow list + user prompt - **OTel observability**: every allow/deny/event is captured as an OTel span with structured attributes - **Cacheable**: user approvals persist across requests (in-memory cache) but are session-local - **Shared transport**: both `web_fetch` and `web_search` share the same `*http.Client` → same `securityTransport` -- **Fail-closed**: no network requests succeed until the agent confirms the domain is trusted -- **IP-block policy**: raw IP addresses (e.g., `10.x.x.x`, `192.168.x.x`, public IPs) are rejected — FQDNs required -- **Layered trust**: workspace + global allow lists + session allow list + user prompt -- **OTel observability**: every allow/deny/event is captured as an OTel span with structured attributes -- **Cacheable**: user approvals persist across requests (in-memory cache) but are session-local -- **Shared transport**: both `web_fetch` and `web_search` share the same `*http.Client` → same `securityTransport` +## TLS Trust and CA Compromise + +The IP-block policy and trust hierarchy govern *where* the agent may connect; +certificate validation governs *whether the peer is who it claims to be*. The +two layers are independent, and an AI agent depends on the second more than +ordinary software does: the agent carries provider API keys in every request, +fetches attacker-chosen URLs by design, and can download and execute code from +hosts it has never seen. A compromised certificate authority — one whose +intermediate keys leak and can sign certs for *any* domain — silently defeats +every hostname-level defense at once (allow lists, IP blocks, FQDN +requirements). + +Phosphor performs no custom certificate handling. TLS verification is delegated +to the runtime trust store: + +- **Windows / macOS**: the platform certificate store, so OS security updates + and browser-grade revocation (CRLSets, OneCRL) cover CA compromise. +- **Linux**: Phosphor builds with `CGO_ENABLED=0`, so Go uses its compiled-in + copy of the Mozilla root program. Keeping the Go toolchain patched — + `go.mod` pins the minimum version — is therefore part of the security + posture, not just a build detail. + +Concretely: when an intermediate CA in a trusted root program is compromised, +the remedy is a toolchain/OS update, and users should update promptly the same +way they would for a browser. ## Files Modified -- `internal/agent/tools/web_fetch.go` — added security transport, allow lists, TUI prompt wiring -- `internal/agent/tools/web_search.go` — added sanitization, OTel tagging, random delay, nil client panic -- `internal/agent/coordinator.go` — added `tuiAllowPrompt` field and wiring -- `internal/agent/agentic_fetch_tool.go` — updated call site to pass `c.tuiAllowPrompt` +- `pkg/agent/tools/web_fetch.go` — added security transport, allow lists, TUI prompt wiring +- `pkg/agent/tools/web_search.go` — added sanitization, OTel tagging, random delay, nil client panic +- `pkg/agent/coordinator.go` — added `tuiAllowPrompt` field and wiring +- `pkg/agent/agentic_fetch_tool.go` — updated call site to pass `c.tuiAllowPrompt` ## Notes -- `go.mod` requires Go 1.26.4+ (running 1.25.0 locally — LSP errors may appear for tooling checks) +- `go.mod` requires Go 1.27.3+ (patched trust store; see TLS Trust and CA Compromise above) - `fmt`, `net`, `strings`, `html/template`, `encoding/json`, `time`, `embed` imports are all used in body - `maybeDelaySearch` removed from `web_search.go` (already present in `search.go`) - `randomDelay` added for DDG request-budget threshold