Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 22 additions & 117 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,56 +14,17 @@ your app ──▶ cordon ──▶ api.openai.com / api.anthropic.com
└─ your app receives: email john@acme.com re card 4012-8888-8888-1881
```

## Run it
## Quickstart

```bash
docker run -d --name cordon --init -p 127.0.0.1:8080:8080 \
-v cordon-data:/app/data -e ADMIN_TOKEN=change-me \
ghcr.io/askalf/cordon:v0.2.0
```

Then change one thing in your client: the base URL.
Then change one thing in your client: the base URL. Anthropic clients use `http://localhost:8080`, OpenAI clients use `http://localhost:8080/v1`. Your provider key goes through untouched; cordon never holds it. Every response says what it did: `X-Redacted: 2`, `X-Redacted-Types: EMAIL:1,CREDIT_CARD:1`.

```bash
# Anthropic client: base URL http://localhost:8080 (was https://api.anthropic.com)
curl localhost:8080/v1/messages \
-H 'content-type: application/json' \
-H "x-api-key: $ANTHROPIC_API_KEY" -H 'anthropic-version: 2023-06-01' \
-d '{"model":"claude-haiku-4-5","max_tokens":64,"messages":[{"role":"user",
"content":"email john@acme.com re card 4012-8888-8888-1881"}]}'
```

```bash
# OpenAI client: base URL http://localhost:8080/v1 (was https://api.openai.com/v1)
curl localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -H "authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user",
"content":"email john@acme.com re card 4012-8888-8888-1881"}]}'
```

Your provider key goes through untouched; cordon never holds it. What comes back, captured from the published image:

```
HTTP/1.1 200 OK
X-Redact-Mode: reversible
X-Redacted: 2
X-Redacted-Types: EMAIL:1,CREDIT_CARD:1

{"content":[{"type":"text","text":"email john@acme.com re card 4012-8888-8888-1881"}], ...}
```

What the provider was sent:

```
"content":"email <EMAIL_5285D1_1> re card <CREDIT_CARD_5285D1_1>"
```

The line appended to the audit log (counts and types, never values):

```json
{"ts":1790043464004,"tenant":"auth:cdba95a3…","provider":"anthropic","model":"claude-haiku-4-5",
"mode":"reversible","entityCounts":{"EMAIL":1,"CREDIT_CARD":1},"total":2,"prevHash":"0","hash":"7b6e07…"}
```
A full captured round trip (both clients, the headers, what the provider was sent and the audit line) is in [docs/reference.md](docs/reference.md#a-full-round-trip).

## What it catches

Expand All @@ -78,91 +39,35 @@ Deterministic detection: regex plus checksum validators, no ML dependencies, ful

All four sets are on by default; narrow per tenant or per request with `X-Redact-Sets: pii,pci`.

## What it does not do

- **Names, free-text addresses, medical conditions.** There is no NER. A person's name in prose passes through. The detector is an interface (`src/detect`), so a Presidio-style sidecar can be added; it is not included.
- **Embeddings, `count_tokens`, images.** Only the three generation endpoints (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`) are redacted; other `/v1/*` paths, including `/v1/responses/{id}`, pass through verbatim. Image and file parts are left untouched.
- **Token counts.** Streaming usage figures are the provider's, computed on the de-identified text.

If you need one of those, say so in an issue. The scope above is deliberate, not accidental.

## Modes

Per tenant (policy) or per request (`X-Redact-Mode` header):
## How it behaves

- **`reversible`** *(default)*: placeholders go up, real values come back in the reply, including mid-stream. Tokens carry a per-request random nonce (`<EMAIL_5285D1_1>`, not `<EMAIL_1>`) so a caller's own placeholder-shaped text can never be rewritten to a real value.
- **`strip`**: irreversible placeholders (`[EMAIL]`); nothing is restored. For when the answer never needs the real value.
- **`off`**: passthrough, still audited as a bypass.
- **Fails closed.** If detection throws, the request is **blocked**, never forwarded with PII intact (`FAIL_MODE=closed`, the default). The test suite asserts the upstream is never called on that path.
- **Three modes**, per tenant or per request (`X-Redact-Mode`):
- **`reversible`** *(default)*: placeholders go up, real values come back in the reply, including mid-stream. Tokens carry a per-request random nonce (`<EMAIL_5285D1_1>`, not `<EMAIL_1>`) so a caller's own placeholder-shaped text can never be rewritten to a real value.
- **`strip`**: irreversible placeholders (`[EMAIL]`); nothing is restored. For when the answer never needs the real value.
- **`off`**: passthrough, still audited as a bypass.
- **Tamper-evident audit.** Every request appends a hash-chained record of counts and types, never values; `npm run audit` verifies the chain.
- **Per-tenant policy**: consistent pseudonyms, data residency (regional upstreams), durable policy store.
- **Signed releases**: multi-arch GHCR images with keyless Sigstore provenance and an SBOM.

```
X-Redact-Mode: strip → "text":"email [EMAIL] re card [CREDIT_CARD]"
```

## Fail closed

If detection throws, the request is **blocked**, never forwarded with PII intact (`FAIL_MODE=closed`, the default). The test suite asserts the upstream is never called on that path.

## Audit

Every request appends one record to a hash-chained JSONL log (`AUDIT_LOG`): `{ts, tenant, provider, model, mode, entityCounts, sets, total, prevHash, hash}` with `hash = sha256(prevHash + canonicalJSON(record))`. Records carry counts and types only. Any edit, deletion or reorder breaks the chain.

```bash
npm run audit # verify the chain, print a tamper report
curl localhost:8080/admin/audit/verify -H 'x-admin-token: …'
```

## Per-tenant policy

```bash
curl localhost:8080/admin/tenant -H 'x-admin-token: …' -H 'content-type: application/json' \
-d '{"tenant":"acme","mode":"reversible","activeSets":["pii","pci"],
"consistentPseudonyms":true,"upstreamOverride":{"anthropic":"https://eu.anthropic.example"}}'
```

- **Consistent pseudonyms**: `<EMAIL_3F2A…>` derived as `HMAC(TENANT_SECRET, value)`, so the same person maps to the same token across requests (the model can correlate) while the value is never stored. Requires a strong `TENANT_SECRET` (16+ chars); this mode fails closed without one. `ALLOW_WEAK_PSEUDONYM_SECRET=1` overrides for dev only.
- **Data residency**: route a tenant to a regional upstream base.
- **Durable policy**: `POLICY_STORE=./policies.json` persists tenant policy across restarts on the same volume as the audit log; unset keeps it in memory.
- **Tenant identity**: `X-Tenant: <id>`, else derived from the API key.

Ops: `GET /healthz`, `GET /metrics` (and `/metrics.prom`), `GET /dashboard` (single-file view of redactions by type, mode and set mix, fail-closed count, tenant policies, audit-chain status), `GET /admin/stats`. Admin routes require `x-admin-token` when `ADMIN_TOKEN` is set.

## Configuration

See [`.env.example`](./.env.example). The knobs that matter: `FAIL_MODE` (default `closed`), `DEFAULT_MODE`, `ACTIVE_SETS`, `CONSISTENT_PSEUDONYMS` with `TENANT_SECRET`, `AUDIT_LOG`, `ADMIN_TOKEN`, `POLICY_STORE`, `OPENAI_BASE` / `ANTHROPIC_BASE`.

## Deploy

No cache, no shared state: the vault is per request and ephemeral, policy is a JSON file, the audit log is a local file. One container, no Redis or database.

```bash
docker compose up -d --build # from a clone: cordon on 127.0.0.1:8080, audit log on a volume
./deploy.sh # idempotent clone/pull/build/healthcheck to a remote box
```

Every tagged release publishes a multi-arch image (linux/amd64, linux/arm64) to GHCR with keyless Sigstore provenance and an SBOM. Verify it came from this repository's release workflow:

```bash
gh attestation verify oci://ghcr.io/askalf/cordon:v0.2.0 --repo askalf/cordon
```

Sharing one Claude or ChatGPT subscription through [dario](https://github.com/askalf/dario) without leaking PII: dario's [cordon integration guide](https://github.com/askalf/dario/blob/main/docs/integrations/cordon.md).
## What it does not do

## Development
- **Names, free-text addresses, medical conditions.** There is no NER. A person's name in prose passes through. The detector is an interface (`src/detect`), so a Presidio-style sidecar can be added; it is not included.
- **Embeddings, `count_tokens`, images.** Only the three generation endpoints (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`) are redacted; other `/v1/*` paths, including `/v1/responses/{id}`, pass through verbatim. Image and file parts are left untouched.
- **Token counts.** Streaming usage figures are the provider's, computed on the de-identified text.

```bash
npm install
npm run dev # cordon on :8080 from source
npm test
```
If you need one of those, say so in an issue. The scope above is deliberate, not accidental.

The test suite runs against a stub upstream that echoes the body it received, so every suite asserts two things at once: the model never saw raw PII, and the client still got the real values back.
## Reference

- **detect**: every pattern fires; Luhn / mod-97 / ABA reject false positives; set gating; overlap resolution.
- **apply**: string and content-array bodies de-identified with structure preserved, images untouched; reversible round trip.
- **streaming**: a placeholder split across a frame boundary is still restored.
- **strip / off / fail-closed**: strip persists placeholders; off passes through; a detection error blocks and the upstream is never called.
- **audit**: the chain verifies, tampering is detected, the log is proven to contain no values.
- **passthrough**: `count_tokens` and other non-generation paths forward verbatim.
- [docs/reference.md](docs/reference.md): the full round trip, audit log format and verification, per-tenant policy, ops endpoints (`/healthz`, `/metrics`, `/dashboard`, `/admin/*`), configuration.
- [docs/deploy.md](docs/deploy.md): compose, `deploy.sh`, verifying a release's attestation, using cordon with [dario](https://github.com/askalf/dario).
- [docs/development.md](docs/development.md): running from source and what the test suite proves.
- [CHANGELOG.md](CHANGELOG.md) · [SECURITY.md](SECURITY.md) · [CONTRIBUTING.md](CONTRIBUTING.md)

## Part of Own Your Stack

Expand Down
18 changes: 18 additions & 0 deletions docs/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Deploying cordon

Back to the [README](../README.md).

No cache, no shared state: the vault is per request and ephemeral, policy is a JSON file, the audit log is a local file. One container, no Redis or database.

```bash
docker compose up -d --build # from a clone: cordon on 127.0.0.1:8080, audit log on a volume
./deploy.sh # idempotent clone/pull/build/healthcheck to a remote box
```

Every tagged release publishes a multi-arch image (linux/amd64, linux/arm64) to GHCR with keyless Sigstore provenance and an SBOM. Verify it came from this repository's release workflow:

```bash
gh attestation verify oci://ghcr.io/askalf/cordon:v0.2.0 --repo askalf/cordon
```

Sharing one Claude or ChatGPT subscription through [dario](https://github.com/askalf/dario) without leaking PII: dario's [cordon integration guide](https://github.com/askalf/dario/blob/main/docs/integrations/cordon.md).
18 changes: 18 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Developing cordon

Back to the [README](../README.md).

```bash
npm install
npm run dev # cordon on :8080 from source
npm test
```

The test suite runs against a stub upstream that echoes the body it received, so every suite asserts two things at once: the model never saw raw PII, and the client still got the real values back.

- **detect**: every pattern fires; Luhn / mod-97 / ABA reject false positives; set gating; overlap resolution.
- **apply**: string and content-array bodies de-identified with structure preserved, images untouched; reversible round trip.
- **streaming**: a placeholder split across a frame boundary is still restored.
- **strip / off / fail-closed**: strip persists placeholders; off passes through; a detection error blocks and the upstream is never called.
- **audit**: the chain verifies, tampering is detected, the log is proven to contain no values.
- **passthrough**: `count_tokens` and other non-generation paths forward verbatim.
78 changes: 78 additions & 0 deletions docs/reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# cordon reference

Back to the [README](../README.md).

## A full round trip

Both client shapes, pointed at cordon instead of the provider:

```bash
# Anthropic client: base URL http://localhost:8080 (was https://api.anthropic.com)
curl localhost:8080/v1/messages \
-H 'content-type: application/json' \
-H "x-api-key: $ANTHROPIC_API_KEY" -H 'anthropic-version: 2023-06-01' \
-d '{"model":"claude-haiku-4-5","max_tokens":64,"messages":[{"role":"user",
"content":"email john@acme.com re card 4012-8888-8888-1881"}]}'
```

```bash
# OpenAI client: base URL http://localhost:8080/v1 (was https://api.openai.com/v1)
curl localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -H "authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user",
"content":"email john@acme.com re card 4012-8888-8888-1881"}]}'
```

Your provider key goes through untouched; cordon never holds it. What comes back, captured from the published image:

```
HTTP/1.1 200 OK
X-Redact-Mode: reversible
X-Redacted: 2
X-Redacted-Types: EMAIL:1,CREDIT_CARD:1

{"content":[{"type":"text","text":"email john@acme.com re card 4012-8888-8888-1881"}], ...}
```

What the provider was sent:

```
"content":"email <EMAIL_5285D1_1> re card <CREDIT_CARD_5285D1_1>"
```

The line appended to the audit log (counts and types, never values):

```json
{"ts":1790043464004,"tenant":"auth:cdba95a3…","provider":"anthropic","model":"claude-haiku-4-5",
"mode":"reversible","entityCounts":{"EMAIL":1,"CREDIT_CARD":1},"total":2,"prevHash":"0","hash":"7b6e07…"}
```

## Audit

Every request appends one record to a hash-chained JSONL log (`AUDIT_LOG`): `{ts, tenant, provider, model, mode, entityCounts, sets, total, prevHash, hash}` with `hash = sha256(prevHash + canonicalJSON(record))`. Records carry counts and types only. Any edit, deletion or reorder breaks the chain.

```bash
npm run audit # verify the chain, print a tamper report
curl localhost:8080/admin/audit/verify -H 'x-admin-token: …'
```

## Per-tenant policy

```bash
curl localhost:8080/admin/tenant -H 'x-admin-token: …' -H 'content-type: application/json' \
-d '{"tenant":"acme","mode":"reversible","activeSets":["pii","pci"],
"consistentPseudonyms":true,"upstreamOverride":{"anthropic":"https://eu.anthropic.example"}}'
```

- **Consistent pseudonyms**: `<EMAIL_3F2A…>` derived as `HMAC(TENANT_SECRET, value)`, so the same person maps to the same token across requests (the model can correlate) while the value is never stored. Requires a strong `TENANT_SECRET` (16+ chars); this mode fails closed without one. `ALLOW_WEAK_PSEUDONYM_SECRET=1` overrides for dev only.
- **Data residency**: route a tenant to a regional upstream base.
- **Durable policy**: `POLICY_STORE=./policies.json` persists tenant policy across restarts on the same volume as the audit log; unset keeps it in memory.
- **Tenant identity**: `X-Tenant: <id>`, else derived from the API key.

## Ops endpoints

`GET /healthz`, `GET /metrics` (and `/metrics.prom`), `GET /dashboard` (single-file view of redactions by type, mode and set mix, fail-closed count, tenant policies, audit-chain status), `GET /admin/stats`. Admin routes require `x-admin-token` when `ADMIN_TOKEN` is set.

## Configuration

See [`.env.example`](../.env.example). The knobs that matter: `FAIL_MODE` (default `closed`), `DEFAULT_MODE`, `ACTIVE_SETS`, `CONSISTENT_PSEUDONYMS` with `TENANT_SECRET`, `AUDIT_LOG`, `ADMIN_TOKEN`, `POLICY_STORE`, `OPENAI_BASE` / `ANTHROPIC_BASE`.
Loading