Start with the Claude Code/Codex user guide: local serving, direct-provider learning capture, or explicitly authorized hybrid routing. The guide includes startup examples, slash controls, billing boundaries and what is implemented versus planned.
Sentinel's services and helpers are Go, including capture hooks, CLI controls, status lines,
lab management and release/smoke helpers. make build produces llm-sentinel,
sentinel-gateway and sentinel-tools; all three ship as native release binaries.
MLX-Flash is an external inference service with its own dependencies. Sentinel
communicates over HTTP and launches only its executable. Future OSS/Jes decision
services use the same boundary, without Python adapters inside Sentinel. A thin
Claude Code mod provides a native panel and direct
controls through the existing Go helpers.
After a successful Build push on current main, Auto Release inspects the PR
merged at that exact commit. feat!: (including scopes) prepares a major release,
feat: a minor release, and fix:, perf: or refactor: a patch release.
The labels release:major, release:minor and release:patch override the title;
unknown or multiple bump labels fail. Other titles, including documentation and
maintenance changes, skip release preparation unless given a bump label.
Preparation opens a release/vX.Y.Z branch and a PR titled
chore: release vX.Y.Z, labeled release. It writes a v-prefixed VERSION and
a dated CHANGELOG.md section with commit subjects since the highest stable
Git tag. Existing history stays intact and Unreleased notes move into the new
section. Review and edit these draft notes, then merge the preparation PR
manually. Only its successful main push Build permits automatic publication;
the label, title, VERSION and matching changelog section must agree. The existing
release workflow still builds all three binaries with release version metadata.
Tags are immutable: retries reuse a tag only when it points at the same tested
commit. New release preparation requires current main. A reviewed release can
publish its exact successful Build commit after newer merges, provided that
commit is still an ancestor of main. Newer changes are excluded from that tag.
An existing preparation branch/PR is reused without force pushes;
closed preparations require maintainer attention.
Bot-created PRs do not start normal PR CI with GITHUB_TOKEN, so Auto Release
explicitly dispatches Build on the preparation branch with checkout pinned to
its commit SHA. Dispatch validation does not authorize publication. The workflow
uses repository GITHUB_TOKEN permissions for contents, pull requests and
Actions; repository settings must permit Actions to create PRs. No personal
access token is needed. Existing optional publishing credentials remain scoped
to the release workflow.
The deterministic Go helper can also prepare files locally:
git log --reverse --format=%s v3.5.0..HEAD > /tmp/sentinel-release-notes.txt
go run ./cmd/sentinel-tools release prepare --root . --latest v3.5.0 \
--bump minor --date 2026-10-06 --notes /tmp/sentinel-release-notes.txtUse the actual highest stable tag, intended bump, and review date for your release.
| New gateway capability | Documentation |
|---|---|
| Model translation and CLI feature reuse | Compatibility matrix |
| Client protocols and local model roles | Local setup, model roles |
| Subscription/direct learning and optional hybrid capture | Capture hooks, billing and policy |
| Live backend UI, telemetry and CLI controls | Browser dashboard, status line, commands |
| Small decision helpers → OSS bridge → our Jes | Migration design |
| Actual quality/accounting evidence | Client inspection, progress checks, dataset admission |
| Regression and real Metal API proofs | Integration verification |
| API, Claude Code and Codex proofs on each PR | CI compatibility proofs |
| Repository coverage, native quality gates and artifact provenance | Verified quality foundation |
The current decision policy is deterministic; Jes and the proposed Kev/Jeff decision adapters are not connected. Selected local models generate answers. Protocol compatibility does not establish equivalent Claude/OpenAI quality or measured cost savings. The older optimization-module documentation below is separate from this gateway's measured results. See the advertised-feature audit for source evidence and the distinction between connected features, standalone libraries and planned work.
| Feature status | Current implementation |
|---|---|
| Connected | Local Messages/Responses/Chat Completions, effort roles, validated client tools, opt-in capture/hybrid routing, controls, lab dashboard and MLX prefix-cache integration |
| Standalone or partial | Legacy semantic/exact answer cache, compression, graph/indexing, Batch API and optimization pipeline; these are not applied to live gateway requests |
| Planned | Jes decisions, semantic quality scoring, shadow comparison, automatic dataset admission and model training |
Older percentage-savings, coverage and feature-complete statements are historical
claims, not current product guarantees. Native cache counters measure prefill
reuse; they do not establish provider-billed monetary savings. The legacy
llm-sentinel dashboard has known metrics initialization defects; use
sentinel-tools dashboard for the current lab UI.
Run make lab-quality for explicit local task-quality probes
through Claude Messages and Codex Responses. The lab dashboard shows the latest
results; complete synthetic transcripts and failures remain in local archives.
These bounded checks do not establish full CLI or commercial model equivalence.
Run make lab-cli-quality for separate real Claude Code and Codex task probes,
including file edits and independently checked Go tests. Both suites appear in the dashboard.
Model-free protocol edge cases and fuzz campaigns
exercise all three client APIs, validated streams and malformed runtime output in CI.
These modules and historical design examples predate the current local gateway. They do not form a connected automatic optimization path. Their illustrative savings figures below have not been validated for the current product. Module tests and storage/client implementations exist, but several model integrations are stubs and runtime wiring is incomplete:
- ⚡ Batch API (50% savings) — Anthropic Batch API integration for non-interactive workloads
- 🔍 Knowledge Graph Queries (99% savings) — Answer relationship questions from indexed code
- 💾 Semantic Caching (98% savings) — Reuse responses for similar queries
- 📋 Exact Deduplication (100% savings) — Cache identical requests
- 🗜️ Input Optimization (40-60% savings) — Parameter compression, term shortening, structure optimization
- 🧠 Intent Detection — Route based on query complexity (detailed analysis vs quick answer)
- 📊 Transparent Cost Tracking — Show user when optimization applied vs fresh response
- 🔐 Security-First — Input validation, injection detection, configurable thresholds
For supported startup and measured behavior, use the current client guide above.
Anthropic Batch API for non-interactive workloads. 50% cost reduction on bulk analysis, overnight jobs, and scheduled tasks:
User: "Analyze all 50 files in repo for security"
↓
Non-interactive detector: Bulk workload detected
↓
Route to Batch API (instead of regular API)
↓
Anthropic processes in background (5min-24h)
↓
Cost: 50% discount on input/output tokens
↓
Savings: 50% per file × 50 files = 1250 tokens per file
Total: 125,000 tokens saved (est. $0.375 at Haiku pricing)
When to Use:
- ✅ Bulk file analysis, overnight jobs, scheduled tasks
- ✅ Non-urgent analysis that can wait 5-24 hours
- ❌ Interactive queries (too slow for user-facing)
- ❌ Real-time code review (use regular API)
Combined Savings (Batch API + Semantic Cache):
- Batch API: 50% discount
- Semantic cache on repeated analyses: 98% savings
- Combined: ~55% total when stacked
SQLite-backed knowledge graph for instant code relationship lookups. Answer queries from indexed code relationships without calling Claude:
User: "Find all functions calling authenticate()"
↓
Graph lookup: Traverse CALLED_BY edges from authenticate node
↓
Result: Instant relationship query (0 tokens)
↓
Savings: 99% vs Claude API call (2500+ tokens)
Supported Queries:
Find all functions calling X— Direct graph traversalShow functions that call authenticate— Pattern matching on relationshipsList imports of module Y— Edge traversalFind class inheritance chain— Path finding- Example: "Find all functions calling authenticate()" → 10ms, 0 tokens vs 2500+ from Claude
How It Works:
- CodeIndexer watches source files for changes (Go, Python, TypeScript)
- AST parsing extracts functions, classes, imports, and relationships
- Relationships stored in SQLite graph with confidence scores
- Recursive CTE queries find multi-hop paths efficiently (<10ms typical)
Moved below. See "Feature 3" in next section.
Reuse responses for similar queries using vector embeddings:
First query: "Find all functions that validate user input"
↓
Claude generates response (2500 tokens)
↓
Response cached with embedding
↓
Similar query: "List functions validating input"
↓
Embedding similarity: 0.92 (>0.85 threshold)
↓
Return cached response (50 tokens embedding cost)
↓
Savings: 98% (2500 - 50 = 2450 tokens saved)
Configuration:
- Threshold: 0.85 (strict, <0.1% false positive rate)
- False positive limit: 0.5% (auto-disable if exceeded)
- Hit rate target: 50%+ achievable on repeated query patterns
Cache identical requests using SHA256 hashing:
Request 1: "Find functions calling authenticate()"
↓
Hash: SHA256(tool + params + query) = abc123...
↓
Cache: Stored response
↓
Request 2: Identical query
↓
Hash match: Found in cache
↓
Return cached response (0 tokens)
↓
Savings: 100%
Typical Hit Rate: 20-30% of requests are repeats
Reduce input token usage through multiple compression techniques:
Before optimization:
├─ Tool definitions: 300+ tools listed (1000+ tokens)
├─ Parameter names: Long descriptive names (100+ tokens)
├─ Whitespace: Extra indentation, newlines (50+ tokens)
└─ Total: 2000 tokens
After optimization:
├─ Tool stripping: Keep only 5 relevant tools (400 tokens)
├─ Parameter abbreviation: {im: true, mr: 100} instead of {include_metadata: true, max_results: 100}
├─ Structured format: {task: "find", lang: "python"} instead of prose
└─ Total: 1200 tokens
Savings: 40% (800 tokens)
Techniques Applied:
- Tool stripping: 300 tools → 5 relevant (200-300 tokens saved)
- Parameter compression: Abbreviate keys, remove defaults (100-150 tokens saved)
- Input formatting: Prose → structured JSON (50-100 tokens saved)
- Whitespace removal: Strip unnecessary indentation (30-50 tokens saved)
- Combined savings: 40-60% on input tokens
Classify query intent to decide cache safety:
Query: "Detailed security analysis of this code"
↓
Classifier: Intent = DETAILED_ANALYSIS
↓
Decision: Skip cache (reasoning needed, not cached)
↓
Route: Fresh Claude response (full quality)
↓
Safety: No cached wrong answers
Query: "Quick summary of functions"
↓
Classifier: Intent = QUICK_ANSWER
↓
Decision: Cache safe (summary can be reused)
↓
Route: Use semantic cache if available (98% savings)
↓
Cost: Minimal tokens
Intent Types:
QUICK_ANSWER— Cacheable summaries, lookupsDETAILED_ANALYSIS— Reasoning-heavy, requires fresh responseROUTINE— Identical repeated queriesLEARNING— Exploratory scenariosFOLLOW_UP— Refinements on previous answer
Show user when optimization applied vs fresh response:
Cached response:
"✓ Cached response (98% savings, semantic match)"
Cost: $0.001 (vs $0.05 if fresh)
Fresh response:
"✓ Fresh response from Claude (no caching needed)"
Cost: $0.05 (detailed analysis)
Optimization applied:
"⚡ Optimized response (40% input savings)"
Cost: $0.03 (input compressed, Claude generated full analysis)
Metrics Tracked:
- Cache hit rate (target: 50%+)
- Token savings percentage
- Cost savings (estimated)
- Requests by optimization layer
- Latency by operation
Multi-layer input validation preventing injection attacks:
Security Validations (ALWAYS applied):
✅ SQL injection detection (DROP, ', UNION SELECT, etc)
✅ Command injection prevention (|, &, ;, $(, etc)
✅ XSS protection (<script>, javascript:, onerror=, etc)
✅ Rate limiting (1000 req/min per IP)
✅ Input schema validation
✅ Cannot be disabled (always on)
Test Coverage: 50+ attack patterns detected with 0 false positives
Dashboard-based tool management for CLI, MCP, REST, and database tools without YAML editing:
Dashboard → Tools Tab → Add Tool Form
↓
Type: [CLI / MCP / REST / Database / Binary]
Name: [my_tool]
Path: [/usr/local/bin/tool]
Settings: {json: "values"}
↓
Test Connection → Health Check
↓
Save → Config persisted to YAML
Features:
✅ Add, edit, delete tools via web UI
✅ Tool health status indicators
✅ Type-specific path validation (CLI vs MCP socket)
✅ Real-time tool list with current status
✅ Config auto-saves to disk (survives restart)
✅ No YAML editing required (non-technical users supported)
Use Cases:
- Add custom scripts without modifying config files
- Manage MCP server sockets through UI
- Test tool connectivity with one click
- Audit all configured tools in one place
graph TD
A["📥 Request Arrives<br/>Find all functions calling authenticate"] --> B
B["⏭️ Layer 1: Cache Bypass<br/>User said --no-cache?"] -->|YES| C1["⚡ Skip cache<br/>Go fresh"]
B -->|NO| D
D["💾 Layer 2: Exact Dedup<br/>SHA256 match in cache?"] -->|YES| C2["✅ Cache Hit<br/>0 tokens"]
D -->|NO| E
E["🔒 Layer 3: Security<br/>Injection detected?"] -->|YES| C3["🛑 Blocked<br/>Malicious input"]
E -->|NO| F
F["📊 Layer 4: Knowledge Graph<br/>Can graph answer this?"] -->|YES| C4["⚡ Graph Result<br/>0 tokens, <10ms"]
F -->|NO| G
G["🗜️ Layer 5: Input Optimization<br/>Strip tools<br/>Compress params"] --> H
H["🤖 Layer 6: Claude API<br/>Send optimized request"] --> I
I["📝 Layer 7: Response Cache<br/>Store for semantic matching"] --> J["📤 Return Response<br/>with transparency footer"]
C1 --> J
C2 --> J
C3 --> J
C4 --> J
J --> K["📈 Result: 60-75% token savings<br/>on mixed workloads"]
style A fill:#4F46E5,stroke:#312E81,color:#fff
style K fill:#10B981,stroke:#065F46,color:#fff
style C2 fill:#10B981,stroke:#065F46,color:#fff
style C4 fill:#10B981,stroke:#065F46,color:#fff
style C3 fill:#EF4444,stroke:#7F1D1D,color:#fff
graph LR
A["🔄 Claude Escalate<br/>Metrics Collected"] --> B["📈 Prometheus<br/>Pull Endpoint<br/>:8080/metrics"]
A --> C["📤 OTEL Push<br/>gRPC to Collector<br/>:4317"]
B --> B1["Prometheus<br/>Server"]
B1 --> B2["Grafana"]
C --> C1["OTEL Collector"]
C1 --> C2["Datadog<br/>Jaeger<br/>Prometheus<br/>New Relic"]
B2 --> D["📊 Dashboards<br/>Alerts<br/>Analytics"]
C2 --> D
style A fill:#4F46E5,stroke:#312E81,color:#fff
style B2 fill:#10B981,stroke:#065F46,color:#fff
style D fill:#10B981,stroke:#065F46,color:#fff
| Optimization | Per Hit | Typical Hit Rate | Overall Impact |
|---|---|---|---|
| Exact Deduplication | 100% | 20-30% | 20-30% savings |
| Semantic Cache | 98% | 30-50% | 15-30% savings |
| Knowledge Graph | 99% | 10-20% (for relationship queries) | 5-15% savings |
| Input Optimization | 40-60% | 100% (always applied) | 40-60% savings |
| Combined (Realistic Mix) | - | - | 60-75% savings |
Example: 1000 requests with mixed patterns
- Unoptimized: $15.00
- With Input Optimization (40-60%): $6.00-9.00
- + Exact Dedup (20% hit rate): $4.80-7.20
- + Semantic Cache (35% hit rate): $2.40-3.60
- + Knowledge Graph (10% hit rate): $1.20-1.80
- All Combined: $1.20-1.80 (88-92% savings) ✅
Conservative Estimate (60-75% proven):
- 1000 requests @ $15.00 baseline
- With v0.5.0 optimizations: $3.75-6.00
- Monthly savings: $108-$136 (1000 req/mo → $180-270 cost reduction annually)
docker build -t claude-escalate .
docker run -d -p 9000:8077 -v escalate-data:/data \
claude-escalate:latest dashboard --port 8077
# Access: http://localhost:9000/dashboardSee DOCKER_SETUP.md for full guide.
git clone https://github.com/szibis/LLMSentinel.git
cd claude-escalate
make build
./bin/claude-escalate version./bin/claude-escalate dashboard --port 8077
# Service listens on all interfaces (0.0.0.0:8077)Open http://localhost:8077/dashboard in your browser
- Real-time metrics
- Task classification results
- Budget status
- Analytics charts
curl -X POST http://localhost:8080/api/config/budgets \
-H "Content-Type: application/json" \
-d '{"daily_limit": 10.00, "weekly_limit": 50.00}'This legacy setup does not connect the standalone optimization modules to Claude requests. Use the current client guide for live serving.
- Installation & Setup — Installation options
- Quick Start Guide — 5-minute setup
- Docker Deployment — Containerized setup
- Architecture Overview — How it works
- ML Classification — Task detection
- Analytics Guide — Dashboard usage
- Budget Management — Cost control
- REST API Reference — All endpoints
- Batch Processing — Queue & flush
- Sentiment Detection — Task routing
- Deployment Guide — Production setup
- Monitoring & Alerts — Prometheus metrics
- Troubleshooting — Common issues
- Security Policy — OWASP Top 10
- Testing & Quality — 312 tests, full coverage
- Contributing — Development guide
| Category | Result | Details |
|---|---|---|
| Unit Tests | ✅ 530 passing | All optimization layers covered |
| Code Coverage | ✅ 85%+ | All critical paths tested |
| Integration Tests | ✅ 40+ tests | Optimization pipeline interactions |
| Knowledge Graph Tests | ✅ 12 tests | Indexing, AST parsing, graph traversal |
| Input Optimization Tests | ✅ 15 tests | Dedup, formatting, compression |
| Security Tests | ✅ 50+ patterns | SQL injection, XSS, command injection |
| Performance Tests | ✅ Latency benchmarks | <10ms graph queries, <200ms fresh |
| Race Detection | ✅ All tests with -race | Zero data races |
| Code Quality | ✅ Clean | golangci-lint passing |
graph TB
subgraph "📥 Input"
A["User Query<br/>Tool Request<br/>MCP/CLI/REST"]
end
subgraph "🔧 Optimization Pipeline"
B["Cache Bypass<br/>Check"]
C["Exact Dedup<br/>SHA256"]
D["Security<br/>Validation"]
E["Knowledge<br/>Graph Lookup"]
F["Input<br/>Optimization"]
G["Claude<br/>API Call"]
end
subgraph "💾 Storage"
H["Cache DB<br/>Embeddings"]
I["Graph DB<br/>Relationships"]
J["Config<br/>YAML"]
end
subgraph "📊 Monitoring"
K["Prometheus<br/>/metrics"]
L["OTEL Push<br/>to Collector"]
end
A --> B
B --> C
C --> D
D --> E
E --> F
F --> G
C -.-> H
E -.-> I
J -.-> G
G --> M["📤 Response<br/>to User"]
G -.-> K
G -.-> L
style A fill:#4F46E5,stroke:#312E81,color:#fff
style M fill:#10B981,stroke:#065F46,color:#fff
style K fill:#3B82F6,stroke:#1E40AF,color:#fff
style L fill:#3B82F6,stroke:#1E40AF,color:#fff
Core Modules:
internal/optimization/— 7-layer pipelineinternal/cache/— Semantic caching + exact dedupinternal/graph/— SQLite knowledge graph with CTE queriesinternal/indexing/— Code indexing (Go, Python, TypeScript)internal/intent/— Query classificationinternal/security/— Injection detection (50+ patterns)internal/gateway/— Tool adapters (MCP, CLI, REST)internal/metrics/— Prometheus + OTEL export
Storage:
~/.claude-escalate/graph.db— Knowledge graph~/.claude-escalate/cache.db— Semantic cache~/.claude-escalate/config.yaml— Configuration
Deployment:
- Single binary (12-15 MB)
- SQLite embedded (no external DB)
- Prometheus metrics endpoint
- OpenTelemetry push support
- Dependencies: fsnotify only
docker build -t llm-sentinel .
docker run --rm -p 8077:8077 llm-sentinel# After a release from the current workflow:
gh release download --repo szibis/LLMSentinel --pattern llm-sentinel-linux-amd64
chmod +x llm-sentinel-linux-amd64
./llm-sentinel-linux-amd64 dashboard --port 8077git clone https://github.com/szibis/LLMSentinel.git
cd LLMSentinel
make build # Builds Go binary
make dev # Starts dashboard on :8080docker-compose up # Service + dashboard
# Access: http://localhost:8080- Go 1.27.2 (for building from source)
- Node.js 18+ (for building web dashboard)
- Linux or macOS (Intel/ARM)
- 8 MB disk space (binary + cache)
- 20 MB RAM (service + metrics + cache)
OWASP Top 10 Compliance:
- ✅ Input validation on all APIs (SQL injection, XSS, command injection)
- ✅ Integer overflow protection in cost calculations
- ✅ Memory safety (bounds checking, leak detection)
- ✅ No remote access by default (localhost only)
- ✅ Encrypted configuration support
- ✅ Audit logs for all cost decisions
- ✅ No hardcoded credentials
- ✅ Data exposure prevention (no secrets in metrics/logs)
- ✅ Cryptographic security validation
- ✅ Concurrency safety (race-free)
Security Testing:
- 30+ security tests (SQL injection, path traversal, command injection)
- 7 fuzzing tests for input validation
- Memory leak detection (runtime analysis)
- Race condition detection (all tests with -race flag)
- Gosec security linting enabled
Contributions welcome! Areas for enhancement:
- Extended ML models for task classification
- Real-time alerts and notifications
- Team/multi-user support
- Advanced forecasting models
- IDE plugins (VS Code, JetBrains)
MIT License — See LICENSE file for details.
- Issues: GitHub Issues
- Discussions: GitHub Discussions
- Documentation: Full Docs
- Download & Install (2 min)
- Start Service (1 min)
- View Dashboard (instant)
- Set Budgets (1 min)
- View ML Classifications (see optimization)
- Monitor Analytics (track savings)
Status: Active development. The local gateway/lab is connected; older optimization modules remain partial. Current test, security and performance evidence must come from the relevant CI run or documented live probe.
See native prompt reuse and local role profiles for cache accounting, model compatibility and measured checks.