Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 153 additions & 0 deletions Docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# ORACLE Documentation

> **ORACLE** — Optimized Repository Analysis & Candidate Learning Engine
> An evidence-grounded Implementation Familiarity Assessment System

---

## Documentation Index

### 🚀 Getting Started

| Document | Description |
|---|---|
| [README](./getting-started/README.md) | What ORACLE does, quick navigation, system requirements |
| [Installation](./getting-started/installation.md) | Local setup, environment, running the server |

---

### 🏛️ Architecture

| Document | Description |
|---|---|
| [System Overview](./architecture/system-overview.md) | Top-level architecture, agent pipeline, design principles |
| [Module Inventory](./architecture/module-inventory.md) | All modules, LOC, ownership, active/review/archived status |
| [Execution Flow](./architecture/execution-flow.md) | Step-by-step pipeline: repo clone → AST → viva → assessment |
| [Data Flow](./architecture/data-flow.md) | Data structures, transformations, WebSocket payload schemas |
| [Agent Overview](./architecture/agent-overview.md) | Per-agent responsibilities, boundaries, and decision routing |

---

### 🔌 API Reference

| Document | Description |
|---|---|
| [WebSocket API](./api-docs/websocket-api.md) | Full WebSocket protocol, input/output schemas, streaming format |

---

### 🧠 Viva Intelligence

| Document | Description |
|---|---|
| [Viva Overview](./viva-intelligence/viva-overview.md) | How the viva system works, indicators, familiarity levels, examples |
| [Session Flow](./viva-intelligence/session-flow.md) | Turn-by-turn session walkthrough with state and examples |

---

### 🎨 Frontend

| Document | Description |
|---|---|
| [UI Overview](./frontend/ui-overview.md) | Dashboard panels, UI alignment status, known minor issues |

---

### 🚢 Deployment

| Document | Description |
|---|---|
| [Environment Variables](./deployment/environment-variables.md) | All env vars, required scopes, security rules |

---

### ⚙️ CI/CD

| Document | Description |
|---|---|
| [Workflows](./ci-cd/workflows.md) | GitHub Actions workflows: calibration, Discord notify |
| [Discord Notifications](./ci-cd/discord-notifications.md) | Commit-to-Discord bot setup guide |

---

### 🧪 Testing

| Document | Description |
|---|---|
| [Calibration Framework](./testing/calibration.md) | Automated P/R metrics, confidence calibration, CI thresholds |
| [Human Testing Protocol](./testing/human-testing-protocol.md) | 4-phase real-human validation protocol |

---

### 🔐 Security

| Document | Description |
|---|---|
| [Security Overview](./security/overview.md) | Fairness audit, trust audit, overconfidence detection, access control |

---

### ⚠️ Error Handling

| Document | Description |
|---|---|
| [Known Issues](./error-handling/known-issues.md) | Active known issues, error codes, recovery behaviour |
| [Bias Mitigation](./error-handling/bias-mitigation.md) | All 6 FairnessAuditor patterns, mitigation strategies |

---

### 📊 Monitoring & Logging

| Document | Description |
|---|---|
| [Observability](./monitoring-logging/observability.md) | Runtime tracing, WebSocket log streaming, calibration dashboard |

---

### 🤝 Contributing

| Document | Description |
|---|---|
| [Guidelines](./contributing/guidelines.md) | Code standards, PR process, naming conventions, review checklist |
| [MAIN Agent Issues](./contributing/main-agent-issues.md) | 6 pre-scoped contributor issues with file structures |

---

### 🗺️ User Flows

| Document | Description |
|---|---|
| [Analysis Workflow](./user-flows/analysis-workflow.md) | End-to-end user journey: open UI → analyse → review → assess |

---

## Project Phase Status

| Phase | Status |
|---|---|
| Phase 1 – Architecture Assessment & Cleanup | ✅ Complete |
| Phase 2 – Evidence-Grounded Intelligence (3 engines) | ✅ Complete |
| Phase 3.5 – Stabilization & Reality Hardening | 🔄 In Progress |
| Phase 4 – Real Human Testing (10–15 participants) | ⏳ Planned |

---

## Performance Baseline (Phase 2 Calibration)

| Component | Precision | Recall | F1 |
|---|---|---|---|
| Observable Signals | 0.847 | 0.823 | 0.835 |
| Failure Propagation | 0.805 | 0.778 | 0.791 |
| Viva Question Validity | 0.856 | — | — |
| Viva Question Grounding | 0.912 | — | — |
| Confidence Calibration RMSE | — | — | 0.062 ✅ |

---

## Quick Links

- **Run the system:** [Installation Guide](./getting-started/installation.md)
- **Understand the pipeline:** [Execution Flow](./architecture/execution-flow.md)
- **Read the API:** [WebSocket API](./api-docs/websocket-api.md)
- **Run calibration:** [Calibration Framework](./testing/calibration.md)
- **Contribute:** [Contributing Guidelines](./contributing/guidelines.md)
230 changes: 230 additions & 0 deletions Docs/api-docs/websocket-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
# WebSocket API Reference

ORACLE communicates with the browser UI via a persistent WebSocket connection.

**Endpoint:** `ws://localhost:8001/ws/analyze`
**Protocol:** JSON messages over WebSocket
**Server:** FastAPI + `python-multipart`, `uvicorn`

---

## Connection Flow

```
Browser FastAPI Server
│ │
│──── ws://localhost:8001/ws/analyze ──→│
│ │
│──── { "repo_url": "..." } ────────→│
│ │
│←─── { "type": "log", ... } ───────│ (streaming)
│←─── { "type": "log", ... } ───────│ (streaming)
│ ... many log messages ... │
│←─── { "type": "result", ... } ────│ (final payload)
│ │
│──── close ────────────────────────│
```

---

## Input Message

Sent once after WebSocket connection is established.

```json
{
"repo_url": "https://github.com/org/repo-name"
}
```

| Field | Type | Required | Description |
|---|---|---|---|
| `repo_url` | `string` | ✅ Yes | Full HTTPS GitHub URL of the repository to analyse |

---

## Streaming Messages (Backend → UI)

Sent continuously during analysis. Used to update the terminal log panel.

```json
{
"type": "log",
"message": "[Oracle] Building execution graph...",
"log_type": "info"
}
```

| Field | Type | Values | Description |
|---|---|---|---|
| `type` | `string` | `"log"` | Always `"log"` for streaming messages |
| `message` | `string` | Any | Human-readable progress message |
| `log_type` | `string` | `info` \| `warn` \| `success` \| `error` | Controls colour in terminal panel |

---

## Final Result Payload (Backend → UI)

Sent once when analysis is complete.

```json
{
"type": "result",
"data": { ... }
}
```

### `data` Object Schema

#### Framework Detection

```json
"backend_framework": {
"value": "FastAPI",
"confidence": 0.95,
"evidence": ["backend/src/main.py:12 — @app.get('/health')"]
}
```

| Field | Type | Description |
|---|---|---|
| `value` | `string` | Detected framework name |
| `confidence` | `float` | Detection confidence (0–1) |
| `evidence` | `List[string]` | Code locations that support detection |

Same schema applies to:
- `architecture_pattern` — e.g. "REST + WebSocket"
- `authentication_system` — e.g. "JWT Bearer"

---

#### Execution Graph

```json
"execution_graph": {
"nodes": [
{
"id": "node_001",
"label": "GET /users",
"type": "ROUTE",
"metadata": {
"file_path": "backend/src/main.py",
"line_number": 42,
"snippet": "@app.get('/users')"
}
}
],
"edges": [
{
"source": "node_001",
"target": "node_002",
"relationship": "calls"
}
]
}
```

**Node types:** `ROUTE` | `MIDDLEWARE` | `DB_QUERY` | `STATE_STORE` | `AUTH_HANDLER`

---

#### Viva Intelligence Targets

```json
"viva_intelligence_targets": [
{
"topic": "N+1 Query in /users endpoint",
"question_target": "What performance concern does the /users endpoint have?",
"difficulty": "hard",
"importance_score": 0.95,
"category": "Architecture",
"depth_score": 8.5,
"related_node": "node_001",
"confidence": 0.88,
"reasoning_summary": "Endpoint loads 100+ related resources without batching"
}
]
```

**Difficulty values:** `"hard"` | `"medium"` | `"foundational"`
**Category values:** `"Architecture"` | `"Tradeoff"` | `"Security"` | `"Scalability"` | `"Failure-Path"` | `"Runtime"`

---

#### Runtime Risks

```json
"runtime_risks": [
{
"severity": "HIGH",
"value": "Unhandled async exception in payment service",
"confidence": 0.87,
"evidence": ["backend/src/services/payment.py:88"]
}
]
```

---

#### Failure Paths

```json
"failure_paths": [
{
"value": "DB connection loss causes cascading timeout in /checkout",
"confidence": 0.82,
"evidence": ["backend/src/routes/checkout.py:55"]
}
]
```

---

#### Evaluation Metrics

```json
"evaluation_metrics": {
"metrics": {
"stack_accuracy": 0.95,
"auth_detection_accuracy": 0.91
},
"mismatches": [],
"expected": {
"expected_stack": "FastAPI + PostgreSQL",
"expected_protected_routes": ["/admin", "/dashboard"],
"expected_architecture": "Layered REST"
}
}
```

---

## Error Handling

If analysis fails, the backend sends:

```json
{
"type": "log",
"message": "[Oracle] ERROR: Failed to clone repository. Check GITHUB_TOKEN.",
"log_type": "error"
}
```

The WebSocket connection is then closed.

---

## Backend Source Reference

- Handler: `backend/src/main.py` — `websocket_analyze()` at line 42
- OracleAgent: `backend/src/agents/oracle/agent.py`
- StructuredContext model: `backend/src/models/`

---

## Related Docs

- [Payload Schemas](./payload-schemas.md) — Full Pydantic model reference
- [Data Flow](../architecture/data-flow.md) — How data is assembled
- [UI Overview](../frontend/ui-overview.md) — How the UI consumes this payload
Loading
Loading