Skip to content

Commit e89d849

Browse files
author
MPCoreDeveloper
committed
docs: add ROADMAP.md with full feature status and link from README
1 parent cd3b049 commit e89d849

2 files changed

Lines changed: 253 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
[![.NET 10](https://img.shields.io/badge/.NET-10-blue?style=for-the-badge)](https://dotnet.microsoft.com/download)
1313
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow?style=for-the-badge)](https://opensource.org/licenses/MIT)
1414
[![Platforms](https://img.shields.io/badge/Platforms-Windows%20%7C%20Linux%20%7C%20macOS-informational?style=for-the-badge)](#)
15+
[![Roadmap](https://img.shields.io/badge/Roadmap-View%20Plan-blueviolet?style=for-the-badge)](./ROADMAP.md)
1516
</div>
1617

1718
---
@@ -271,6 +272,7 @@ dotnet add package SharpCoreDB.Functional.EntityFrameworkCore --version 1.9.2
271272

272273
## Important documentation links
273274

275+
- **Roadmap:** [`ROADMAP.md`](./ROADMAP.md) — shipped features, near-term plans, and long-term vision
274276
- Documentation hub: `docs/INDEX.md`
275277
- Project docs index: `docs/README.md`
276278
- Feature matrix: `docs/FEATURE_MATRIX_v1.7.2.md`

‎ROADMAP.md‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
# SharpCoreDB Roadmap
2+
3+
<div align="center">
4+
5+
**Last updated: v1.9.2 · Maintained by [@MPCoreDeveloper](https://github.com/MPCoreDeveloper)**
6+
7+
</div>
8+
9+
> This roadmap reflects what is **actually in the codebase today** and what is planned next.
10+
> Features are derived from real source files, not marketing copy.
11+
> Community votes on [GitHub Issues](https://github.com/MPCoreDeveloper/SharpCoreDB/issues) directly influence priority order.
12+
13+
---
14+
15+
## Legend
16+
17+
| Symbol | Meaning |
18+
|--------|---------|
19+
| ✅ | Shipped and production-ready |
20+
| 🔶 | Partially implemented — foundation exists, full feature in progress |
21+
| 🗓️ | Planned near-term (target: v2.0 / v2.1) |
22+
| 🔭 | Long-term research / high-complexity item |
23+
| 💬 | Needs community input before scoping |
24+
25+
---
26+
27+
## ✅ Already Shipped (≤ v1.9.2)
28+
29+
### Core Engine
30+
- ✅ **AES-256-GCM single-file encrypted database**
31+
- ✅ **ACID transactions + WAL** (`RecoveryManager`, crash recovery tests passing)
32+
- ✅ **B-tree and hash indexing**
33+
- ✅ **Full-text search**
34+
- ✅ **SIMD acceleration** — `Vector256.LoadUnsafe` in columnar aggregate hot paths
35+
- ✅ **Memory pooling + JIT-oriented performance optimizations**
36+
- ✅ **100+ aggregate functions** (COUNT, SUM, AVG, STDDEV, PERCENTILE, CORRELATION, …)
37+
- ✅ **Window functions** (ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD)
38+
- ✅ **Query plan cache** (`QueryPlanCache`, `ExecutionPlan`, `QueryOptimizer`)
39+
40+
### Network Server
41+
- ✅ **gRPC over HTTPS** (HTTP/2 + HTTP/3) — primary protocol
42+
- ✅ **Binary TCP handler, REST API, WebSocket streaming** — secondary protocols
43+
- ✅ **Multi-database hosting + system databases** (`master`, `msdb`, `tempdb`)
44+
- ✅ **TLS 1.2+ enforced** — no plain HTTP endpoints
45+
- ✅ **JWT authentication + optional mTLS**
46+
- ✅ **RBAC** (Admin / Writer / Reader roles)
47+
- ✅ **Rate limiting** (fixed-window, per-IP, configurable)
48+
- ✅ **Connection pooling** (1,000+ concurrent connections)
49+
- ✅ **Health checks + Prometheus-compatible metrics endpoint**
50+
- ✅ **Graceful shutdown + production deployment** (Docker, Windows Service, Linux systemd, macOS launchd)
51+
52+
### Security
53+
- ✅ **Row-Level Security (RLS)** — `RowLevelPolicyEngine` with `Enforced`/`Audit` modes and per-tenant discriminator-column filtering (`src/SharpCoreDB.Server.Core/Security/`)
54+
55+
### Analytics & Search
56+
- ✅ **Vector search** — HNSW indexing with SIMD acceleration, 10M+ vector workloads validated
57+
- ✅ **Graph traversal** — BFS, DFS, bidirectional, A* pathfinding
58+
- ✅ **GraphRAG** — community detection (Louvain, LPA), centrality metrics (degree, betweenness, eigenvector), subgraph analysis
59+
60+
### Optional Packages
61+
- ✅ **`SharpCoreDB.EventSourcing`** — append-only per-stream storage, global ordered feed, in-memory + persistent stores, snapshot policy
62+
- ✅ **`SharpCoreDB.Projections`** — checkpoint persistence, OpenTelemetry-ready projection metrics
63+
- ✅ **`SharpCoreDB.CQRS`** — command/handler abstractions, aggregate root, outbox with dead-letter workflow
64+
- ✅ **`SharpCoreDB.EntityFrameworkCore`** — full Guid-keyed entity CRUD, relationship materialization, 22/22 integration tests passing
65+
- ✅ **`SharpCoreDB.Distributed`** — multi-master replication with vector clocks, distributed transactions (2PC)
66+
- ✅ **`SharpCoreDB.Provider.Sync`** — Dotmim.Sync provider for bidirectional cloud/edge data sync
67+
- ✅ **Time-series cold tiering** — `BucketTier.Hot/Cold`, archival manager, retention policies (`src/SharpCoreDB/TimeSeries/`)
68+
69+
### Tooling
70+
- ✅ **`SharpCoreDB.WebViewer`** — Razor Pages admin portal (table browser, query runner, live connection status)
71+
- ✅ **.NET client SDK** (`SharpCoreDB.Client`, ADO.NET-style)
72+
- ✅ **JavaScript/TypeScript SDK** (npm)
73+
- ✅ **Python client** (`PySharpDB`)
74+
75+
---
76+
77+
## 🔶 In Progress
78+
79+
### Visual Query Execution Plan Explorer
80+
- **Status:** Plan cache and optimizer internals exist; WebViewer UI not yet built
81+
- **What's needed:** Tree/graph view in WebViewer showing join types, cost estimates, and row counts per node — similar to pgAdmin's EXPLAIN visualizer
82+
- **Tracking:** [#issue](https://github.com/MPCoreDeveloper/SharpCoreDB/issues)
83+
84+
### Column-Level Security (CLS)
85+
- **Status:** RLS is fully shipped. CLS (per-column GRANT masks and data redaction) is the next security layer
86+
- **What's needed:** Column redaction policies, `MASKED WITH` syntax, integration with the RBAC engine
87+
88+
---
89+
90+
## 🗓️ Near-Term Roadmap (v2.0 / v2.1)
91+
92+
### Let's Encrypt / ACME Auto-Renewal
93+
> **Why:** Today the server requires manual PFX/PEM cert paths. Self-hosted deployments must manage certificate rotation by hand.
94+
95+
- Integrate ACME protocol (via [Certes](https://github.com/fszlin/certes) or `LettuceEncrypt`) into Kestrel startup
96+
- Zero-touch certificate provisioning and auto-renewal for self-hosted server
97+
- `appsettings.json` switch: `"AcmeEnabled": true` alongside existing `TlsCertificatePath`
98+
- Target: Linux/Docker-first, then Windows Service
99+
100+
```json
101+
"Security": {
102+
"AcmeEnabled": true,
103+
"AcmeDomain": "mydb.example.com",
104+
"AcmeEmail": "admin@example.com"
105+
}
106+
```
107+
108+
---
109+
110+
### Enterprise Backup Orchestrator
111+
> **Why:** The WAL and `RecoveryManager` are the foundation, but there is no scheduled/streaming backup engine yet.
112+
113+
- Scheduled full, incremental, and differential backups
114+
- Remote target support: **Azure Blob Storage**, **AWS S3**, **SFTP**
115+
- Backup retention policies (keep last N, time-window based)
116+
- Backup catalog with integrity verification (checksum + test-restore)
117+
- `SharpCoreDB.Backup` optional NuGet package
118+
- REST + gRPC management endpoints (`/backup/start`, `/backup/list`, `/backup/restore`)
119+
120+
---
121+
122+
### Visual Query Execution Plan Explorer (WebViewer)
123+
> **Why:** `QueryPlanCache` and `ExecutionPlan` already expose all node data. Only the UI is missing.
124+
125+
- Interactive tree/graph visualizer in WebViewer
126+
- Show operator type, estimated/actual row count, cost, index used
127+
- Highlight bottleneck nodes (slowest % of total cost)
128+
- Export plan as JSON or SVG
129+
130+
---
131+
132+
### Column-Level Security (CLS / Data Masking)
133+
> **Why:** Completes the security story started with RLS in v1.9.2.
134+
135+
- `MASKED WITH (FUNCTION = ...)` DDL syntax
136+
- Built-in masking functions: `default()`, `email()`, `partial()`, `random()`
137+
- Policy enforcement inside the SQL execution engine (not at API proxy layer)
138+
- `GRANT UNMASK` privilege to bypass masking for privileged roles
139+
140+
---
141+
142+
## 🔭 Long-Term Roadmap (v2.x+)
143+
144+
### Point-in-Time Recovery (PITR)
145+
> **Why:** Application bugs or human errors in high-throughput clusters carry a high risk of data loss without microsecond-level rollback.
146+
147+
- **Prerequisite:** Enterprise Backup Orchestrator (above) must ship first
148+
- Continuous WAL/transaction log shipping to remote storage
149+
- LSN-stamped log stream with sub-second granularity
150+
- `RESTORE DATABASE mydb TO TIMESTAMP '2025-06-01 14:32:00.000'`
151+
- Distributed-node replay coordination (builds on existing `TransactionLog` in `SharpCoreDB.Distributed`)
152+
- Target: no data loss window under 1 second in single-node, <5 seconds in distributed cluster
153+
154+
---
155+
156+
### Automated Data Tiering (Hierarchical Storage)
157+
> **Why:** At huge scale, keeping all data on local NVMe is cost-prohibitive. Cold historical blocks should transparently migrate to object storage.
158+
159+
- **Status:** Time-series `ArchivalManager` already implements `BucketTier.Hot/Cold` for time-series data. Needs to extend to general page-based storage.
160+
- Transparent hot (NVMe) → cold (Azure Blob / AWS S3) block migration
161+
- Query router continues to serve cold data without schema or SQL changes
162+
- Policy-driven tiering rules (age threshold, access frequency, size)
163+
- Read-back warming cache for frequently accessed cold blocks
164+
- `SharpCoreDB.Tiering` optional NuGet package
165+
166+
---
167+
168+
### Deep Query Execution Plan Visualizer (Advanced)
169+
> Beyond the near-term WebViewer plan explorer — advanced profiling tooling.
170+
171+
- Runtime query profiling with actual vs estimated row counts
172+
- Per-operator memory and CPU time breakdown
173+
- Historical plan regression detection (alert when plan changes cause slowdowns)
174+
- Exportable plan traces compatible with external tools
175+
176+
---
177+
178+
### Telemetry & Observability Expansion
179+
> **Status:** Prometheus metrics and OpenTelemetry projection metrics already ship. This expands coverage.
180+
181+
- Full distributed tracing (OTel spans across gRPC + embedded operations)
182+
- Query-level trace context propagation
183+
- Built-in Grafana dashboard template (`SharpCoreDB.Grafana.json`)
184+
- Structured log enrichment (query ID, database, user, latency histogram)
185+
186+
---
187+
188+
### Management Dashboard Expansion (WebViewer Pro)
189+
> **Status:** WebViewer ships today. These are planned additions.
190+
191+
- 📊 Live connection monitor (active queries, blocked sessions, lock waits)
192+
- 📈 Performance dashboard (QPS, latency P50/P95/P99, cache hit rate)
193+
- 🗄️ Backup / restore management UI
194+
- 🔒 Security audit log viewer (RLS policy hits, failed auth attempts)
195+
- 📋 Query history + plan comparison
196+
- 🧩 Schema designer with type picker (including ULID and GUID)
197+
198+
---
199+
200+
## 💬 Community Input Needed
201+
202+
These are ideas raised by the community that need more design work or votes before committing to a release target:
203+
204+
| Feature | Discussion |
205+
|---------|-----------|
206+
| **Zero-Knowledge Sync** | E2E encrypted sync where the server never sees plaintext |
207+
| **Offline Queue** | Queue writes while disconnected, replay on reconnect |
208+
| **Vector Sync** | Sync embeddings between edge and server for local-first AI |
209+
| **Graph Sync** | Sync graph edges/nodes in the Provider.Sync pipeline |
210+
| **WebSocket Push** | Real-time data push instead of poll-based sync |
211+
| **Selective Column Sync** | Sync only specific columns per table |
212+
213+
> 👉 **Vote or add ideas at:** [GitHub Issues](https://github.com/MPCoreDeveloper/SharpCoreDB/issues)
214+
215+
---
216+
217+
## How Priority Is Determined
218+
219+
1. **Community votes** — Issues with the most 👍 reactions get prioritized
220+
2. **Security and reliability** — Security features and data safety items move up automatically
221+
3. **Ecosystem completeness** — Features that unlock new use cases (PITR, CLS, ACME) over pure performance work
222+
4. **Effort/impact ratio** — Let's Encrypt (medium effort, huge DevEx win) ships before PITR (very high effort)
223+
224+
---
225+
226+
## Version Targets (Tentative)
227+
228+
| Version | Focus |
229+
|---------|-------|
230+
| **v2.0** | Let's Encrypt/ACME, Visual EXPLAIN in WebViewer, Column-Level Security |
231+
| **v2.1** | Enterprise Backup Orchestrator, Backup retention + remote targets |
232+
| **v2.2** | PITR (requires v2.1 backup foundation) |
233+
| **v2.x** | Automated Data Tiering, full OTel distributed tracing, advanced plan profiling |
234+
235+
> Targets are indicative. Issues and PRs from the community can accelerate any item.
236+
237+
---
238+
239+
## Contributing
240+
241+
Have a feature idea? Found a bug? Want to work on a roadmap item?
242+
243+
- 🐛 [Open a bug report](https://github.com/MPCoreDeveloper/SharpCoreDB/issues/new)
244+
- 💡 [Propose a feature](https://github.com/MPCoreDeveloper/SharpCoreDB/issues/new)
245+
- 🔀 [Submit a PR](https://github.com/MPCoreDeveloper/SharpCoreDB/pulls)
246+
247+
All contributions follow the standards in `.github/CODING_STANDARDS_CSHARP14.md`.
248+
249+
---
250+
251+
**Made with ❤️ for the .NET community**

0 commit comments

Comments
 (0)