-
Notifications
You must be signed in to change notification settings - Fork 0
feat: establish RockQL compiler foundation #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
2fade6f
docs: define RockQL vision and MVP
SayanthRock d5f5fb0
build: add Rust workspace
SayanthRock 3eb4c1e
build: pin Rust toolchain
SayanthRock ca593f3
chore: add repository ignores
SayanthRock 0810483
feat(ast): add AST crate
SayanthRock cc36963
feat(ast): define initial RockQL syntax tree
SayanthRock abaa266
feat(parser): add parser crate
SayanthRock d7d010c
feat(parser): parse initial RockQL transformations
SayanthRock 312a80d
feat(sql): add SQL generator crate
SayanthRock eef80b5
feat(sql): compile RockQL AST to SQL
SayanthRock 1337c1b
feat(cli): add RockQL command-line package
SayanthRock e7f5b2f
feat(cli): implement compile check ast and format commands
SayanthRock 6ee1195
test(sql): add parser test dependency
SayanthRock 9495fc6
docs: add RockQL example query
SayanthRock db1478f
ci: add Rust quality checks
SayanthRock 9cac939
ci: add dependency review and CodeQL
SayanthRock e83c4f5
docs: add contribution guide
SayanthRock 78e214c
ci: expose rustfmt diff for initial validation
SayanthRock 39f5e98
ci: let rustfmt update the draft branch
SayanthRock 15ac329
style: apply rustfmt
github-actions[bot] 0f7c238
ci: restore read-only compiler validation
SayanthRock c6c5ec5
ci: let Clippy update the draft branch
SayanthRock 6dd2110
ci: capture Clippy diagnostics on the draft branch
SayanthRock cb90f60
ci: capture Clippy diagnostics
github-actions[bot] cbbc592
build: update RockQL MSRV to Rust 1.97
SayanthRock ef4d2ab
build: use Rust 1.97.1 toolchain
SayanthRock fa76e2b
ci: validate with Rust 1.97.1
SayanthRock d6cb974
chore: remove temporary Clippy diagnostics
SayanthRock 9f110cc
docs: document Rust 1.97 requirement
SayanthRock File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ci-${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| rust: | ||
| name: Rust checks | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Install Rust | ||
| uses: dtolnay/rust-toolchain@1.97.1 | ||
| with: | ||
| components: rustfmt, clippy | ||
|
|
||
| - name: Check formatting | ||
| run: cargo fmt --all --check | ||
|
|
||
| - name: Run Clippy | ||
| run: cargo clippy --workspace --all-targets --all-features -- -D warnings | ||
|
|
||
| - name: Run tests | ||
| run: cargo test --workspace --all-features | ||
|
|
||
| - name: Build release CLI | ||
| run: cargo build --release --package rockql-cli | ||
|
|
||
| - name: Compile example | ||
| run: ./target/release/rockql compile examples/employees.rockql --target sqlite | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| name: Security | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
| schedule: | ||
| - cron: "23 3 * * 1" | ||
|
|
||
| concurrency: | ||
| group: security-${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| dependency-review: | ||
| name: Dependency review | ||
| if: github.event_name == 'pull_request' | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
SayanthRock marked this conversation as resolved.
|
||
| - name: Review dependency changes | ||
| uses: actions/dependency-review-action@v5 | ||
|
|
||
| codeql: | ||
| name: CodeQL (Rust) | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| security-events: write | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
SayanthRock marked this conversation as resolved.
|
||
|
|
||
| - name: Initialise CodeQL | ||
| uses: github/codeql-action/init@v4 | ||
| with: | ||
| languages: rust | ||
| build-mode: none | ||
|
|
||
| - name: Analyse | ||
| uses: github/codeql-action/analyze@v4 | ||
| with: | ||
| category: /language:rust | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| /target | ||
| **/node_modules | ||
| **/dist | ||
| .idea | ||
| .vscode | ||
| .DS_Store | ||
| *.db | ||
| *.sqlite | ||
| *.sqlite3 | ||
| .env | ||
| .env.* | ||
| !.env.example |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # Contributing to RockQL | ||
|
|
||
| RockQL is early-stage software. Small, focused pull requests with tests are preferred over large changes that mix language design, compiler internals, and UI work. | ||
|
|
||
| ## Development setup | ||
|
|
||
| Requirements: | ||
|
|
||
| - Rust 1.97.1 or a compatible newer toolchain. | ||
| - Git. | ||
|
|
||
| ```bash | ||
| git clone https://github.com/Sayanthrock-Developer/ROCKQL.git | ||
| cd ROCKQL | ||
| cargo test --workspace | ||
| ``` | ||
|
|
||
| ## Quality checks | ||
|
|
||
| Run the same checks used by CI before opening a pull request: | ||
|
|
||
| ```bash | ||
| cargo fmt --all --check | ||
| cargo clippy --workspace --all-targets --all-features -- -D warnings | ||
| cargo test --workspace --all-features | ||
| cargo build --release --package rockql-cli | ||
| ``` | ||
|
|
||
| ## Language changes | ||
|
|
||
| A language change should include: | ||
|
|
||
| 1. A clear syntax example. | ||
| 2. Parser and AST updates. | ||
| 3. SQL output examples for every affected dialect. | ||
| 4. Positive and negative tests. | ||
| 5. Documentation updates. | ||
| 6. A compatibility note when existing syntax changes. | ||
|
|
||
| Do not silently reinterpret valid syntax. Diagnostics should identify the exact line and column whenever possible. | ||
|
|
||
| ## Compiler boundaries | ||
|
|
||
| Keep the stages separate: | ||
|
|
||
| - Parsing creates syntax, not SQL. | ||
| - Resolution validates names and types. | ||
| - Relational IR represents database operations. | ||
| - Optimisation rewrites IR, not source text. | ||
| - Dialect generators own target-specific SQL. | ||
|
|
||
| The initial compiler intentionally excludes database administration, migrations, write operations, credential storage, collaboration, and paid AI services. | ||
|
|
||
| ## Pull requests | ||
|
|
||
| Use a descriptive title and explain: | ||
|
|
||
| - The problem being solved. | ||
| - The implementation approach. | ||
| - Tests performed. | ||
| - Known limitations. | ||
|
|
||
| By contributing, you agree that your contribution is licensed under Apache License 2.0. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| [workspace] | ||
| members = [ | ||
| "compiler/rockql-ast", | ||
| "compiler/rockql-parser", | ||
| "compiler/rockql-sql", | ||
| "compiler/rockql-cli", | ||
| ] | ||
| resolver = "2" | ||
|
|
||
| [workspace.package] | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
| license = "Apache-2.0" | ||
| repository = "https://github.com/Sayanthrock-Developer/ROCKQL" | ||
| rust-version = "1.97" | ||
|
|
||
| [workspace.dependencies] | ||
| anyhow = "1.0" | ||
| clap = { version = "4.5", features = ["derive"] } | ||
| serde = { version = "1.0", features = ["derive"] } | ||
| serde_json = "1.0" | ||
| thiserror = "2.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,158 @@ | ||
| # ROCKQL | ||
| # RockQL | ||
|
|
||
| > **Readable data pipelines for every database.** | ||
|
|
||
| RockQL is an open-source query language and compiler for writing database queries as readable top-to-bottom pipelines and compiling them into standard SQL. | ||
|
|
||
| ```rockql | ||
| from employees | ||
| filter salary > 50_000 | ||
| derive yearly_salary = salary * 12 | ||
| sort {-yearly_salary} | ||
| take 10 | ||
| ``` | ||
|
|
||
| ```sql | ||
| SELECT | ||
| *, | ||
| salary * 12 AS yearly_salary | ||
| FROM employees | ||
| WHERE salary > 50000 | ||
| ORDER BY yearly_salary DESC | ||
| LIMIT 10; | ||
| ``` | ||
|
|
||
| **Write the flow. Generate the SQL.** | ||
|
|
||
| ## Status | ||
|
|
||
| RockQL is at the compiler-foundation stage. The first implementation includes: | ||
|
|
||
| - A Rust workspace. | ||
| - A serialisable abstract syntax tree. | ||
| - A parser with line and column diagnostics. | ||
| - `from`, `filter`, `select`, `derive`, `sort`, and `take`. | ||
| - Generic SQL, SQLite, and PostgreSQL output. | ||
| - `compile`, `check`, `ast`, and `format` CLI commands. | ||
| - Unit tests and GitHub Actions CI. | ||
|
|
||
| The language and APIs are experimental until the v1.0 compatibility policy is published. | ||
|
|
||
| ## Install from source | ||
|
|
||
| ```bash | ||
| git clone https://github.com/Sayanthrock-Developer/ROCKQL.git | ||
| cd ROCKQL | ||
| cargo install --path compiler/rockql-cli | ||
| ``` | ||
|
|
||
| ## CLI | ||
|
|
||
| Compile a file: | ||
|
|
||
| ```bash | ||
| rockql compile query.rockql --target postgres | ||
| ``` | ||
|
|
||
| Compile stdin: | ||
|
|
||
| ```bash | ||
| echo "from users | filter active == true | take 10" \ | ||
| | rockql compile --target sqlite | ||
| ``` | ||
|
|
||
| Validate syntax: | ||
|
|
||
| ```bash | ||
| rockql check query.rockql | ||
| ``` | ||
|
|
||
| Print the AST: | ||
|
|
||
| ```bash | ||
| rockql ast query.rockql | ||
| ``` | ||
|
|
||
| Format source: | ||
|
|
||
| ```bash | ||
| rockql format query.rockql | ||
| rockql format query.rockql --write | ||
| ``` | ||
|
|
||
| ## Initial syntax | ||
|
|
||
| ```rockql | ||
| from orders | ||
| filter status == "completed" | ||
| select customer_id, amount | ||
| derive tax = amount * 0.18 | ||
| sort {-amount} | ||
| take 20 | ||
| ``` | ||
|
|
||
| Supported transformations: | ||
|
|
||
| - `from` | ||
| - `filter` | ||
| - `select` | ||
| - `derive` | ||
| - `sort` | ||
| - `take` | ||
|
|
||
| Both newline pipelines and `|`-separated pipelines are accepted. | ||
|
|
||
| ## Workspace | ||
|
|
||
| ```text | ||
| compiler/ | ||
| ├── rockql-ast/ Shared syntax model | ||
| ├── rockql-parser/ Source parser and diagnostics | ||
| ├── rockql-sql/ SQL dialect generator | ||
| └── rockql-cli/ Cross-platform command-line interface | ||
| ``` | ||
|
|
||
| Planned components include the resolver, relational IR, optimiser, formatter crate, WebAssembly bindings, browser playground, Android app, desktop app, language server, Tree-sitter grammar, and VS Code extension. | ||
|
|
||
| ## Compiler direction | ||
|
|
||
| ```text | ||
| RockQL source | ||
| ↓ | ||
| Lexer / parser | ||
| ↓ | ||
| Abstract Syntax Tree | ||
| ↓ | ||
| Name and type resolver | ||
| ↓ | ||
| Relational intermediate representation | ||
| ↓ | ||
| Query optimiser | ||
| ↓ | ||
| SQL dialect generator | ||
| ↓ | ||
| Formatted SQL | ||
| ``` | ||
|
|
||
| ## Roadmap | ||
|
|
||
| - **v0.1:** compiler foundation, SQLite generator, CLI, tests. | ||
| - **v0.2:** WebAssembly compiler, Monaco playground, diagnostics, query sharing. | ||
| - **v0.3:** joins, grouping, aggregation, variables, functions, PostgreSQL and MySQL. | ||
|
SayanthRock marked this conversation as resolved.
|
||
| - **v0.4:** formatter, language server, VS Code extension, Tree-sitter grammar. | ||
| - **v0.5:** visual pipeline editor, schema browser, DuckDB, CSV/JSON/Parquet. | ||
| - **v1.0:** stable syntax, compatibility policy, benchmarks, signed releases, Android app, JavaScript and Python bindings. | ||
|
|
||
| ## Project boundaries | ||
|
|
||
| Early RockQL versions focus on querying and transforming data. Database administration, migrations, write operations, cloud credential storage, team collaboration, paid AI services, and complex optimisation are intentionally out of scope. | ||
|
|
||
| ## Design direction | ||
|
|
||
| RockQL interfaces use charcoal and near-black surfaces, white primary text, muted supporting text, a restrained electric-blue or violet accent, rounded surfaces, clean monospaced code areas, subtle glass layers, and strong contrast without dashboard clutter. | ||
|
|
||
| ## Licence | ||
|
|
||
| Licensed under the [Apache License 2.0](LICENSE). | ||
|
|
||
| RockQL is an independent implementation. Third-party source code must retain its original licence, copyright notices, and required attribution. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| [package] | ||
| name = "rockql-ast" | ||
| version.workspace = true | ||
| edition.workspace = true | ||
| license.workspace = true | ||
| repository.workspace = true | ||
| rust-version.workspace = true | ||
|
|
||
| [dependencies] | ||
| serde.workspace = true |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.