diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..5b4261c --- /dev/null +++ b/LICENSE @@ -0,0 +1,204 @@ +Copyright 2026 Priyanshu Jha + + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md index 24587ad..c0bca46 100644 --- a/README.md +++ b/README.md @@ -1,35 +1,329 @@ # LeanCI -Dependency-aware AI pull request review with measured Paritok compression. +**Dependency-aware AI pull request review with measured [Paritok](https://github.com/Paritok) compression.** -## First end-to-end run (M3.6) +LeanCI is a GitHub Action that reviews PRs beyond the diff: it expands Python imports, call sites, and related tests, runs an OpenAI-compatible agent through a local Paritok proxy, then posts findings plus an honest **cost receipt** (tokens / estimated USD / reduction %). -### Required repository secrets +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) +[![Paritok](https://img.shields.io/badge/compression-Paritok-0B5FFF)](https://github.com/Paritok) -Configure these under **Settings → Secrets and variables → Actions**: +--- + +## Project overview + +LeanCI makes deep, cross-file PR review affordable by putting **Paritok token compression** on the agent tool path. Judges and adopters can verify savings with a dual-run receipt, not marketing claims. + +| | | +| --- | --- | +| **Type** | Composite GitHub Action + Python package (`leanci`) | +| **Language (MVP)** | Python 3.11+ | +| **License** | Apache-2.0 | +| **Demo** | `demo/` planted cross-file bug + `mode=dual_run` workflow | + +--- + +## Problem statement + +AI PR reviewers are useful but expensive: tool-heavy agent loops send large contexts (files, search hits, history) to the LLM on every turn. Teams either: + +1. **Review only the diff** and miss cross-file contract breaks, or +2. **Expand context aggressively** and burn tokens until the review no longer pencils out. + +Cross-file bugs (e.g. a helper’s return type changes while callers still assume the old shape) are exactly what shallow diff review misses—and what deep review costs the most to find. + +--- + +## Solution + +LeanCI combines three ideas: + +1. **Bounded dependency expansion** — seed from the PR diff, then add imports, reverse call sites, and related tests under hard caps. +2. **Paritok on the compressed path** — the agent talks to a local Paritok OpenAI-compatible proxy; Paritok compresses eligible tool/content traffic to the upstream model. +3. **Measured economics** — optional `dual_run` executes an uncompressed baseline on the same expansion set and publishes a side-by-side **cost receipt** with honesty labels when compression is weak. + +The demo proves the loop: a planted `validate_charge` tuple/bool contract break is caught citing both `payments/charge.py` and `api/checkout.py`, with metrics + assert gate. + +--- + +## Architecture + +```text +GitHub Action (action/) + └─ ParitokGateway (local proxy) ──► Paritok GPU + upstream LLM + └─ python -m leanci + ├─ DiffCollector + ├─ DependencyExpander → ContextManifest + ├─ AgentRuntime ↔ ToolHost (read/search) + │ └─ LLMClient(compressed via Paritok) + ├─ DualRunController (optional) + │ └─ LLMClient(uncompressed provider URL) + ├─ FindingNormalizer + ├─ ReceiptBuilder + PricingTable + ├─ GitHubPublisher (idempotent PR comment) + └─ MetricsExporter (leanci-metrics.json artifact) +``` + +**Data that leaves the runner:** PR source needed for the review (to Paritok GPU + upstream LLM), GitHub API comment traffic, and Actions logs/artifacts. There is no LeanCI-hosted database. + +--- + +## Workflow + +### Consumer (default): Paritok-only + +1. PR opened / synchronized → `.github/workflows/leanci.yml` +2. Checkout with `fetch-depth: 0` +3. Start Paritok proxy (`OPENAI_BASE_URL=http://127.0.0.1:8080/v1`) +4. Diff → expand → agent (compressed) → normalize → receipt → PR comment +5. Upload `leanci-metrics.json` + +### Demo: dual-run + planted-bug assert + +1. Changes under `demo/` (and related paths) → `.github/workflows/demo-leanci.yml` +2. Same pipeline with `mode: dual_run` and `LEANCI_PATH_PREFIX=demo/` +3. Assert job runs `scripts/assert_planted_bug.py` on the metrics artifact + +--- + +## Screenshots / GIF + +> Placeholders for the Devpost / README media. Drop real assets under `docs/media/` and replace links. + +| Asset | Path | Caption | +| --- | --- | --- | +| Architecture diagram | `docs/media/architecture.png` *(add)* | LeanCI + Paritok data flow | +| PR comment screenshot | `docs/media/pr-comment.png` *(add)* | Findings + cost receipt | +| Dual-run receipt | `docs/media/receipt.gif` *(add)* | Side-by-side token/cost columns | +| Actions run | `docs/media/actions-run.png` *(add)* | Green demo dual_run + assert | + +```markdown + +``` + +Live reference (example dual-run demo): see PR discussion and Actions on this repository’s demo PR / [LeanCI Demo workflow](.github/workflows/demo-leanci.yml). + +--- + +## Installation + +### Local package (dev / tests) + +```bash +git clone https://github.com/CodewithJha/leanci.git +cd leanci +python -m pip install -e ".[dev]" +python -m pytest +``` + +Optional: `uv sync` / `uv run pytest` if you use uv. + +### As a GitHub Action (this repo) + +The composite action lives at [`action/`](action/) and is invoked with `uses: ./action` from workflows in this repository. + +### As a reusable Action (consumer repos) + +After tagging a release (e.g. `v0.1.0-mvp`): + +```yaml +uses: CodewithJha/leanci/action@v0.1.0-mvp +``` + +--- + +## GitHub Action setup + +### 1. Add a workflow + +Minimal consumer template (also see [`.github/workflows/leanci.yml`](.github/workflows/leanci.yml)): + +```yaml +name: LeanCI + +on: + pull_request: + types: [opened, synchronize, reopened] + workflow_dispatch: {} + +permissions: + contents: read + pull-requests: write + +jobs: + leanci: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: CodewithJha/leanci/action@v0.1.0-mvp # or ./action in this repo + with: + # mode: paritok # default + # mode: dual_run # demo / measurement only + openai_url: https://api.groq.com/openai # example upstream + model: llama-3.1-8b-instant + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PARITOK_API_KEY: ${{ secrets.PARITOK_API_KEY }} + OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} +``` + +### 2. Action inputs (selected) + +| Input | Default | Notes | +| --- | --- | --- | +| `mode` | `paritok` | `paritok` \| `dual_run` | +| `model` | `gpt-4.1-mini` | Upstream model id | +| `openai_url` | _(empty)_ | Passed to Paritok as `--openai-url` | +| `max_files` / `max_bytes` | `20` / `400000` | Expansion caps | +| `max_tool_turns` / `max_findings` | `24` / `8` | Agent / publish caps | +| `severity_floor` | `medium` | Drop findings below floor | +| `fail_on_error` | `true` | Fail job on hard review errors | + +### 3. Confirm success + +- Workflow green +- PR comment containing `` +- Artifact `leanci-metrics` → `leanci-metrics.json` + +Target setup time: **≤ 10 minutes** once secrets exist. + +--- + +## Required secrets + +Configure under **Settings → Secrets and variables → Actions**: | Secret | Required | Purpose | | --- | --- | --- | -| `PARITOK_API_KEY` | Yes | Auth for the Paritok GPU proxy (`use_gpu_server: true`) | -| `OPENAI_API_KEY` | Yes | Upstream model provider key (forwarded through Paritok) | -| `GITHUB_TOKEN` | Automatic | Provided by Actions; workflow passes it for PR comments | +| `PARITOK_API_KEY` | Yes | Paritok GPU proxy (`use_gpu_server: true`) | +| `OPENAI_API_KEY` | Yes | Upstream provider key (OpenAI-compatible) | +| `GITHUB_TOKEN` | Automatic | Actions-provided; used for PR comments | + +Do **not** commit keys. Runtime config is written under `$RUNNER_TEMP` (see [`paritok.yaml.example`](paritok.yaml.example)); `PARITOK_API_KEY` is injected via the process environment. + +--- + +## Supported providers + +LeanCI’s LLM client is **OpenAI-compatible** (`/v1/chat/completions` + tools). Anything Paritok can forward works in principle. + +| Provider | Notes | +| --- | --- | +| **Groq** | Used in this repo’s CI demos (`openai_url: https://api.groq.com/openai`). Cloudflare UA quirks handled via a small local forwarder when needed. | +| **OpenAI** | Default uncompressed dual-run base when `LEANCI_OPENAI_URL` is unset. | +| **OpenRouter / others** | Set `openai_url` + matching `OPENAI_API_KEY`; watch rate limits and tool-calling quality. | + +Dual-run uncompressed traffic uses `LEANCI_OPENAI_URL` (provider base, **not** the Paritok proxy). + +--- + +## Demo instructions + +### Planted bug + +See [`demo/README.md`](demo/README.md): + +- `demo/src/payments/charge.py` — `validate_charge` returns `(ok, reason)` +- `demo/src/api/checkout.py` — still treats the return as a boolean (tuple is always truthy) +- `demo/tests/test_checkout_happy.py` — happy path only; CI stays green + +### Run the demo gate + +1. Ensure `PARITOK_API_KEY` and `OPENAI_API_KEY` are set. +2. Open a PR that touches `demo/` (or re-run **LeanCI Demo** via `workflow_dispatch`). +3. Wait for `leanci-dual` then `assert-planted-bug`. +4. Inspect the PR comment receipt and download the metrics artifact. +5. Locally (optional): + +```bash +python scripts/assert_planted_bug.py --metrics path/to/leanci-metrics.json +``` + +### Demo video script + +Recording beats: [`scripts/demo_video_script.md`](scripts/demo_video_script.md) (≤ 3 minutes). + +--- + +## Repository structure + +```text +leanci/ +├── LICENSE +├── README.md +├── pyproject.toml +├── paritok.yaml.example +├── action/ # Composite GitHub Action +├── src/leanci/ # Package: orchestrator, expand, agent, dual, receipt… +├── demo/ # Planted-bug toy service + dual_run template +├── scripts/ +│ ├── assert_planted_bug.py +│ └── demo_video_script.md +├── docs/ +│ ├── PRD.md +│ ├── devpost_submission.md +│ ├── RELEASE_CHECKLIST.md +│ ├── media/ # Screenshot/GIF placeholders +│ └── superpowers/specs/ # TDD +├── tests/ +└── .github/workflows/ + ├── leanci.yml # Consumer template (paritok) + ├── demo-leanci.yml # dual_run + assert + └── demo-assert.yml +``` + +--- + +## Limitations + +- **Python-first expansion** — AST imports + ripgrep heuristics; not full program analysis. +- **Heuristic, capped context** — may miss distant callers when caps hit. +- **Line numbers** may be approximate; findings prefer `file` + `symbol` when unsure. +- **Dual-run doubles LLM spend** — opt-in; intended for demos/measurement. +- **Provider tool-calling quality varies** — some models emit malformed tool XML (retried / nudged). +- **Compression % is workload-dependent** — tiny demos may show ~0% reduction; receipts label near-ineffective compression honestly (do not market a fixed %). +- **Private repos** — code leaves the runner to Paritok + the LLM provider; hackathon demo is public. + +### Threat model (brief) + +| Trust boundary | What crosses it | +| --- | --- | +| GitHub runner → Paritok GPU | Review context / tool payloads (compressed path) | +| Runner / forwarder → upstream LLM | Chat + tools (compressed via Paritok; uncompressed on dual_run) | +| Runner → GitHub API | PR comment bodies (no secrets) | + +Secrets stay in Actions secrets / env; never in committed YAML. + +--- + +## Future work + +Aligned with PRD stretch goals: + +- TypeScript expansion (S1) +- Inline review comments (S2) +- Hard token/budget ceilings (S3) +- `.leanci.yml` policy packs (S4) +- Org cost rollups (S5) +- Cross-PR compression cache (S6) +- Check-run gating on severity (S8) -Do not put secrets in `paritok.yaml` or commit them. The Action writes a key-free `paritok.yaml` under `$RUNNER_TEMP` and injects `PARITOK_API_KEY` via the process environment. See `paritok.yaml.example`. +--- -### Workflow +## Acknowledgements -- File: `.github/workflows/leanci.yml` -- Triggers: `pull_request` (`opened`, `synchronize`, `reopened`) and `workflow_dispatch` -- Permissions: `contents: read`, `pull-requests: write` -- Runs composite action `./action` (starts Paritok proxy, then `python -m leanci`) +- **[Paritok](https://github.com/Paritok)** — token compression middleware that makes tool-heavy review economically viable; LeanCI’s compressed path is designed around Paritok’s OpenAI-compatible proxy and `/stats` attribution. +- GitHub Actions / checkout / artifact ecosystem. +- Upstream model providers used in demos (e.g. Groq OpenAI-compatible API). +- Spec and product direction in [`docs/PRD.md`](docs/PRD.md) and the MVP TDD under `docs/superpowers/specs/`. -### Minimal PR checklist +--- -1. Commit and push LeanCI pipeline code to the default branch (or the branch the workflow runs from). -2. Set `PARITOK_API_KEY` and `OPENAI_API_KEY` repository secrets. -3. Open a pull request (or push to an existing PR) that changes Python files. -4. Confirm the **LeanCI** workflow run succeeds. -5. Confirm the PR has a LeanCI review comment (``) with findings and a cost receipt. -6. Confirm the `leanci-metrics` artifact (`leanci-metrics.json`) was uploaded. +## License -Default mode is `paritok` (single compressed run). Dual-run is opt-in via Action input `mode: dual_run`. +Copyright 2026 Priyanshu Jha. Licensed under the [Apache License 2.0](LICENSE). diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md new file mode 100644 index 0000000..fb1690f --- /dev/null +++ b/docs/RELEASE_CHECKLIST.md @@ -0,0 +1,42 @@ +# Release / submission checklist (M5) + +Use before Devpost final submit and before tagging `v0.1.0-mvp`. + +## Legal & repo hygiene + +- [ ] `LICENSE` is Apache-2.0 with correct copyright year/name +- [ ] README documents secrets, threat model, Paritok credit +- [ ] No secrets, API keys, or `.env` files committed +- [ ] `paritok.yaml.example` remains key-free +- [ ] Public GitHub repository + +## Product / demo proof + +- [ ] Consumer workflow `.github/workflows/leanci.yml` runs on a sample PR +- [ ] Demo workflow dual_run + assert green on `demo/` PR +- [ ] PR comment contains `` + cost receipt +- [ ] `leanci-metrics` / `leanci-demo-metrics` artifact present +- [ ] `scripts/assert_planted_bug.py` PASS on demo metrics +- [ ] Dual-run soft-fail behavior understood if uncompressed leg fails + +## Documentation assets + +- [ ] README complete (overview → acknowledgements) +- [ ] `scripts/demo_video_script.md` reviewed (≤3 min) +- [ ] `docs/devpost_submission.md` copied into Devpost; placeholders filled +- [ ] Screenshots/GIFs added under `docs/media/` (replace placeholders) +- [ ] Demo video recorded and uploaded (≤3 min) +- [ ] Paritok email entered on Devpost +- [ ] “How Paritok was used” paragraph pasted from Devpost draft + +## Optional polish + +- [ ] Pin Action / Paritok versions for reproducibility +- [ ] Tag `v0.1.0-mvp` after M4+M5 merge to `main` +- [ ] Final timed demo run ≤8 minutes wall clock +- [ ] Paritok GitHub issue with repro (stretch S7) + +## Do not do at freeze + +- [ ] No engine feature work after freeze without a release note +- [ ] Do not claim fixed token-savings % unless the receipt on screen shows it diff --git a/docs/devpost_submission.md b/docs/devpost_submission.md new file mode 100644 index 0000000..8b67648 --- /dev/null +++ b/docs/devpost_submission.md @@ -0,0 +1,132 @@ +# Devpost submission draft — LeanCI + +Fill these fields on Devpost. Replace bracketed placeholders before submit. + +--- + +## Title + +**LeanCI: Dependency-aware PR review with measured Paritok compression** + +--- + +## One-line pitch (tagline) + +Deep, cross-file AI PR review that stays affordable—because Paritok compression is in the agent loop, and every run ships an honest cost receipt. + +--- + +## Elevator pitch (short) + +LeanCI is a GitHub Action that expands beyond the PR diff (imports, call sites, tests), runs a tool-using review agent through a local Paritok proxy, and posts findings plus a dual-run cost receipt. A planted cross-file bug demo proves both quality and measurable economics. + +--- + +## Problem + +CI-friendly AI review usually means **diff-only** context. Real regressions often live in **callers and contracts** outside the changed lines. Expanding context and tool traffic makes reviews better—and burns tokens until teams turn the feature off. Existing “AI reviewer” tools rarely prove **token economics** with a reproducible baseline. + +--- + +## Solution + +LeanCI packages: + +1. **Bounded dependency expansion** for Python PRs +2. **Paritok-backed compressed agent path** (OpenAI-compatible local proxy) +3. **Optional dual-run** uncompressed baseline on the same expansion set +4. **PR comment + metrics artifact** with findings, receipt, and planted-bug parity +5. **Demo harness + assert script** so judges can verify the planted contract break + +--- + +## Features + +- Composite GitHub Action (`action/`) — setup Python, Paritok, ripgrep, run review +- Diff → expand → agent tools (`read_file`, `search_repo`, …) → JSON findings +- Cost receipt: original vs compressed input tokens, estimated USD, reduction %, honesty labels +- Dual-run mode for demos (`mode: dual_run`) +- Idempotent PR comment marker `` +- Metrics JSON artifact (`leanci-metrics.json`) +- `demo/` planted tuple/bool charge validation bug + weak test +- `scripts/assert_planted_bug.py` PASS/FAIL gate + +--- + +## Technical architecture + +**Stack:** Python 3.11+, GitHub Actions composite action, Paritok proxy (`paritok[proxy]`), OpenAI-compatible upstream (e.g. Groq), stdlib + ripgrep expansion. + +**Pipeline:** Action starts Paritok → `python -m leanci` loads config from env → collect diff → expand/rank/cap → build manifest → agent loop via `LLMClient` (compressed `OPENAI_BASE_URL`) → optional dual uncompressed client via provider `LEANCI_OPENAI_URL` → normalize findings → Paritok `/stats` → receipt → GitHub comment → metrics file → artifact upload. + +**How Paritok is used:** LeanCI does not reimplement compression. The compressed review binding points at the local Paritok OpenAI-compatible endpoint; tool-heavy chat traffic is eligible for Paritok compression; receipts read Paritok stats for measured original vs compressed input tokens. + +--- + +## Challenges we ran into + +- Provider rate limits (TPM/TPD) and Cloudflare/User-Agent quirks on some OpenAI-compatible hosts +- Models occasionally emitting malformed tool-call markup (`tool_use_failed`) mid-loop +- Tiny demo workloads can show **near-zero** compression—forcing honest labeling instead of vanity % +- Keeping dual-run opt-in so consumer CI does not silently 2× spend + +--- + +## Accomplishments that we're proud of + +- End-to-end Action on real PRs: comment + receipt + metrics artifact +- Planted cross-file bug caught with dual-run parity on compressed and uncompressed paths +- Assert script green in CI +- Honesty-first receipts when compression is ineffective +- Clear separation: consumer `paritok` workflow vs demo `dual_run` gate + +--- + +## What we learned + +- Economics and quality must be demonstrated together; a receipt without a finding (or vice versa) is a weak hackathon story +- Expansion caps and path scoping matter as much as the LLM when free-tier limits are tight +- Judges need reproducibility: public repo, Apache-2.0, secrets documented, assert script + +--- + +## What's next (future improvements) + +- Multi-language expansion (TypeScript) +- Inline GitHub review comments +- Hard budget ceilings with partial results +- Policy file (`.leanci.yml`) +- Org-level cost rollups +- Stronger compression demos on tool-heavy fixtures + +--- + +## Built with + +- Python +- GitHub Actions +- Paritok +- OpenAI-compatible APIs (e.g. Groq) +- ripgrep + +--- + +## Submission metadata (fill in) + +| Field | Value | +| --- | --- | +| **Project URL / repo** | https://github.com/CodewithJha/leanci | +| **Demo video** | _(upload ≤3 min; script in `scripts/demo_video_script.md`)_ | +| **Paritok account email** | `[YOUR_PARITOK_EMAIL]` | +| **How Paritok was used** | Local OpenAI-compatible proxy in the Action; compressed agent path; `/stats` for receipt token columns; dual-run compares against uncompressed provider traffic | +| **License** | Apache-2.0 | +| **Team** | Priyanshu Jha | + +--- + +## Suggested Devpost “Try it out” steps + +1. Open the public repo README secrets + workflow sections. +2. Fork or use workflow_dispatch on **LeanCI Demo**. +3. Open/inspect a demo PR comment and download `leanci-metrics`. +4. Run `python scripts/assert_planted_bug.py --metrics leanci-metrics.json`. diff --git a/docs/media/README.md b/docs/media/README.md new file mode 100644 index 0000000..448c645 --- /dev/null +++ b/docs/media/README.md @@ -0,0 +1,12 @@ +# Media placeholders + +Add Devpost / README visuals here before final submit: + +| File | Suggested content | +| --- | --- | +| `architecture.png` | Diff → expand → Paritok agent → receipt | +| `pr-comment.png` | LeanCI PR comment with findings + cost table | +| `receipt.gif` | Short scroll of dual-run receipt columns | +| `actions-run.png` | Green LeanCI Demo jobs (dual + assert) | + +Keep secrets out of frames. Prefer 1280×720 or larger. diff --git a/scripts/demo_video_script.md b/scripts/demo_video_script.md new file mode 100644 index 0000000..cb6db8b --- /dev/null +++ b/scripts/demo_video_script.md @@ -0,0 +1,63 @@ +# LeanCI demo video script (≤ 3 minutes) + +**Target length:** 2:45–3:00 +**Tone:** economics-first, show don’t tell +**Visuals:** IDE/PR → Actions → PR comment receipt → optional Paritok dashboard + +--- + +## Beat sheet + +| Time | Section | On screen | Voiceover (approx.) | +| --- | ---: | --- | --- | +| 0:00–0:25 | **Problem** | Innocent-looking PR diff on `charge.py` | “AI PR review is either shallow—diff only—or deep and expensive. Cross-file contract bugs hide outside the diff, and that’s exactly where token costs explode.” | +| 0:25–0:55 | **Architecture** | Simple diagram: Diff → Expand → Agent via Paritok → Receipt | “LeanCI expands imports and call sites under caps, runs the agent through a local Paritok proxy for compression, then posts findings with a measured cost receipt—not a marketing percentage.” | +| 0:55–2:10 | **Live demo** | Open demo PR → Actions **LeanCI Demo** green → PR comment | “Here’s our planted bug: `validate_charge` now returns a tuple, but checkout still treats it as a bool. Python tuples are always truthy—failed charges still go through. Tests stay green. LeanCI dual-run expands to both files, catches the break on compressed and uncompressed paths, and updates one PR comment.” | +| 2:10–2:40 | **Results** | Zoom receipt + metrics / assert PASS | “Parity: planted bug found on both paths. Metrics artifact uploaded. Assert script PASS. On tiny demos compression may be modest—we label that honestly. On tool-heavy loops, Paritok is why deep review can pencil out.” | +| 2:40–3:00 | **Key innovations** | Title cards | “Three ideas: dependency-aware expansion, Paritok on the agent path, and dual-run receipts with honesty labels. LeanCI—deep review you can measure.” | + +--- + +## Spoken script (full) + +### 1. Problem (~25s) + +AI pull request reviewers force a bad tradeoff. If you only read the diff, you miss cross-file bugs—like a helper’s return type changing while callers keep the old assumption. If you expand context and tool-loop aggressively, token costs spike until the review no longer makes sense for CI. + +### 2. Architecture (~30s) + +LeanCI is a GitHub Action. It collects the PR diff, expands Python imports, reverse call sites, and related tests under hard caps, then runs a tool-using agent. On the compressed path, the agent talks to a **local Paritok proxy**, which compresses eligible traffic to your upstream OpenAI-compatible model. Findings are normalized, a cost receipt is built from Paritok stats and a dated price table, and one idempotent PR comment is published—plus a metrics JSON artifact. + +### 3. Live demo (~75s) + +Open the demo PR in `demo/`. The change looks small: `validate_charge` returns `(ok, reason)`. Checkout still does `if validate_charge(...):`—and because a non-empty tuple is always truthy, invalid amounts still capture. The happy-path test doesn’t catch it. + +Trigger **LeanCI Demo** with `mode=dual_run`. Watch the Action start Paritok, run compressed then uncompressed on the same expansion set, and post the review. The comment cites **both** `payments/charge.py` and `api/checkout.py`. The assert job downloads metrics and passes. + +### 4. Results (~30s) + +Show: findings severity, parity flags true/true when both legs succeed, artifact upload, wall time under the demo budget when possible. Call out the receipt columns. If reduction is low on a tiny fixture, say so—honesty is part of the product. + +### 5. Key innovations (~20s) + +1. **Dependency-aware expansion** for bugs the diff alone won’t show. +2. **Paritok in the loop** so deep agent review can be economical. +3. **Dual-run + assert** so savings and finding quality are measurable, not claimed. + +Close on logo / repo URL / Apache-2.0. + +--- + +## B-roll checklist + +- [ ] Diff view of `charge.py` + `checkout.py` +- [ ] Actions run (Paritok start → dual_run → assert) +- [ ] PR comment with `` receipt table +- [ ] `leanci-metrics.json` snippet (parity + tokens) +- [ ] Optional: Paritok dashboard attribution + +## Recording notes + +- Prefer 1080p; zoom browser to 125% for readability. +- Blur or omit secret values in the Actions env panel. +- Do not claim a fixed “70% savings” unless the on-screen receipt shows it.