Skip to content

Enable end-to-end LeanCI PR review pipeline - #1

Merged
CodewithJha merged 29 commits into
mainfrom
feat/e2e-review-pipeline
Jul 31, 2026
Merged

Enable end-to-end LeanCI PR review pipeline#1
CodewithJha merged 29 commits into
mainfrom
feat/e2e-review-pipeline

Conversation

@CodewithJha

@CodewithJha CodewithJha commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • Wire the GitHub Action through ParitokGateway and the orchestrator so PR runs execute a real review instead of dry-run config printing.
  • Add LLM client, agent loop, findings normalizer, pricing/receipt, GitHub publisher, metrics export, and dual-run controller (gated behind mode=dual_run).
  • Document required secrets and the first end-to-end verification checklist in the README.

Test plan

  • Confirm LeanCI workflow runs on this PR
  • Confirm Paritok proxy starts successfully
  • Confirm review executes and posts/updates a PR comment with findings + cost receipt
  • Confirm leanci-metrics artifact uploads
  • Confirm a follow-up push updates the same comment (no duplicates)
  • Local: pytest (254 tests) green

Priyanshu Jha added 13 commits July 30, 2026 18:12
Extend the TDD §14.1 taxonomy so later review stages can fail with typed, user-visible errors.
Start the local proxy, export OPENAI_BASE_URL, run LeanCI, and always stop the proxy. Upload metrics artifacts from the composite Action.
Support compressed (Paritok proxy) and uncompressed (provider) chat completions with retries for transient failures.
Run tool-calling turns against ToolHost, repair invalid JSON once, and package prompt markdown for installable builds.
Validate raw JSON into typed Finding objects and enforce severity floor plus max_findings.
Provide model rate lookups and assumption labels used by honest review receipts.
Compute token reduction and estimated USD with honesty labels for missing baselines and weak compression.
Create or update the marked review comment with findings, receipt table, and failure-comment rendering.
Serialize the §17.1 metrics contract to leanci-metrics.json for Actions upload.
Sequence diff through publish and metrics, post failure comments on errors, and run the pipeline from python -m leanci.
Run both LLM bindings sequentially on the same manifest and compute planted-bug parity flags.
Record required PARITOK and OpenAI secrets plus the minimal PR verification steps for M3.6.
Bare paritok lacks uvicorn; install paritok[proxy] as required by the Paritok CLI.
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

LeanCI could not complete

Error class: LLMError
What happened: LLM HTTP 400 from http://127.0.0.1:8080/v1/chat/completions: Bad Request — {"error":{"message":"Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.","type":"invalid_request_error","code":"tool_use_failed","failed_generation":"<function=expand_context[]{"shadow_id": "e07d01a14ceaf205"}</function>"}}
Next steps:

Priyanshu Jha added 16 commits July 30, 2026 18:29
Allow --openai-url so the Action can use OpenRouter (or other providers) with the existing OPENAI_API_KEY secret.
Avoid 402 Payment Required while validating the review pipeline end-to-end.
openrouter/free queued past 120s; give Paritok+upstream more room and pin gpt-oss-20b:free for CI e2e.
Use Gemini's OpenAI-compatible chat endpoint so e2e can run without OpenRouter credits.
gemini-2.0-flash has free-tier quota limit 0 on this key; flash-latest still serves.
Surface provider/Paritok 400 details in CI comments instead of only the reason phrase.
Echo extra_content.google.thought_signature so multi-turn function calls do not 400.
Honor Retry-After with a longer 429 floor and shrink CI tool/file caps so reviews fit under RPM.
Four attempts with 60–90s floors so a single review can clear the RPM window.
flash-latest (3.6) is stuck at free-tier RPM; lite has a working quota on this key.
Use llama-3.3-70b-versatile with roomier tool-turn caps so reviews can finish.
Front Paritok→Groq with a UA-bearing local hop, and finalize JSON on the last agent turn.
Switch to 8b-instant and tighter file/diff caps so the first chat request stays under 12k tokens.
Drop to 2 files and a shorter diff cap so free-tier chat requests fit.
8b fits TPM but fails tool-calling; 70b should stay under 12k tokens with 2 files.
The e2e agent finished; pricing KeyError was the only remaining hard fail.
@CodewithJha
CodewithJha force-pushed the feat/e2e-review-pipeline branch from c54f66d to 73eba26 Compare July 31, 2026 07:21
@CodewithJha
CodewithJha merged commit 4ba7871 into main Jul 31, 2026
1 check failed
CodewithJha added a commit that referenced this pull request Jul 31, 2026
Enable end-to-end LeanCI PR review pipeline
CodewithJha added a commit that referenced this pull request Aug 1, 2026
Enable end-to-end LeanCI PR review pipeline
CodewithJha added a commit that referenced this pull request Aug 1, 2026
Enable end-to-end LeanCI PR review pipeline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant