Feature Request
Problem: The Walker and Scorer components currently process files sequentially. In repositories with thousands of files (e.g., large monorepos), the time to generate context exceeds acceptable CLI wait times, particularly when calculating both Git metadata scores and semantic relevance for every node.
Proposed solution:
Introduce an asynchronous processing layer in src/codectx/walker.py and src/codectx/ranker/scorer.py. Using Python's asyncio or concurrent.futures, the file walking and initial metadata extraction should be decoupled from the scoring logic to allow I/O-bound tasks (reading files) and CPU-bound tasks (parsing/scoring) to overlap.
Alternatives considered:
- Persistent Daemon: Keeping a background process to watch file changes (too complex for a CLI tool).
- Subsampling: Only analyzing a random subset of files (reduces context quality significantly).
Affected pipeline stage: [ Walker / Ranker ]
Checklist:
Feature Request
Problem: The
WalkerandScorercomponents currently process files sequentially. In repositories with thousands of files (e.g., large monorepos), the time to generate context exceeds acceptable CLI wait times, particularly when calculating both Git metadata scores and semantic relevance for every node.Proposed solution:
Introduce an asynchronous processing layer in
src/codectx/walker.pyandsrc/codectx/ranker/scorer.py. Using Python'sasyncioorconcurrent.futures, the file walking and initial metadata extraction should be decoupled from the scoring logic to allow I/O-bound tasks (reading files) and CPU-bound tasks (parsing/scoring) to overlap.Alternatives considered:
Affected pipeline stage: [ Walker / Ranker ]
Checklist: