Repository navigation
Architecture Map, Flow | Graph codebase view, and separate test graph - #19
Merged
Merged
Conversation
Implement core semantic graph generation following foglamp.dev/scan schema: - New detectors: detect_crons(), detect_tools(), detect_agents(), detect_composition_root() - build_semantic_graph() generates 14-node curated architecture map from detected components - Creates entry (routes), service, store, external, cron, agent, model, tool nodes - Generates edges with kinds: triggers, calls, reads, writes - Tracks component membership for drill-down filtering - Returns foglamp-compatible schema (project, stats, topModels/Tools/Integrations, graph) Verified on fastapi/full-stack-fastapi-template: 12 nodes, 21 edges, 0 agents/models/tools Next: render semantic graph in HTML with drill-down UI and import details toggle
Add SEMANTIC_GRAPH_JS renderer: deterministic column-by-kind SVG layout (cron/entry -> agent/service -> model/tool -> store/external), grouped node clustering with accent colors, click-to-detail panel, and a "Show import details" button that isolates the corresponding files in the existing Codebase Graph section below (via a new isolate-nodes hook added to GRAPH_JS) instead of duplicating a second graph engine. Fix bugs found via headless-Chrome visual + interaction testing (CDP): - group tags were appended to the wrong SVG parent, rendering off-canvas - entry/service linking was N x M fan-out instead of real import-based correlation (checked per-file text reference to the service module) - entry-node grouping used a URL-path-prefix heuristic that breaks when routers don't repeat their mount prefix in the route decorator string; replaced with file-based grouping (router filename -> feature name) - store/external branding used a nonexistent domain field from the detectors; added a static brand-domain lookup instead - external nodes included detect_external_sources()'s synthetic "Users / API Clients" and "Cron / Scheduler" entries, which aren't real third-party integrations - cron nodes fanned out to every service in the repo with no real evidence connecting them; now only link to the composition root Add 27 tests covering all 4 new detectors and build_semantic_graph(), including regressions for each bug above. Full suite: 107 passed. Verified end-to-end against fastapi/full-stack-fastapi-template via headless Chrome and the DevTools protocol: click a semantic node, panel opens, click "Show import details", and the exact files are isolated and highlighted in the Codebase Graph's existing search/isolate mechanism.
…derer Screenshotted the actual foglamp.dev/scan output (headless Chrome + software WebGL) since it hadn't been visually inspected before, and it looked meaningfully different from the first implementation: - Real layout reads left to right by data flow (source -> logic -> sink), not by a fixed kind-based column grid. Replaced the column assignment with a proper Sugiyama-style topological layering: each boxed group collapses to one meta-node, Kahn's algorithm assigns a longest-path layer, then real nodes inherit their meta-node's layer and expand back out. Fully isolated nodes (no in or out edges, e.g. a store nothing was confidently linked to) are pushed to the last layer instead of defaulting to layer 0, where they misleadingly read as a source. - Groups render as an actual bounding box around their stacked members (only when a group has >=2 real members - a "group" of one is just a plain node, not a box), with a short internal arrow between two members when a real edge connects them - not just a shared accent color. - Added small inline-SVG icon glyphs per kind (lightning bolt, filled dot, db-cylinder, globe) in a colored chip, replacing the old kind-badge text in the corner - no external icon font or network fetch. - Edges are visible by default (not dimmed until hover) with an always-on-brand color for labeled edges, plus a genuine per-edge animated flow dot via native SVG <animateMotion> along an <mpath> - the arrow animation from the reference site, with no extra JS animation loop needed. - Legend restyled as pill chips (Triggers / Services / Stores / External) matching the reference's fixed bottom bar language, deduplicated so entry+cron don't produce two "Trigger" pills. Verified via a synthetic 2-node-per-group fixture (the real fastapi template's deterministic detector doesn't happen to produce a >=1-member group large enough to exercise box rendering) - screenshots at each step confirm boxes, icons, and labeled edges all render as intended. Full suite still 107/107; click-to-detail-panel and the import-details drill-down re-verified working after the rewrite.
… output Compared against an actual foglamp.dev/scan screenshot and closed the remaining gaps: gateway and frontend nodes, synthesized Sentry and SMTP sink nodes from the composition root, a working CSS dash-flow animation on edges (the JS scaffolding from the prior session had no matching keyframes), fan-out-aware edge anchoring, a dotted-grid canvas background, a bottom-pinned pill legend, a floating Integrations card, and muted-gray edge labels instead of pink. Also fixes an UnboundLocalError in topIntegrations construction that had never been exercised end-to-end before, with a regression test.
The service->store edge only fired when the service file spelled out the store's name literally (e.g. "postgres" in crud.py). That misses the common case of a layered app where the connection lives in a separate db/core module and the service only ever sees a Session/db object (SQLModel, SQLAlchemy, Django ORM, Mongoose, etc.) - Postgres was showing up completely disconnected in the FastAPI template demo because of this. Add a fallback: when a service shows generic session.add()/.exec()/ .objects.create()-style usage and there's exactly one non-cache store detected repo-wide, link them. With more than one candidate store the link stays unmade rather than guessing which one is meant.
Architecture Map service-node labels were purely mechanical: a fixed stem dictionary (crud/services/service/repository/dao) or nothing. Generically-named business-logic files got no label improvement at all. Add enrich_service_labels(): opt-in, shells out to the user's local `claude` CLI in headless mode (-p --output-format json) to relabel existing service nodes with content-derived labels instead of the stem dictionary. Never invents new nodes/edges. No-op whenever `claude` isn't on PATH, the call times out, or the response is malformed -- falls back to the deterministic labels, never raises. Wired through all four distribution surfaces identically: - CLI: --enrich-labels flag - MCP: enrich_labels parameter on generate_workflow - Copilot extension: enrichLabels request field - Skill: always passes --enrich-labels (already an authenticated claude session, so the cost is already accepted) Verified against a fresh full-stack-fastapi-template clone: labels went from "CRUD service" / "routing + middleware" to "User/Item data access" / "CORS middleware + Sentry + routing", within the existing label/sub/detail caps. A run without the flag reproduces the prior deterministic output exactly. 9 new tests (all mock the subprocess boundary -- no real claude process in CI): claude-not-on-PATH no-op, enrich_labels=False never shells out, valid relabel applied, oversized fields truncated to caps, unknown ids ignored, malformed JSON / non-zero exit / timeout all fall back safely, no-op when there are no service nodes. Full suite: 110 -> 119.
…and UI styling improvements
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Architecture Map and reworks the codebase graph section of the generated
WORKFLOW.htmlso it's easier to read.Architecture Map (components)
Codebase (files): Flow | Graph toggle
prefers-reduced-motionand pauses off-screen.Tests (files)
tests/,test_*.py,conftest.py,*.spec.ts,*_test.go, ...) are split into their own section with the same Flow | Graph toggle, so fixtures no longer flood the main graph with service edges. On this repo the main graph drops from 123 to 35 edges.Page polish
Testing
python3 -m pytest: 122 passed, including new tests for the test/source split and the merged section markup.