Skip to content

feat(wiki): collect and extract Scala files in the code graph (#981) - #996

Open
yudongyouqing wants to merge 9 commits into
Tencent:mainfrom
yudongyouqing:feat/scala-extractor
Open

yudongyouqing wants to merge 9 commits into
Tencent:mainfrom
yudongyouqing:feat/scala-extractor

Conversation

@yudongyouqing

Copy link
Copy Markdown

Problem

codebase --extract / repo import silently dropped every .scala file, so Java/Scala mixed projects produced a Java-only code graph (#981): isCodeFile() did not whitelist .scala, and EXTRACTOR_REGISTRY had no scala entry.

Changes

Follows the minimal plan from #981, modeled on the Swift heuristic extractor:

  • Collector (code-collector.ts): whitelist .scala, map it to the scala language, mark Main.scala / App.scala as key files.
  • New regex extractor (extractors/scala.ts, registered in extractors/index.ts):
    • class / object / enum / case class / case object → component
    • def (past annotations and modifiers, incl. transparent inline def) → component
    • trait → interface
    • type names ending in Error / Exception → error (INFERRED)
    • sys.env(...) / System.getenv(...) reads → config
    • import in all four forms (plain, brace selector, _ wildcard, Scala 3 *) → relation (normalized to the dotted path prefix)
    • a match clause (case Invoice(id) =>) is not mistaken for a case class declaration
  • Consistency: scala added to the extension strips in call-chain-tracer.ts (entry patterns + import normalization) and import-repo.ts cross-repo import matching, alongside java/swift.
  • skill-data sync: entry-file list in kb-doc-generator.md, language map in scan_repo.py.

Test plan

  • npx tsc --noEmit, npm run lint clean
  • New suite src/__tests__/scala-extractor.test.ts (6 tests: registry, declarations, modifiers/match-clause safety, all four import forms, error/config inference, .scala collection with key-file marking)
  • All 11 suites importing the touched modules pass (121 tests), including swift/java/AST regression suites and cross-repo-edges
  • Real-CLI verification (built CLI 0.22.0, Windows 11, git-initialized mixed Java/Scala fixture):
$ teamai codebase --extract <mixed-repo> --project demo
[extract] demo complete
  Files: 3
  Facts: 13 (relation:3, component:6, error:2, interface:1, config:1)
  Graph: 16 nodes, 5 edges

Graph nodes include component/DefaultGateway, component/Invoice, component/Main, component/PaymentError, component/charge, interface/PaymentGateway, error/PaymentError, config/PAYMENTS_API_KEY, and source-manifest.json lists both .scala files. Before this change, only the .java file was collected.

Out of scope (follow-ups)

Closes #981

🤖 Generated with Claude Code

…t#981)

.scala files were silently dropped by the code collector, so Java/Scala
mixed projects produced a Java-only graph. Whitelist the extension, map
it to the scala language, and add a regex heuristic extractor following
the Swift extractor's shape: classes/objects/enums and defs as
components, traits as interfaces, Error/Exception-suffix types as
errors, sys.env/System.getenv reads as configs, and imports (plain,
brace, `_` and `*` wildcards) as relation facts.

Also add scala to the extension strips in call-chain-tracer and
import-repo cross-repo matching, mark Main.scala/App.scala as key
files, and sync the skill-data language map and entry-file list.

Fixes Tencent#981

Co-Authored-By: Claude Code <noreply@anthropic.com>
@jeff-r2026 jeff-r2026 self-assigned this Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:54 emits dotted relation names such as com.foo.Bar, but buildCodeGraph() and resolveRelationTarget() compare them against slash-separated paths such as src/main/scala/com/foo/Bar.scala. Therefore an ordinary internal Scala import produces no dependency edge or traced call-chain step, leaving the headline code-graph support incomplete. Normalize Scala import paths to the representation expected by both consumers, and add a graph-edge regression test.

  • [P2 non-blocking] src/wiki-engine/call-chain-tracer.ts:27 adds Scala to lowercase, case-sensitive main/app patterns, while the collector marks Main.scala and App.scala as key files. A standard object Main extends App in Main.scala is therefore never selected as a call-chain entry point. Make these filename patterns case-insensitive or explicitly support the capitalized names.

The PR description includes sufficient representative real-CLI verification for this runtime change.

… review)

Relation facts carried dotted package paths (com.foo.Bar), which neither
buildCodeGraph's fuzzy file match nor the call-chain tracer's module map
could match against slash-separated file paths — an internal Scala
import produced no DEPENDS_ON edge. Emit the import target as a path
(com/foo/Bar) instead; both consumers then resolve it (exact, /index and
basename lookups all split on /). Add a regression test for the edge.

Also make the main/server/app entry-file patterns case-insensitive so
Scala's Main.scala / App.scala key files are selected as call-chain
entry points.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

Both findings are addressed in 0a33a3d:

  • P1 — the Scala extractor now emits relation targets as slash-separated paths (import com.foo.Bar → com/foo/Bar), which is what both consumers match on: buildCodeGraph()'s fuzzy path match and the call-chain tracer's module map (its exact, /index and basename lookups all split on /). Added a regression test asserting the DEPENDS_ON edge for an internal import, and re-verified on the real CLI:

    DEPENDS_ON  src/main/scala/com/demo/payments/PaymentGateway.scala -> src/main/scala/com/demo/core/Invoice.scala [code-heuristic]
    

    (Side observation, out of scope here: the Java extractor has the same latent gap — its dotted FQCN imports never match a path either.)

  • P2 — the main/server/app entry-file patterns in call-chain-tracer.ts are now case-insensitive, so Scala's Main.scala / App.scala key files are selected as entry points; the fixture run now reports Call chains: 2 chains (max depth 1).

tsc, oxlint and the affected suites (scala/swift/codebase-extract/import-repo/call-chain/wiki-engine) all pass.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:58 discards brace-import selectors. For import com.demo.core.{Invoice} with Invoice.scala and InternalAdmin.scala, it emits only com/demo/core; buildCodeGraph() therefore adds dependencies to both files via substring matching, while resolveRelationTarget() resolves neither because no module is named core. Emit a relation per selected symbol and add graph/tracer regression coverage.

The two earlier findings are resolved: direct imports now use slash-separated paths, and Scala entry-file matching is case-insensitive. The PR description includes sufficient representative real-CLI verification.

…t#996 review)

A brace selector was discarded and only the package path emitted, so
buildCodeGraph tied the importer to every file in the directory via
substring matching while the call-chain tracer resolved nothing (no
module is named after a package segment). Emit one relation per
imported symbol — com.foo.{Bar, Baz => B} becomes com/foo/Bar and
com/foo/Baz; renames take the name before =>, and the in-brace _
wildcard / nested selectors fall back to the package path, which is
what a wildcard honestly names.

Add regression coverage for both consumers: the graph edge test
asserts a {Invoice} import links exactly Invoice.scala and not a
sibling in the same package, and a traceCallChains test asserts the
entry step resolves Invoice and never touches the sibling.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

Addressed in 4aa533b: a brace selector now expands to one relation per imported symbol (com.foo.{Bar, Baz => B} → com/foo/Bar, com/foo/Baz), so the graph edge lands on the named file instead of every file in the package directory, and the tracer resolves the symbol's basename. Renames take the name before =>; the in-brace _ wildcard and nested selectors fall back to the package path, which is what a wildcard honestly names.

Regression coverage for both consumers, as requested:

  • graph: a {Invoice} import produces exactly one DEPENDS_ON edge — to Invoice.scala, none to a sibling InternalAdmin.scala in the same package
  • tracer: a traceCallChains test asserts the entry step's callsTo contains Invoice and no step touches the sibling

Real-CLI re-verification (mixed fixture, import com.demo.core.{Invoice} in PaymentGateway.scala):

DEPENDS_ON  src/main/scala/com/demo/payments/PaymentGateway.scala -> src/main/scala/com/demo/core/Invoice.scala   (the only dependency edge; Main.scala in the same package directory is not linked)

tsc, oxlint and the scala/call-chain/wiki-engine suites pass.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:62 mishandles selectors combining exclusions or named imports with a wildcard. For import com.demo.core.{Invoice => _, _}, selectedSymbols() returns Invoice, so the graph creates a dependency on the explicitly excluded file while omitting every file actually included by _. Preserve wildcard/package coverage and exclude hidden symbols.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:101 does not recognize Scala 3 as renames. import com.demo.core.{Invoice as Inv} falls back to com/demo/core, causing buildCodeGraph() to link every file in that package rather than only Invoice.scala.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:59 emits the full path for member imports such as import com.demo.core.Invoice.apply. Neither relation consumer can match .../Invoice/apply to Invoice.scala, so this ordinary Scala import produces no dependency edge.

The two findings from the first review and the brace-import finding from the second review are resolved. The PR description includes sufficient representative real-CLI verification.

A `{Invoice => _, _}` selector inverted its meaning: the hidden name got
its own relation (a dependency on the excluded file) while the wildcard's
package-wide import produced nothing. Parse the selector into a
wholePackage flag plus named symbols — `_` marks the package, an alias of
`_` hides its name, Scala 3 `as` renames like `=>` take the name before
the alias — and emit only the package path when the selector imports
wholesale.

Member imports (`com.foo.Bar.apply`) named a path no file has; trim the
lowercase tail after the last type name, since the file that defines the
type is what the consumers match.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All three addressed in df6231f:

  • P1 (hiding + wildcard) — the selector now parses into a whole-package flag plus named symbols: _ marks the package, an alias of _ (Invoice => _, Scala 3 Invoice as _) hides its name and emits nothing, and a wholesale import (wildcard present, or only hidden names left) keeps the package path. {Invoice => _, _} therefore emits only com/demo/core — no per-symbol relation for the hidden name.
  • P2 (as renames) — {Invoice as Inv} parses like {Invoice => Inv} and resolves to com/demo/core/Invoice.
  • P2 (member imports) — com.foo.Bar.apply trims the lowercase tail after the last type name (the type's file is what the consumers match) and resolves to com/foo/Bar.

Unit coverage: the import-forms test now includes as renames and a member import, and a new test asserts {Invoice => _, _} and {Order => _} each emit only the package path.

Real-CLI re-verification (fixture exercising exactly these forms):

src/main/scala/com/demo/core/Main.scala              -> src/main/scala/com/demo/core/Invoice.scala   (import com.demo.core.Invoice.apply)
src/main/scala/com/demo/payments/PaymentGateway.scala -> src/main/scala/com/demo/core/Invoice.scala   (import com.demo.core.{Invoice as Inv})
src/main/scala/com/demo/payments/PaymentGateway.scala -> src/main/scala/com/demo/core/Main.scala       (import com.demo.core.{Invoice => _, _} — package-wide, no relation names the hidden Invoice)

tsc, oxlint and the scala suite (10 tests) pass.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:65 still does not honor wildcard exclusions. import com.demo.core.{Invoice => _, _} emits the package relation com/demo/core; buildCodeGraph() then substring-matches both Invoice.scala and every other file under that package, creating a dependency on the explicitly excluded Invoice. The previous exclusion finding remains unresolved.

  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:66 represents wildcard imports only as a package path, but resolveRelationTarget() indexes files and basenames—not directories. For a Main.scala importing com.demo.core._, the graph gets edges, but the call-chain step has no targets because no module named core exists.

  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:61 silently misparses multiline brace imports. For import com.demo.core.{ followed by selectors on later lines, the first line succeeds as a bare package import, widening the dependency to every file in the package rather than the selected symbols.

  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:61 only reads the first expression in a valid comma-separated import such as import com.demo.A, com.demo.B; the unanchored regex emits A and silently drops B.

The earlier direct-import, capitalized entry-file, rename, and lowercase member-import findings are resolved. The PR description includes sufficient representative real-CLI verification.

… imports

A wildcard import still emitted the package path, so buildCodeGraph's
substring match tied the importer to every file under the package —
including a name the selector hides — and the call-chain tracer resolved
nothing (no module is named after a package segment). Expand a wildcard
over the collected files of the package instead, skipping hidden names,
and keep the package path only when no collected file matches (an
external package).

Also parse the import forms the line-based scan mishandled: a brace
selector wrapped across lines by scalafmt opened as a bare package
import, and a comma-separated `import a.B, c.D` dropped every clause
after the first. Track a wrapped selector until its `}`, and split a
line into clauses (brace contents keep their own commas).

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All four addressed in e1b95f0:

  • P1 (wildcard exclusions) — the root cause was that a wildcard emitted the package path, whose substring match reaches every file under the package including a hidden name. A wildcard now expands over the collected files of that package — exact relative paths — skipping names the selector hides, so {Invoice => _, _} links only the sibling files and never Invoice.scala. The package path survives only when no collected file matches (an external package).
  • P2 (wildcard → tracer) — the expansion emits full file paths, which resolveRelationTarget() hits via its exact-path key (the fixture now traces chains of depth 2 where it previously stopped).
  • P2 (wrapped brace imports) — an unterminated { opens a pending selector that accumulates lines until its }, so scalafmt-wrapped imports parse as their symbols instead of a bare package.
  • P2 (comma-separated imports) — import a.B, c.D splits into clauses (brace contents keep their own commas); every clause emits.

Unit coverage: five new tests (expansion with hidden names, plain ._ expansion, wrapped selector, comma-separated clauses, package-path fallback). The 14-test scala suite plus the consumer suites (call-chain, wiki-engine, graph-aggregate, cross-repo-edges, 86 tests) pass.

Real-CLI re-verification — fixture exercising all four forms:

src/main/.../Main.scala               -> src/main/.../core/Invoice.scala   (import com.demo.core.Invoice.apply)
src/main/.../payments/PaymentGateway  -> src/main/.../core/Invoice.scala   (import com.demo.core.{Invoice as Inv})
src/main/.../payments/PaymentGateway  -> src/main/.../core/Main.scala       (import com.demo.core.{Invoice => _, _} — Invoice hidden, no edge from the wildcard)

8 relation facts — exactly one per import clause in the fixture, confirming the wrapped and comma-separated forms parse completely.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Findings

  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:149 recognizes _ but not Scala 3 *. For import com.demo.core.{Invoice as _, *}, it emits a bare package relation, causing buildCodeGraph() to re-add the explicitly hidden Invoice; import com.demo.core.* at src/wiki-engine/code-knowledge/extractors/scala.ts:73 likewise never expands, so call-chain resolution returns no targets.
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:187 expands wildcards using only the Scala-language batch. In a mixed project where Main.scala imports com.demo.core.{Invoice => _, _} and Invoice.java/Order.java provide that package, expansion is empty and the package fallback links both Java files—including the excluded Invoice. If a Scala candidate exists, Java wildcard targets are omitted instead.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:191 assumes an imported symbol has a same-named source file. Scala commonly defines Invoice inside Models.scala; import com.demo.core.Invoice then matches neither consumer, producing no graph edge or call-chain step.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:201 treats descendant directories as direct package members. import com.demo.core._ therefore emits a dependency on com/demo/core/internal/Admin.scala, although that class belongs to the com.demo.core.internal subpackage.

Earlier direct-import normalization, capitalized entry matching, brace selectors, renames/member imports, multiline imports, and comma-separated imports are resolved. The PR description includes sufficient representative real-CLI verification.

… files

Four import gaps:

- `*` never expanded: the clause regex dropped it before the wildcard
  check, and an in-brace `*` entry was not recognized as a wildcard at
  all — both treated as a plain package import whose substring match
  re-added names the selector hides.
- Wildcard expansion saw only the Scala batch, so a package provided by
  Java files fell back to the package path (linking a hidden name), and
  Scala candidates masked Java ones. extractForLanguage now passes every
  collected file to the extractor.
- A named import assumed a same-named file (`Invoice` in `Invoice.scala`)
  though Scala freely puts many types in one `Models.scala`. The symbol
  now resolves to the package file that declares it, falling back to the
  conventional path.
- Wildcard membership matched descendant directories too, so a
  subpackage's files rode along on the parent package's import; direct
  package members only.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All four addressed in 13cbdf6:

  • P1 (Scala 3 * wildcards) — two gaps, both fixed: the clause regex now keeps a trailing .* so import com.demo.core.* expands, and parseSelectors treats a * entry like _. {Invoice as _, *} expands over the package files minus the hidden name.
  • P1 (cross-language wildcards) — extractForLanguage now passes every collected file to the extractor, and a wildcard expands over all of them: a package provided by Java files expands to those files (minus hidden names), and Scala candidates no longer mask Java ones. Optional parameter, other extractors untouched.
  • P2 (declaring file) — a named import resolves to the direct package member whose content declares the symbol (built from the same declaration regexes), so import com.demo.core.Order links Models.scala when that is where Order lives; the conventional same-name path remains the fallback, and resolution now applies to plain (non-brace) imports too.
  • P2 (subpackages) — wildcard membership is direct members of the package directory only; com.demo.core.internal.Admin.scala no longer rides along on com.demo.core._.

Unit coverage: five new tests (Scala 3 * expansion, Java-file expansion minus hidden, named import → declaring file, subpackage exclusion, plus the pre-existing _ expansion). The scala suite (19 tests) and all consumer suites pass — 104 tests total.

Real-CLI re-verification (mixed Java/Scala fixture):

src/main/scala/com/demo/app/Main.scala      -> src/main/java/com/demo/core/Order.java   (import com.demo.core.{Invoice => _, *} — Invoice.java hidden, no edge)
src/main/scala/com/demo/app/Main.scala      -> src/main/scala/com/demo/core/Models.scala
src/main/scala/com/demo/payments/Gateway.scala -> src/main/scala/com/demo/core/Models.scala   (import com.demo.core.Order — Order is declared in Models.scala)

Invoice.java receives no edge; traced call chains reach depth 3.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Findings

  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:154 resolves wildcards only against the files supplied to the current extraction. During teamai codebase --extract --incremental, files contains only changed files, so editing an importer while its targets remain unchanged makes expansion fail and fall back to the package path. The rebuilt graph then reintroduces excluded files and subpackages through fuzzy matching; named imports from unchanged nonconventional files such as Models.scala also lose their edge.
  • [P1 blocking] src/ci/extract-mr.ts:259 still filters changed source files with /\.(ts|tsx|js|jsx|py|go|rs|java)$/. A PR containing only .scala changes therefore reports no codebase graph changes, despite Scala now being a supported collected language.
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:237 excludes wildcard-hidden symbols by filename only. If Models.scala declares only Invoice, import com.demo.core.{Invoice => _, _} still emits a relation to Models.scala, producing a dependency on the explicitly excluded symbol.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:235 expands wildcards over every collected language. For example, src/main/resources/com/demo/core/schema.sql is treated as a member of com.demo.core; if it contains an extracted SQL fact, import com.demo.core._ creates a false dependency on that resource. Restrict expansion to JVM-importable sources.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:164 treats every all-lowercase plain import as a package. A valid Scala 3 top-level definition such as def validate in Helpers.scala, imported through import com.demo.core.validate, cannot resolve to that file, so neither graph consumer records the dependency.

The previously reported full-extraction issues are resolved. The PR description includes sufficient representative real-CLI verification.

An incremental run re-extracts only changed files, so the extractor's
resolution context went blind: wildcard expansion fell back to the
package path (fuzzy-matching hidden names and subpackages back in) and
named imports lost the file that declares the symbol. extractCodeFacts
now takes the run's full file list plus the previous run's declarations
(rebuilt from the cached facts — component facts carry name and file),
so unchanged files resolve without being re-read.

Also: hidden-name exclusion now consults the declarations, not just the
file name — Models.scala declaring only a hidden Invoice is skipped too;
wildcard membership is JVM-importable sources only, so a schema.sql
beside the package does not become a dependency; a plain import always
tries its last segment as a symbol, which resolves Scala 3 top-level
defs (import com.demo.core.validate → Helpers.scala); and a file's
several imports of the same target collapse to one relation. The MR
diff filter in ci/extract-mr.ts now collects swift and scala as well.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All five addressed in bb480c1:

  • P1 (incremental blindness) — the root cause was that the extractor's resolution context was built from the changed-files batch alone. extractCodeFacts now receives an ExtractorContext: the run's full file list (changed files plus content-less stubs for unchanged ones) and the previous run's declarations, rebuilt from the cached facts — component facts already carry name and file, so unchanged files resolve without being re-read. The fresh parse overrides the cached declarations for changed files.
  • P1 (ci/extract-mr filter) — scala added; swift added alongside it, since the collector supports it and its omission looked like the same oversight.
  • P1 (hidden by declaration, not file name) — wildcard exclusion consults the declarations map: a Models.scala whose only declared symbol is hidden is skipped as well; a file declaring a mix of hidden and live symbols stays (the wildcard genuinely imports the live ones).
  • P2 (non-JVM members) — wildcard membership is .scala/.java direct members only, so schema.sql beside the package is not a dependency.
  • P2 (top-level defs) — a plain import always resolves its last segment against the package's declarations (defs are now indexed), so import com.demo.core.validate reaches Helpers.scala; the conventional path remains the fallback. Also, a file importing the same target twice (wildcard plus a named symbol) now emits one relation instead of two.

Unit coverage: five new tests (incremental context with stubs + cached declarations, declared-hidden exclusion, sql-resource exclusion, top-level def resolution, plus reworked cross-language/dedupe expectations) — 24 scala tests, 118 with the consumer suites, all passing.

Real-CLI verification of the incremental path specifically — full extraction, then only Main.scala touched, --incremental:

full:        Main.scala -> Order.java, Models.scala; Gateway.scala -> Models.scala
incremental: Files: 1 (the changed importer) — identical three DEPENDS_ON edges,
             identical 12 facts / 13 nodes

Hidden Invoice.java, schema.sql, and the internal/ subpackage stay out of the graph in both runs.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Findings

  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:173 materializes wildcard imports into the files present during extraction, but incremental runs preserve relations from unchanged importers. After an initial Main.scala imports core.*, adding only core/Order.scala never re-extracts Main.scala, so its cached relation omits Order.scala and the rebuilt graph has no dependency edge. Target additions, removals, and declaration moves must invalidate affected Scala importers or retain a symbolic wildcard relation.
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:321 treats every nested def as a package-level declaration when filtering wildcard exclusions. For class Invoice { def total = ... } and import core.{Invoice => _, _}, the declaration set contains both Invoice and total, so Invoice.scala is retained even though total is not imported from the package and Invoice was explicitly hidden.
  • [P2 non-blocking] src/codebase-extract.ts:617 reconstructs cached declarations from every non-relation fact, including configs and other non-importable facts. After an importer-only incremental run, a file containing only class Invoice plus sys.env("API_KEY") has cached names Invoice and API_KEY; import core.{Invoice => _, _} therefore incorrectly restores a dependency on that file.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:173 treats every wildcard prefix as a package directory. A normal object import such as import com.demo.Models.*, where object Models is declared in Domain.scala, falls back to com/demo/Models; neither graph consumer can resolve that to Domain.scala, so the dependency disappears.

The previously reported direct-import, entry-point, selector, multiline, Scala 3 wildcard, cross-language, subpackage, and CI-filter cases are resolved. The PR description contains sufficient representative real-CLI verification.

A wildcard materializes to the package files present at extraction
time, so an incremental run that only adds a package member left the
unchanged importer's cached relations stale — no edge to the new file.
Each wildcard now also emits a symbolic `scala-wildcard:<package>`
relation (matched by no consumer) and the incremental pass re-extracts
every importer whose wildcard package gained or lost a scala/java file.

Nested defs are no longer package-level declarations: the declarations
index tracks brace depth and column, so a class member `def total` does
not rescue a file whose only top-level name the selector hides.

Cached declarations rebuild from component/interface facts only — a
config fact like `API_KEY` is not an importable symbol and used to
restore dependencies on files the selector hid.

A wildcard on an object (`import com.demo.Models.*`) now resolves to
the file declaring the object when the package directory holds nothing,
instead of falling back to a path no consumer matches.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All four addressed in b527575:

  • P1 (wildcard staleness in incremental runs) — materializing wildcards at extraction time indeed could not see members added later. Every wildcard now also emits a symbolic scala-wildcard:<package> relation (its name matches no file, so neither consumer resolves it), and the incremental pass reads those markers: a scala/java file added to, or removed from, a package forces re-extraction of every importer holding that package's marker. Verified on the real CLI — full run with import com.demo.core.{Invoice => _, _}, then only Order.java added to the package, --incremental:

    incremental: Files: 2  (the new member + the invalidated importer)
    Main.scala -> Order.java    (new edge appears)
    Main.scala -> Models.scala  (previous edge preserved)
    
  • P1 (nested defs in the declarations index) — the index now records a declaration only at brace depth zero and column zero, so class Invoice { def total } contributes Invoice alone; a file whose only top-level name is hidden is excluded even when the wildcard would have "seen" the nested def.

  • P2 (cached declarations from config facts) — the incremental context rebuilds declarations from component/interface facts only; API_KEY no longer counts as an importable name.

  • P2 (wildcard on an object) — import com.demo.Models.* with the package directory empty falls back to resolving Models against the package's declarations, landing on Domain.scala like a named import would; an all-lowercase package prefix keeps the package path.

Unit coverage: 24 scala tests (three new: nested-def exclusion, object wildcard, and the marker expectations across the wildcard tests), 96 consumer-suite tests alongside — all passing.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Findings

  • [P1 blocking] src/codebase-extract.ts:603 invalidates only relations carrying scala-wildcard: markers. A named import is also materialized to its current declaring file: if Invoice moves from Models.scala to Domain.scala while the importer remains unchanged, its cached relation still targets Models.scala, so the rebuilt graph loses the dependency. Named importers must also be invalidated when declaration locations change, or retain a symbolic relation.
  • [P1 blocking] src/codebase-extract.ts:636 reconstructs cached declarations from every component fact, including nested methods. After an importer-only incremental run, class Invoice { def total = ... } produces cached names Invoice and total; import core.{Invoice => _, _} consequently retains that file because total is not hidden, reintroducing the explicitly excluded Invoice.
  • [P1 blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:338 rejects every indented declaration as non-package-level. With ordinary Scala 3 package syntax such as package com.demo.core: followed by an indented case class Invoice in Models.scala, named imports cannot resolve the declaration, and wildcard exclusions still include Models.scala after hiding Invoice.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:179 emits the internal scala-wildcard: invalidation marker as a normal relation fact. It is written into relation evidence and counted by detectKnowledgeGaps() as unresolved; six internal wildcard imports therefore produce a false “external dependencies not documented” gap. Keep invalidation metadata out of user-facing relation facts or explicitly filter it from consumers.

The prior direct-import, entry-point, selector, multiline, wildcard expansion, cross-language, subpackage, CI-filter, object-wildcard, and config-cache findings are resolved. The PR description includes sufficient representative real-CLI verification.

A named import materializes to its current declaring file, so a
declaration moving between files staled the unchanged importer's cached
relation just like a package member change stales a wildcard. The
incremental pass now re-extracts an importer whenever a cached
materialized target (a relation name ending in .scala/.java) is among
the changed or deleted files, alongside the existing wildcard-package
rule.

The declarations index no longer rebuilds from component facts, which
cannot tell a nested member from a package-level name. Each Scala file
now records its top-level names in a `scala-decl:` metadata relation,
and the incremental context rebuilds from those markers alone.

Top-level detection also learns the Scala 3 `package com.demo.core:`
block: its members sit one indent in, and declarations deeper than the
block's first indent level are members, not package names.

The `scala-wildcard:`/`scala-decl:` metadata relations are now filtered
from the relation evidence page, the gap detector, and the graph
builder, so they no longer surface as undocumented dependencies.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@yudongyouqing

Copy link
Copy Markdown
Author

All four addressed in 379fcac:

  • P1 (named imports go stale on declaration moves) — the invalidation rule now covers both materialized forms: a cached relation name that ends in .scala/.java IS the materialized target file, so when it appears among the changed/deleted files the importer re-extracts and re-resolves. Verified on the real CLI: Order moved from Models.scala to a new Domain.scala with both importers untouched — --incremental re-extracted both (the wildcard rule caught Main.scala, the new target rule caught Gateway.scala) and the graph shows Gateway.scala -> Domain.scala with no stale edge.

  • P1 (nested members in cached declarations) — the declarations index no longer rebuilds from component facts. Each Scala file emits a scala-decl:<names> metadata relation carrying exactly its top-level names, and the incremental context rebuilds from those markers alone — nested members were already excluded when the marker was written.

  • P1 (Scala 3 package x: blocks) — top-level detection measures the block's first indent level: members of the package sit there, declarations deeper are members of a type, exactly as with braces.

  • P2 (metadata relations in user-facing output) — the new shared isMetadataRelation() filter is applied at the three consumers: the relation evidence page, detectKnowledgeGaps() (the unresolved-imports count), and buildCodeGraph(). The fixture's relation-src.md and gaps/detected.md contain zero marker occurrences.

Unit coverage: 25 scala tests (one new: package-block declarations, plus the marker expectations across the suite), 121 tests with the consumer suites, all passing; tsc and oxlint clean.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Findings

  • [P1 blocking] src/codebase-extract.ts:610 invalidates named importers only when their cached target already ends in .scala or .java. If an unresolved import com.demo.core.Invoice is cached as com/demo/core/Invoice and a later incremental run adds Models.scala declaring Invoice, the unchanged importer is not re-extracted and the new dependency edge remains missing.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:210 turns an exclusion-only import such as import com.demo.core.{Invoice => _} into the package relation com/demo/core. The graph’s substring matching then creates dependencies on every file in that package even though the clause imports nothing.
  • [P2 non-blocking] src/wiki-engine/code-knowledge/extractors/scala.ts:49 stores each scala-decl: metadata marker as a normal relation fact. Although some consumers filter it, extractCodebase() still includes these markers in facts.total and the relation count, so CLI/JSON statistics and overview fact totals are inflated by one per declaring Scala file.

The exact findings from earlier passes are resolved. The PR description includes sufficient representative real-CLI verification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] 支持 Scala(.scala)代码图谱导入(Java/Scala 混编项目)

2 participants