Repository navigation
feat(wiki): collect and extract Scala files in the code graph (#981) - #996
yudongyouqing wants to merge 9 commits into
Conversation
…t#981) .scala files were silently dropped by the code collector, so Java/Scala mixed projects produced a Java-only graph. Whitelist the extension, map it to the scala language, and add a regex heuristic extractor following the Swift extractor's shape: classes/objects/enums and defs as components, traits as interfaces, Error/Exception-suffix types as errors, sys.env/System.getenv reads as configs, and imports (plain, brace, `_` and `*` wildcards) as relation facts. Also add scala to the extension strips in call-chain-tracer and import-repo cross-repo matching, mark Main.scala/App.scala as key files, and sync the skill-data language map and entry-file list. Fixes Tencent#981 Co-Authored-By: Claude Code <noreply@anthropic.com>
The PR description includes sufficient representative real-CLI verification for this runtime change. |
… review) Relation facts carried dotted package paths (com.foo.Bar), which neither buildCodeGraph's fuzzy file match nor the call-chain tracer's module map could match against slash-separated file paths — an internal Scala import produced no DEPENDS_ON edge. Emit the import target as a path (com/foo/Bar) instead; both consumers then resolve it (exact, /index and basename lookups all split on /). Add a regression test for the edge. Also make the main/server/app entry-file patterns case-insensitive so Scala's Main.scala / App.scala key files are selected as call-chain entry points. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Both findings are addressed in 0a33a3d:
tsc, oxlint and the affected suites (scala/swift/codebase-extract/import-repo/call-chain/wiki-engine) all pass. |
The two earlier findings are resolved: direct imports now use slash-separated paths, and Scala entry-file matching is case-insensitive. The PR description includes sufficient representative real-CLI verification. |
…t#996 review) A brace selector was discarded and only the package path emitted, so buildCodeGraph tied the importer to every file in the directory via substring matching while the call-chain tracer resolved nothing (no module is named after a package segment). Emit one relation per imported symbol — com.foo.{Bar, Baz => B} becomes com/foo/Bar and com/foo/Baz; renames take the name before =>, and the in-brace _ wildcard / nested selectors fall back to the package path, which is what a wildcard honestly names. Add regression coverage for both consumers: the graph edge test asserts a {Invoice} import links exactly Invoice.scala and not a sibling in the same package, and a traceCallChains test asserts the entry step resolves Invoice and never touches the sibling. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Addressed in 4aa533b: a brace selector now expands to one relation per imported symbol ( Regression coverage for both consumers, as requested:
Real-CLI re-verification (mixed fixture, tsc, oxlint and the scala/call-chain/wiki-engine suites pass. |
The two findings from the first review and the brace-import finding from the second review are resolved. The PR description includes sufficient representative real-CLI verification. |
A `{Invoice => _, _}` selector inverted its meaning: the hidden name got
its own relation (a dependency on the excluded file) while the wildcard's
package-wide import produced nothing. Parse the selector into a
wholePackage flag plus named symbols — `_` marks the package, an alias of
`_` hides its name, Scala 3 `as` renames like `=>` take the name before
the alias — and emit only the package path when the selector imports
wholesale.
Member imports (`com.foo.Bar.apply`) named a path no file has; trim the
lowercase tail after the last type name, since the file that defines the
type is what the consumers match.
Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All three addressed in df6231f:
Unit coverage: the import-forms test now includes Real-CLI re-verification (fixture exercising exactly these forms): tsc, oxlint and the scala suite (10 tests) pass. |
The earlier direct-import, capitalized entry-file, rename, and lowercase member-import findings are resolved. The PR description includes sufficient representative real-CLI verification. |
… imports A wildcard import still emitted the package path, so buildCodeGraph's substring match tied the importer to every file under the package — including a name the selector hides — and the call-chain tracer resolved nothing (no module is named after a package segment). Expand a wildcard over the collected files of the package instead, skipping hidden names, and keep the package path only when no collected file matches (an external package). Also parse the import forms the line-based scan mishandled: a brace selector wrapped across lines by scalafmt opened as a bare package import, and a comma-separated `import a.B, c.D` dropped every clause after the first. Track a wrapped selector until its `}`, and split a line into clauses (brace contents keep their own commas). Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All four addressed in e1b95f0:
Unit coverage: five new tests (expansion with hidden names, plain Real-CLI re-verification — fixture exercising all four forms: 8 relation facts — exactly one per import clause in the fixture, confirming the wrapped and comma-separated forms parse completely. |
|
Findings
Earlier direct-import normalization, capitalized entry matching, brace selectors, renames/member imports, multiline imports, and comma-separated imports are resolved. The PR description includes sufficient representative real-CLI verification. |
… files Four import gaps: - `*` never expanded: the clause regex dropped it before the wildcard check, and an in-brace `*` entry was not recognized as a wildcard at all — both treated as a plain package import whose substring match re-added names the selector hides. - Wildcard expansion saw only the Scala batch, so a package provided by Java files fell back to the package path (linking a hidden name), and Scala candidates masked Java ones. extractForLanguage now passes every collected file to the extractor. - A named import assumed a same-named file (`Invoice` in `Invoice.scala`) though Scala freely puts many types in one `Models.scala`. The symbol now resolves to the package file that declares it, falling back to the conventional path. - Wildcard membership matched descendant directories too, so a subpackage's files rode along on the parent package's import; direct package members only. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All four addressed in 13cbdf6:
Unit coverage: five new tests (Scala 3 Real-CLI re-verification (mixed Java/Scala fixture): Invoice.java receives no edge; traced call chains reach depth 3. |
|
Findings
The previously reported full-extraction issues are resolved. The PR description includes sufficient representative real-CLI verification. |
An incremental run re-extracts only changed files, so the extractor's resolution context went blind: wildcard expansion fell back to the package path (fuzzy-matching hidden names and subpackages back in) and named imports lost the file that declares the symbol. extractCodeFacts now takes the run's full file list plus the previous run's declarations (rebuilt from the cached facts — component facts carry name and file), so unchanged files resolve without being re-read. Also: hidden-name exclusion now consults the declarations, not just the file name — Models.scala declaring only a hidden Invoice is skipped too; wildcard membership is JVM-importable sources only, so a schema.sql beside the package does not become a dependency; a plain import always tries its last segment as a symbol, which resolves Scala 3 top-level defs (import com.demo.core.validate → Helpers.scala); and a file's several imports of the same target collapse to one relation. The MR diff filter in ci/extract-mr.ts now collects swift and scala as well. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All five addressed in bb480c1:
Unit coverage: five new tests (incremental context with stubs + cached declarations, declared-hidden exclusion, sql-resource exclusion, top-level def resolution, plus reworked cross-language/dedupe expectations) — 24 scala tests, 118 with the consumer suites, all passing. Real-CLI verification of the incremental path specifically — full extraction, then only Hidden |
|
Findings
The previously reported direct-import, entry-point, selector, multiline, Scala 3 wildcard, cross-language, subpackage, and CI-filter cases are resolved. The PR description contains sufficient representative real-CLI verification. |
A wildcard materializes to the package files present at extraction time, so an incremental run that only adds a package member left the unchanged importer's cached relations stale — no edge to the new file. Each wildcard now also emits a symbolic `scala-wildcard:<package>` relation (matched by no consumer) and the incremental pass re-extracts every importer whose wildcard package gained or lost a scala/java file. Nested defs are no longer package-level declarations: the declarations index tracks brace depth and column, so a class member `def total` does not rescue a file whose only top-level name the selector hides. Cached declarations rebuild from component/interface facts only — a config fact like `API_KEY` is not an importable symbol and used to restore dependencies on files the selector hid. A wildcard on an object (`import com.demo.Models.*`) now resolves to the file declaring the object when the package directory holds nothing, instead of falling back to a path no consumer matches. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All four addressed in b527575:
Unit coverage: 24 scala tests (three new: nested-def exclusion, object wildcard, and the marker expectations across the wildcard tests), 96 consumer-suite tests alongside — all passing. |
|
Findings
The prior direct-import, entry-point, selector, multiline, wildcard expansion, cross-language, subpackage, CI-filter, object-wildcard, and config-cache findings are resolved. The PR description includes sufficient representative real-CLI verification. |
A named import materializes to its current declaring file, so a declaration moving between files staled the unchanged importer's cached relation just like a package member change stales a wildcard. The incremental pass now re-extracts an importer whenever a cached materialized target (a relation name ending in .scala/.java) is among the changed or deleted files, alongside the existing wildcard-package rule. The declarations index no longer rebuilds from component facts, which cannot tell a nested member from a package-level name. Each Scala file now records its top-level names in a `scala-decl:` metadata relation, and the incremental context rebuilds from those markers alone. Top-level detection also learns the Scala 3 `package com.demo.core:` block: its members sit one indent in, and declarations deeper than the block's first indent level are members, not package names. The `scala-wildcard:`/`scala-decl:` metadata relations are now filtered from the relation evidence page, the gap detector, and the graph builder, so they no longer surface as undocumented dependencies. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
All four addressed in 379fcac:
Unit coverage: 25 scala tests (one new: package-block declarations, plus the marker expectations across the suite), 121 tests with the consumer suites, all passing; tsc and oxlint clean. |
|
Findings
The exact findings from earlier passes are resolved. The PR description includes sufficient representative real-CLI verification. |
Problem
codebase --extract/ repo import silently dropped every.scalafile, so Java/Scala mixed projects produced a Java-only code graph (#981):isCodeFile()did not whitelist.scala, andEXTRACTOR_REGISTRYhad noscalaentry.Changes
Follows the minimal plan from #981, modeled on the Swift heuristic extractor:
code-collector.ts): whitelist.scala, map it to thescalalanguage, markMain.scala/App.scalaas key files.extractors/scala.ts, registered inextractors/index.ts):class/object/enum/case class/case object→componentdef(past annotations and modifiers, incl.transparent inline def) →componenttrait→interfaceError/Exception→error(INFERRED)sys.env(...)/System.getenv(...)reads →configimportin all four forms (plain, brace selector,_wildcard, Scala 3*) →relation(normalized to the dotted path prefix)matchclause (case Invoice(id) =>) is not mistaken for acase classdeclarationscalaadded to the extension strips incall-chain-tracer.ts(entry patterns + import normalization) andimport-repo.tscross-repo import matching, alongside java/swift.kb-doc-generator.md, language map inscan_repo.py.Test plan
npx tsc --noEmit,npm run lintcleansrc/__tests__/scala-extractor.test.ts(6 tests: registry, declarations, modifiers/match-clause safety, all four import forms, error/config inference,.scalacollection with key-file marking)cross-repo-edgesGraph nodes include
component/DefaultGateway,component/Invoice,component/Main,component/PaymentError,component/charge,interface/PaymentGateway,error/PaymentError,config/PAYMENTS_API_KEY, andsource-manifest.jsonlists both.scalafiles. Before this change, only the.javafile was collected.Out of scope (follow-ups)
tree-sitter-scalaAST track for precise, type-driven edges.Closes #981
🤖 Generated with Claude Code