Send generated functions to the line that declared them

.github/workflows/ci.yml

@@ -73,12 +73,12 @@ jobs:
           cd internal/lsp/testdata/monorepo/apps/app_with_styler && mix deps.get && mix deps.compile && cd -
           cd internal/lsp/testdata/monorepo/apps/app_with_ecto_migration && mix deps.get && mix deps.compile && cd -
           cd internal/lsp/testdata/monorepo/apps/app_basic && mix deps.get && cd -
-      # Every formatter test and the generated-BEAM completion test need mix on
-      # PATH and skip themselves without it, so this job is the only place they
-      # run — the test/test-race jobs have no Elixir. Match the formatter family
-      # and explicitly include the self-contained BEAM regression fixture.
+      # Every formatter test and the generated-BEAM completion and definition
+      # tests need mix on PATH and skip themselves without it, so this job is the
+      # only place they run — the test/test-race jobs have no Elixir. Match the
+      # formatter family and explicitly include the self-contained BEAM fixtures.
       - name: Run integration tests
-        run: go test ./internal/lsp/ -run 'TestFormatter|TestDidSave_Formatter|TestCompletion_GeneratedFunctionsFromConsumerBEAM' -v -timeout 600s
+        run: go test ./internal/lsp/ -run 'TestFormatter|TestDidSave_Formatter|TestCompletion_GeneratedFunctionsFromConsumerBEAM|TestDefinition_GeneratedFunctionsFromCompiler' -v -timeout 600s
 
   lint:
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -4,6 +4,8 @@
 
 ### Added
 
+- **Go-to-definition reaches the line that declared a generated function** — a function a macro generated used to resolve to the top of its module. Dexter now reads the line from the compiled module's debug info, which is standard compiler output, so no framework is special-cased. A generator that expands each function at the line of the call that declared it, or stamps it with `@file {file, line}`, sends definition, call hierarchy, the references declaration, and `dexter lookup` (including `--strict`) to that line. The line is used only when it was compiled from the file being opened; a BEAM older than the source still gives its line, because Dexter cannot compile the project and the last compile's line is closer than the module line; only edits to the declaring file move it, and the next compile makes it exact. A function whose only recorded line is the module line, such as one a `@before_compile` hook made, goes to the call in its module that declares it by name. When several calls spell the name, as an Ash action and the code interface that runs it do, the macro whose calls name the most of the module's generated functions wins, and a tie keeps the module line. A function with a clause per DSL call goes to every clause. A line past the end of the file is never returned. A module compiled without debug info falls back to its Docs chunk annotation, which is often, but not always, the same line. A generated module with no source of its own, such as one `Module.create` made or a Spark DSL entity, goes to the file it was compiled from, rebased onto the project when it was built elsewhere, and so does go-to-definition on its name. A module a macro made with `defmodule` and a name it computed records no line of its own, so its name goes to its first function's line. A bare call to a generated function of an imported module now resolves. Ash code interfaces go to their `define` line with released Ash, and from the recorded line with an Ash release that includes [ash-project/ash#2971](https://github.com/ash-project/ash/pull/2971) ([#108](https://github.com/remoteoss/dexter/issues/108))
+
 - **`--root`/`-C` names the workspace on every command** — `dexter init --root ~/project`, `dexter lookup --root ~/project MyApp.Repo`, and `dexter lsp --root ~/project` all run as if they had been started in that directory, so an agent, a script, or an editor wrapper can index or query a project from anywhere. Relative paths, a `reindex` target included, resolve from the named root
 
 - **Watching and runtime locations recover without periodic reindexes** — a directory the kernel refuses to watch (an inotify watch limit on a large tree) is tracked instead of disabling the whole native watcher. Dexter reconciles once when coverage is lost, retries only failed registrations, and reconciles once when coverage returns; it does not run recurring full-tree passes that cause CPU spikes. Runtime files use the environment-independent `/tmp/dexter-<uid>` directory so a GUI editor and a shell cannot derive different ownership locks. Elixir and mix detection also searches the standard mise, asdf, and Homebrew locations and falls back to a login shell, so an editor that starts the daemon with a stripped PATH no longer disables stdlib indexing or formatting
@@ -26,6 +28,10 @@
 
 ### Fixed
 
+- **A def inside a macro's `quote` is no longer indexed as a function of the macro's module** — `defmacro route(...) do quote do def handle(...) end end` made the index say that the DSL module defines `handle/2`, which it does not. A consumer that imports the DSL then resolved `Consumer.handle` into the macro's body. Such a def is now skipped, so the call goes to the line in the consumer that declared it, from the compiled BEAM. What `__using__` injects, and a quote in a helper function, are still indexed as before. The index is rebuilt once after the upgrade
+
+- **Compressed BEAM files are read** — a module compiled with the `compressed` option, as some Erlang dependencies are, is a gzip stream around the BEAM container. Dexter rejected it as an invalid BEAM, so its exports were missing from completion and generated-function navigation. It is now decompressed, with the same size limit as an uncompressed file
+
 - **Git worktrees nested inside a project were indexed as part of it** — a linked worktree checked out below the project root (Claude Code's `.claude/worktrees/`, or any gitignored worktree folder) is a full copy of the repository, so every go-to-definition returned one result per checkout. The indexer, the incremental sweep, the native watcher and single-file updates now skip any directory that is a linked worktree, including worktrees of bare repositories; submodules and Mix git dependencies stay indexed. Adding, changing, moving or removing such a worktree, including its `mix.exs` and `mix.lock`, no longer reindexes the enclosing project, and an existing index drops the worktree's files at the next start. A worktree that git still records stays skipped after its `.git` file is gone, as during `git worktree remove`, until `git worktree prune`. Starting Dexter inside such a worktree no longer climbs to the enclosing checkout's `.dexter/dexter.db` either: the worktree is its own project
 - **Branch switches were missed when the project root is a linked worktree or a submodule** — the HEAD poller read `<root>/.git/HEAD`, but in such a checkout `.git` is a file that names the git directory. The poller now follows that file, so a branch switch reconciles the index even when the native file watcher is unavailable
 - **Find references through an injected alias was slow and memory-hungry on large projects** — a module such as `MyApp.Repo`, aliased by a `__using__` block that most of the project uses, made every references query read and tokenize each file that used the injector: about 20,000 files and 2.4 GB of allocations per query on a large monorepo. Only files that contain a candidate reference are read now, which brought that query from 1–2 s to under 0.3 s and its allocations to about 70 MB

docs/architecture.md

@@ -7,7 +7,7 @@ Dexter is a fast Elixir LSP server. It indexes module and function definitions f
 - `cmd/main.go` — CLI entrypoint: `init`, `reindex`, `lookup`, `references`, `lsp`, `version`, and the hidden `daemon` subcommand. `lsp`, `lookup`, `references`, and `reindex` are frontends to the workspace daemon: `lsp` is a raw stdio proxy onto it. Only `init` still opens the store itself, under the same ownership lock the daemon takes, because it needs the workspace to itself.
 - `internal/indexer/` — the cold build: walk and stat on all cores, parse on all cores, then one bulk transaction with the indexes dropped. `dexter init` and the LSP server (when it finds an empty index) both call `FullBuild`. `Options.InProcess` marks the server, which shares the database with live readers and so cannot use the connection-wide bulk pragmas.
 - `internal/parser/` — Elixir parser backed by a hand-rolled tokenizer (`tokenizer.go`). The tokenizer produces a flat token stream (handling heredocs, sigils, strings, comments as opaque tokens; the code inside a `#{}` interpolation is tokenized as well, into the separate `TokenResult.Interp` stream — see below) and `parser_tokenized.go` walks it to extract defmodule, def, defp, defmacro, defdelegate, defguard, defprotocol, defimpl, @type, @callback, alias, import, use, and Module.function references. Handles module nesting, alias resolution for defdelegate targets, and multi-line expressions natively via bracket depth tracking.
-- `internal/beam/` — bounded, bounds-checked readers for the BEAM container, export table, Elixir Docs chunk, and persisted module attributes. These readers extract generated callables and DSL-provider metadata without starting the Erlang VM.
+- `internal/beam/` — bounded, bounds-checked readers for the BEAM container, export table, Elixir Docs chunk, Elixir debug info, and persisted module attributes. These readers extract generated callables, their definition lines, and DSL-provider metadata without starting the Erlang VM.
 - `internal/store/` — SQLite layer. Tables: `files` (path + mtime), `definitions` (module, function, kind, line, file_path, delegate_to, delegate_as), `refs` (module, function, line, file_path, kind).
 - `internal/lsp/` — LSP server. `server.go` handles all LSP methods. `elixir.go` contains pure functions for cursor expression extraction, alias/import/use extraction (tokenizer-based), and use-chain parsing. `rename.go` has rename helpers. `hover.go` has hover formatting. `documents.go` is an in-memory open-buffer store.
 - `internal/workspace/` — the protocol-independent owner of one workspace: the store, the shared `lsp.IndexCoordinator`, the headless language-service instance, stdlib discovery, the native and Git watchers, and the single mutation queue every index change enters. `runtime.go` is the lifecycle and the queue; `watch.go` defines the recursive watcher abstraction, with an FSEvents backend on macOS and an fsnotify backend elsewhere or as fallback.
@@ -137,6 +137,26 @@ The block walk is passed as a thunk and runs only after the compiled consumer re
 
 Generated-symbol resolution is shared by completion, hover, definition, signature help, references, and call-hierarchy preparation. Definition and call hierarchy cannot point at a source definition for a source-less provider, so they walk the provider's lexical module parents and use the closest module that has an indexed source location. Completion resolve and signature help read the same lazily cached BEAM documentation used by hover.
 
+### Definition lines for generated functions
+
+A generated function has no source definition, but the compiled module's `Dbgi` chunk still records where each def came from. `beam.ReadDefinitionLines` walks `{:debug_info_v1, :elixir_erl, {:elixir_v1, map, specs}}` for the `file`, `relative_file`, and `definitions` keys, and steps over clause bodies without allocating. Clause ASTs can nest very deep, so the ETF reader steps over a term with a count of the terms still to skip instead of recursion: nesting costs no stack, and a count larger than the bytes left is rejected as corrupt.
+
+Each def carries two locations. Its `:line` is the line in the module's own file that was being compiled when the def was produced: the macro call, or wherever a before-compile hook ran. A `file: {path, line}` entry, left by `@file` or `quote location: :keep`, is preferred when `path` is the module's own source, because that is where the code asked for the function. When `path` is another file it is the generator's implementation, and `:line` is used instead. A `file:` line at or before the module's own line is not used either: it is a `location: :keep` quote in a macro defined above the module in the same file. Elixir 1.17 and earlier record the module's line under the map's `line` key instead of `anno`, and both are read. Ash's code interfaces take the second shape: each generated def is expanded at its `define` line, so `:line` is that line and `file` still names `code_interface.ex`, which keeps stacktraces unchanged. No framework is recognized by name.
+
+The parser does not index a def inside a `quote` in a macro other than `__using__` (`defmacro route ... do quote do def handle ... end end`) as a function of the macro's module: it is code the macro generates in its caller, and the caller's BEAM records where it was declared. Without that, a consumer that imports the DSL resolved `Consumer.handle` through the import into the macro's body before the BEAM was asked. What `__using__` injects and a quote in a helper function stay indexed, because the use-chain features above resolve them.
+
+`generatedDefinitionResultsFor` is the single entry point that definition, call-hierarchy preparation, and the references declaration share; qualified definition, the references declaration, and `dexter lookup` reach it through `LookupName`. It starts from `generatedDefinitionResults` and moves to the recorded line only when that line belongs to the file being opened: the debug info must have been compiled from that file (the same absolute path, or else the most path components shared from the end, which covers a moved, copied, or symlinked project; when a module has rows in two files, as two umbrella apps can, the most specific match wins, and an equal match trusts neither), and the line must fall after the module's own line. Any other case keeps the module result. As for generated names, source-vs-BEAM mtime is not a gate: Dexter cannot compile the project, so a stale BEAM is the usual state while editing, and the last compile's line is closer than the module line.
+
+`beam.ReadDefinitionLines` also reads the module's own line (the map's `anno`, a line or `{line, column}`) and the line of each clause when a def's clauses were made at different lines (`DebugInfo.Clauses`: a DSL that adds a clause per call, such as `route :get, "/a"`, gets a location per clause).
+
+Lines are checked against the current text, which is the buffer open in the editor if there is one and the file on disk otherwise (`currentSource`). A line at or before the module's line carries no information and is dropped; both lines come from the same compile (the module's line is Dbgi's `anno` of the BEAM that holds the module, which is the parent's for a module a macro nested in it, and the index's only when no BEAM records one), so the comparison holds even when the file has changed since. A line past the end of the current text is dropped too, because a generator can give a def any line (`quote line: 99`) and an edit can remove lines. When nothing is left, which is the case for a def a `@before_compile` hook made at the module line (and for released Ash's code interfaces), the owning module's body is searched for the declaring call: a call, outside heredocs, whose first argument is the function's name as an atom, also trying the name without a trailing `!` or `?`. That is the generic shape of a macro call that declares a name; an atom in a keyword, a typespec, a later argument, or a module attribute does not count, and neither does a call in a sibling module of the same file (found with the tokenizer). When one line matches, it is used. When several do, as an Ash action (`update :publish`) and the code interface that runs it (`define :publish`) do, the macro that made the function is taken to be the one whose calls name the most of the module's generated functions (`define` names every interface; an action names only itself), and a tie gives no answer. The search is not used for a function whose clauses were made at several lines: those came from calls that do not spell its name (`get "/a"` makes a `match/2` clause), so a call that does, such as `plug :match`, is not where they were declared.
+
+A stale BEAM's lines are not corrected. Only edits to the declaring file move them, by the lines added or removed above, and those files (DSL modules, schemas, routers, dependencies) change less often than their callers. Correcting them meant guessing at text the BEAM was compiled from, and every signal tried either gave up often or could send the editor to a line that looked meaningful and was not. The line from the last compile is near, and the next compile makes it exact.
+
+A module compiled without debug info still has its Docs chunk, whose per-entry anno (`beam.Function.Line`) is often the same `:line`, but not always: a generator can give the def one line and its docs another, as Ash's code interfaces do. It is used when there is no Dbgi, and the `CInf` chunk's `:source` (`beam.ReadSourcePath`) decides which file it belongs to. The `CInf` source also covers a generated module with no source row of its own, such as a Spark entity module: its recorded file is usually the generator's own file. A recorded path inside the project is used as it is. Otherwise it is rebased: first each tail of the recorded path under the project root, longest first, which covers a moved or copied checkout and an umbrella's `apps/<app>/lib`, then the Mix layouts for the application named below the last `lib/` (`lib/<app>`, `deps/<app>/lib/<app>`, or `deps/<app>`). The application comes from the recorded path, not the BEAM's, because Spark creates Ash's entity modules in ash's ebin from spark's source. A recorded path outside the project is the last choice: it is right for a `path:` dependency, but a `_build` copied from another worktree records that worktree, which usually still exists. If no file is found, the lexical parent stays the answer. `CInf` records the path as codepoints, which are decoded to UTF-8. A precise line is the function's own definition, so `LookupName` returns it even for a strict (`ExactModule`) lookup; without one, the existing module results stand. `LookupName` on the name of a module that has no source row returns the file from `CInf` at the module's recorded line, or at its first function's line when the module line is zero, as it is for a module a macro made with `defmodule unquote(name)` and a name it computed, so go-to-definition on a module that `Module.create` made works too. A bare call that no source resolves also checks the BEAMs of the file's imported modules for a generated function of that name.
+
+The debug info and compile info are read on the first definition request that needs them and memoized on the module's generated-function cache entry, so the BEAM stamp invalidates them with everything else. A module compiled without debug info, or an Erlang module, is memoized as empty.
+
 For references, the parser records bare injected calls under the direct `use` module because the generated provider is unavailable while source is indexed. At lookup time, generated-symbol resolution queries those injector rows and validates every candidate against the compiled provider active at that candidate's block path. This keeps same-named macros from another DSL or another section out of the result. Statement-level injected calls with arguments are indexed even when they omit both parentheses and a `do` block, as in `authorize_if always()`.
 
 ## References — injector scan