Skip to content

Fix: recall a stale procedure on an exact path or error match without embeddings - #195

Open
aayank13 wants to merge 1 commit into
mainfrom
fix/stale-recall-without-embeddings
Open

aayank13 wants to merge 1 commit into
mainfrom
fix/stale-recall-without-embeddings

Conversation

@aayank13

Copy link
Copy Markdown
Owner

Fixes #163

A procedure that goes 30 days without use turns stale, and recall then asks for a ranked semantic score of at least 0.6 on top of the exact and lexical evidence. Without the embedding model, which is the default, that score is always 0, so a stale procedure failed the gate on every prompt however exact the match. It could not become active again through use, and after 90 days it was archived. With a pagination procedure learned for src/paginate.js and left idle for 40 days, trodden recall "Page 2 of the listing repeats the last item. Fix the pagination in src/paginate.js." showed it as the only candidate, top on lexical and matching the path, and still said "Nothing to inject: the best match was not close enough."

When the procedure has no cosine at all, the stale gate now replaces the cosine threshold with stricter lexical and exact evidence instead: the procedure must still be top on lexical rank with at least 2 of exact weight, as before, and on top of that one of the exact matches has to be a file path or an error signature, and the prompt must not be of a different task kind or about a different artifact than the procedure. So an active procedure without embeddings can still be recalled on a symbol plus a command, but a stale one needs the path or the error itself. The envelope already tells the agent that the procedure has not been used in over a month, and once it is served and settled it becomes active again.

The embedding path is unchanged. Whenever a cosine exists for the procedure, stale recall still needs it ranked in the semantic top ten at 0.6 or more, so the calibration for installs with embeddings stays as it was. The ambiguity check after the gate also still applies, so a stale procedure tied with another family abstains as before.

To reproduce, I built the binary, seeded a scratch TRODDEN_HOME with the pagination procedure scoped to a scratch git repository, backdated its last use by 40 days and ingested a session so aging marked it stale. Before the change, the exact prompt above abstained. After it, the same prompt injects the procedure with the "not used in over a month" line, while "Run npm test after you fix paginate so page 2 stops repeating." (a symbol and the check, but no path) still abstains as not confident, and an unrelated prompt matches nothing.

New tests recall a stale procedure on an exact path without embeddings, and fail on the old gate. Guard tests compare an active and a stale copy of the same procedure: the symbol-plus-command prompt recalls the active one but not the stale one, and an add prompt on the same file, an unrelated prompt and a prompt about another feature recall nothing stale. Two more tests pin the embedding path: a stale procedure is confident only from a ranked cosine of 0.6, never when it is outside the semantic ranking, and a far prompt with embeddings still abstains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Without embeddings, a procedure unused for 30 days can never be recalled again

1 participant