Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/designs/data-directory-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -681,6 +681,10 @@ Those builds, and the reports checkout `teamai recall feedback --negative` count
the team's upvotes from, take the votes directory from `indexableVotesDir`,
which runs the same probe on the reports checkout: another repository's votes
are left out, so no learning of this project carries that team's hotness.
With `recall feedback --dry-run`, config resolution stays read-only and the
command returns after previewing the requested feedback, before any vote read,
migration, lock, or reports-checkout probe. No vote count is predicted; ordinary
diagnostic logging still applies.
The recall hook's vote judge, which starts no git process, reads the same answer
from git's files: the checkout's `.git` file names its gitdir, whose `commondir`
leads to the owning repository's git dir, compared (realpath'd) with this
Expand Down
2 changes: 2 additions & 0 deletions docs/usage-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -1350,6 +1350,8 @@ teamai recall "GPU out of memory"

### Recall adoption and upvotes

Manual feedback uses `teamai recall feedback --positive <docId>` or `--negative <docId>` in the current scope. Add the global `--dry-run` before or after the command to preview the requested feedback without changing votes or migrating config or vote files. The preview validates the scope's config but does not check whether a negative vote can reduce the count; ordinary diagnostic logging still applies.

Recall counts every doc it returns (`recalled_count`). A returned doc is **adopted**, and upvoted once (`upvoted_count`), when the session that ran the recall opens it within 24 hours after the run. Adoption means opening the doc: when the `teamai-recall` subagent summarizes a doc and the main agent works from that summary alone, nothing is opened and no vote follows. Only the opt-in judge (`TEAMAI_UPVOTE_JUDGE=1`, see [Enabling / Disabling Recall](#enabling--disabling-recall)) can credit that use.

**The recall log.** Each run goes to the active scope's local recall log, `<data home>/dashboard/recall.jsonl`, which is owner-only and never pushed. The run holds the agent session from the environment and, for each returned doc, its id, scope and printed `File:` path; a search with no hits is recorded too. The PostToolUse hook adds the shell call that ran `teamai recall` and each read of a file under the team knowledge roots. The log never holds the query, the prompt, tool output or file content. `teamai pull` prunes it: lines older than 30 days go, then the oldest beyond 5,000, but never a read from the last 24 hours that has not voted yet, nor what it needs to vote. `--check`, `--dry-run` and `TEAMAI_RECALL_DISABLED=1` record nothing.
Expand Down
2 changes: 2 additions & 0 deletions docs/usage-guide.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1228,6 +1228,8 @@ teamai recall "GPU 内存不足"

### Recall 采纳与 upvote

手动反馈使用 `teamai recall feedback --positive <docId>` 或 `--negative <docId>`,作用于当前 scope。在命令前后添加全局参数 `--dry-run`,只预览请求的反馈,不修改投票,也不迁移配置或投票文件。预览会验证当前 scope 的配置,但不检查负面反馈能否减少票数;普通诊断日志仍会记录。

recall 会为返回的每篇文档计数(`recalled_count`)。运行 recall 的会话在 run 之后 24 小时内打开某篇返回的文档,该文档即被**采纳**,并获得一次 upvote(`upvoted_count`)。采纳指打开文档:如果 `teamai-recall` subagent 总结了某篇文档,而主 agent 只依据这段总结工作,就没有打开任何文档,也不会投票。只有可选开启的评判(`TEAMAI_UPVOTE_JUDGE=1`,见[开启 / 关闭 Recall](#开启--关闭-recall))能为这种使用计分。

**recall 日志。** 每次 run 都写入当前 scope 的本地 recall 日志 `<data home>/dashboard/recall.jsonl`,该日志仅所有者可读写,从不推送。run 记录环境中的 agent 会话,以及每篇返回文档的 id、scope 和打印出的 `File:` 路径;没有命中的搜索也会记录。PostToolUse hook 追加运行 `teamai recall` 的 shell 调用,以及每次读取团队知识根目录下文件的调用。日志从不包含查询词、prompt、工具输出或文件内容。`teamai pull` 会清理日志:先删除超过 30 天的行,再从最旧的开始删到只剩 5,000 行,但从不删除最近 24 小时内尚未投票的读取,也不删除它投票所需的行。`--check`、`--dry-run` 和 `TEAMAI_RECALL_DISABLED=1` 不记录任何内容。
Expand Down
148 changes: 148 additions & 0 deletions src/__tests__/e2e/recall-feedback-dry-run.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { execFileSync, spawnSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import YAML from 'yaml';
import { projectSlug } from '../../utils/partition.js';
import type { UserVotesV2 } from '../../types.js';

const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..', '..');
const cli = path.join(root, 'dist', 'index.js');
let sandbox: string;
let home: string;
let project: string;

function writeYaml(file: string, value: unknown): void {
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, YAML.stringify(value));
}

function setupScope(scope: 'user' | 'project'): { config: string; votes: string } {
const dataHome = scope === 'user'
? path.join(home, '.teamai')
: path.join(home, '.teamai', 'projects', projectSlug(project));
const repo = path.join(dataHome, 'team-repo');
const config = path.join(dataHome, 'config.yaml');
const votes = path.join(dataHome, scope === 'user' ? 'user-votes' : 'votes', 'tester.yaml');
writeYaml(config, {
repo: { localPath: repo, remote: 'https://example.test/feedback.git' },
username: 'tester', scope, additionalRoles: [],
...(scope === 'project' ? { projectRoot: project } : {}),
});
// A real load migrates this legacy role selection; a preview must not.
writeYaml(path.join(repo, 'manifest', 'roles.yaml'), {
version: 1, roles: [{ id: 'hai', resources: { skills: [], knowledge: [] } }],
});
writeYaml(votes, {
version: 2,
votes: { doc123: { recalled_count: 3, upvoted_count: 2 } },
deltas: {},
});
return { config, votes };
}

function run(args: string[], cwd = sandbox) {
const result = spawnSync(process.execPath, [cli, ...args], {
cwd,
env: { ...process.env, HOME: home, USERPROFILE: home, NO_COLOR: '1' },
encoding: 'utf8', timeout: 10_000,
});
if (result.error) throw result.error;
return { code: result.status, output: result.stdout + result.stderr };
}

/** Include directory creation, but not the CLI's ordinary diagnostic log. */
function snapshot(dir = sandbox): Record<string, string> {
const entries: Record<string, string> = {};
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const file = path.join(dir, entry.name);
if (file === path.join(home, '.teamai', 'debug.log')) continue;
const relative = path.relative(sandbox, file);
if (entry.isDirectory()) {
entries[relative] = '<directory>';
Object.assign(entries, snapshot(file));
} else {
entries[relative] = fs.readFileSync(file).toString('base64');
}
}
return entries;
}

beforeEach(() => {
if (!fs.existsSync(cli)) throw new Error('Run npm run build before this test.');
sandbox = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-feedback-preview-')));
home = path.join(sandbox, 'home');
project = path.join(sandbox, 'project');
fs.mkdirSync(home);
fs.mkdirSync(project);
execFileSync('git', ['init', '-q', project], { stdio: 'pipe' });
});

afterEach(() => {
if (sandbox) fs.rmSync(sandbox, { recursive: true, force: true });
});

describe.each(['user', 'project'] as const)('recall feedback --dry-run in %s scope', (scope) => {
it.each([
['positive', 'before'], ['positive', 'after'],
['negative', 'before'], ['negative', 'after'],
])('previews %s feedback with the flag %s the command without writes', (polarity, position) => {
setupScope('user');
if (scope === 'project') setupScope('project');
const before = snapshot();
const args = ['recall', 'feedback', `--${polarity}`, 'doc123'];
if (position === 'before') args.unshift('--dry-run');
else args.push('--dry-run');

const result = run(args, scope === 'project' ? project : sandbox);
expect(result.code, result.output).toBe(0);
expect(result.output).toContain(`[dry-run] Would submit ${polarity} feedback for: doc123`);
expect(snapshot()).toEqual(before);
});

it('still records real feedback in the selected scope', () => {
const user = setupScope('user');
const selected = scope === 'project' ? setupScope('project') : user;
const userBefore = fs.readFileSync(user.votes, 'utf8');
const cwd = scope === 'project' ? project : sandbox;
for (const [polarity, count] of [['positive', 3], ['negative', 2]] as const) {
const result = run(['recall', 'feedback', `--${polarity}`, 'doc123'], cwd);
expect(result.code, result.output).toBe(0);
expect(result.output).not.toContain('[dry-run]');
const votes = YAML.parse(fs.readFileSync(selected.votes, 'utf8')) as UserVotesV2;
expect(votes.votes.doc123.upvoted_count).toBe(count);
expect(votes.deltas.doc123.upvoted_delta).toBe(count - 2);
}
if (scope === 'project') expect(fs.readFileSync(user.votes, 'utf8')).toBe(userBefore);
});
});

describe('feedback preview storage and validation', () => {
it.each(['positive', 'negative'])('does not migrate old votes or create missing votes for %s feedback', (polarity) => {
const { votes } = setupScope('user');
writeYaml(votes, { votes: { doc123: { at: '2026-10-01T00:00:00Z' } } });
for (const existing of [true, false]) {
if (!existing) fs.rmSync(path.dirname(votes), { recursive: true });
const before = snapshot();
const result = run(['recall', 'feedback', `--${polarity}`, 'doc123', '--dry-run']);
expect(result.code, result.output).toBe(0);
expect(result.output).toContain(`[dry-run] Would submit ${polarity} feedback for: doc123`);
expect(snapshot()).toEqual(before);
}
});

it('refuses an unreadable project config instead of using the user scope', () => {
setupScope('user');
const { config } = setupScope('project');
fs.writeFileSync(config, 'repo: [unclosed\n');
const before = snapshot();
const result = run(['recall', 'feedback', '--positive', 'doc123', '--dry-run'], project);
expect(result.code).toBe(1);
expect(result.output).toContain('No feedback recorded:');
expect(result.output).toContain(config);
expect(result.output).not.toContain('Would submit');
expect(snapshot()).toEqual(before);
});
});
3 changes: 2 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1280,7 +1280,8 @@ recallCmd
.option('--negative <docId>', 'Record negative signal for a document')
.action(async (cmdOpts) => {
const { recallFeedback } = await import('./votes.js');
await recallFeedback({ positive: cmdOpts.positive, negative: cmdOpts.negative });
const globalOpts = program.opts() as GlobalOptions;
await recallFeedback({ positive: cmdOpts.positive, negative: cmdOpts.negative, dryRun: globalOpts.dryRun });
});

recallCmd
Expand Down
12 changes: 9 additions & 3 deletions src/votes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -432,13 +432,14 @@ export async function syncVotesToTeam(
/**
* Record manual feedback for a recalled document.
*/
export async function recallFeedback(opts: { positive?: string; negative?: string }): Promise<void> {
export async function recallFeedback(opts: { positive?: string; negative?: string; dryRun?: boolean }): Promise<void> {
const { resolveConfigForDir, findUnreadableProjectConfig, throwMissingOrInvalid, BROKEN_CONFIG_ADVICE } = await import('./config.js');
// The votes of the cwd's scope (#787). An unreadable project config falls
// back to no other scope: the feedback would reach that scope's team.
const localConfig = await resolveConfigForDir();
const loadOptions = { dryRun: opts.dryRun };
const localConfig = await resolveConfigForDir(undefined, undefined, loadOptions);
if (!localConfig) {
const unreadable = await findUnreadableProjectConfig();
const unreadable = await findUnreadableProjectConfig(undefined, loadOptions);
let reason: string;
if (unreadable) {
const { firstLine } = await import('./skill-content.js');
Expand All @@ -453,6 +454,11 @@ export async function recallFeedback(opts: { positive?: string; negative?: strin
process.exitCode = 1;
return;
}
if (opts.dryRun && (opts.positive || opts.negative)) {
const polarity = opts.positive ? 'positive' : 'negative';
log.info(`[dry-run] Would submit ${polarity} feedback for: ${opts.positive || opts.negative}`);
return;
}
const { getVotesDir } = await import('./types.js');
const votePath = path.join(getVotesDir(localConfig), `${localConfig.username}.yaml`);

Expand Down
Loading