Skip to content

LLM review: ask for literal Unicode so the mathematics is readable - #291

Merged
Vilin97 merged 1 commit into
mainfrom
vilin/review-unicode-fix
Jul 28, 2026
Merged

Vilin97 merged 1 commit into
mainfrom
vilin/review-unicode-fix

Conversation

@Vilin97

@Vilin97 Vilin97 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

…ol/pull/289)) came back with the mathematics unreadable:

> elliptic curves over `^R09` … `|(discr K : ^])| ^U 8.25 ^ finrank ^R0a K`

where it should read `over ℚ` and `|(discr K : ℝ)| ≥ 8.25 ^ finrank ℚ K`.

The mangling is the model's own: nothing between `json.loads` and the posted comment touches those strings, the bytes in the stored comment are literal ASCII, and Greptile's comment through the same fetch path renders `ℚ` fine. It is also not a systematic escape — `ℚ` came out as `^R0a` in three places and `^R09` in another — so it cannot be decoded after the fact, only prevented.

So: one notation rule, appended to every system prompt, asking for the characters as they appear in the diff and offering a graceful fallback (describe the symbol in words) rather than an approximation. A review of Lean that cannot render `ℚ` or `≥` is worth much less in this repository.

Tested by asserting the rule reaches the model for every mode in `REVIEW_MODES`, so a new review kind cannot quietly skip it. Verified for real by re-running `/review` on #289 after this merges.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
BODY
LLM review: ask for literal Unicode so the mathematics is readable

The first challenge review (#289) came back with the mathematics
unreadable: `over ^R09` for `over ℚ`, and
`|(discr K : ^])| ^U 8.25 ^ finrank ^R0a K` for
`|(discr K : ℝ)| ≥ 8.25 ^ finrank ℚ K`.

The mangling is the model's own — nothing between `json.loads` and the
posted comment touches those strings, the stored bytes are literal ASCII,
and another bot's comment through the same fetch path renders `ℚ` fine.
It is not a systematic escape either (`ℚ` came out as both `^R0a` and
`^R09`), so it cannot be decoded after the fact, only prevented.

Append one notation rule to every system prompt: copy the characters as
they appear in the diff, and if a symbol cannot be reproduced, describe
it in words rather than approximate it. A test asserts the rule reaches
the model for every mode in REVIEW_MODES.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown

Greptile Summary

This PR appends a NOTATION_RULE constant to every LLM system prompt in request_review, instructing the model to reproduce Lean Unicode symbols (ℚ, ℝ, ≥, etc.) verbatim rather than mangling them. A companion test verifies the rule is injected for all REVIEW_MODES.

  • review.py: Adds NOTATION_RULE (with a clear inline comment explaining the regression in PR Challenges: Odlyzko root-discriminant bound and Mazur's torsion bound #289) and appends it to system_prompt before the token-budget calculation, so the enlarged prompt is correctly accounted for in scaffold_tokens.
  • test_review.py: Adds test_every_review_mode_asks_for_literal_unicode, which stubs the OpenAI client, iterates over all review modes, and asserts both the key phrase and a sample Unicode character are present in the transmitted system message.

Confidence Score: 5/5

The change is a narrow, targeted addition: one new string constant appended to every system prompt, with correct token-budget accounting and a well-scoped test.

Both changed files are straightforward. The notation rule is appended before scaffold_tokens is computed, so the enlarged prompt is correctly reflected in the diff budget. The test stubs the OpenAI client and exercises all review modes, giving good confidence the rule will reach the model in production.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
python/lean_pool/review.py Adds NOTATION_RULE constant and appends it to the system prompt before token budgeting — straightforward and correct.
python/tests/test_review.py Adds a test that stubs OpenAI, exercises every REVIEW_MODE, and checks the notation rule reaches the system message — good coverage.

Reviews (1): Last reviewed commit: "The first challenge review ([#289](https..." | Re-trigger Greptile

@Vilin97
Vilin97 merged commit 20a9995 into main Jul 28, 2026
9 checks passed
@Vilin97
Vilin97 deleted the vilin/review-unicode-fix branch July 28, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant