From db2f88955abf55022596b0906af3745af5f7e8df Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Tue, 8 Sep 2026 19:49:52 -0700 Subject: [PATCH 1/3] Add a temporary probe file to measure the global Qodo settings repository Qodo documents the global pr-agent-settings repository as an organization-level feature, and this account is a personal one, so whether it is honored has to be observed rather than read. This file carries the name prefix the global best practices reserve, and the review this pull request draws is the measurement. The pull request is closed unmerged once the review lands. Measured for ptr727/ProjectTemplate#1321. Co-Authored-By: Claude Opus 5 (1M context) --- qodo-global-probe.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 qodo-global-probe.md diff --git a/qodo-global-probe.md b/qodo-global-probe.md new file mode 100644 index 00000000..76c3dcc3 --- /dev/null +++ b/qodo-global-probe.md @@ -0,0 +1,20 @@ +# Qodo Global Configuration Probe + +This file exists only to measure whether Qodo honors a global `pr-agent-settings` repository under +a personal account rather than an organization. Qodo's documentation says the repository must live +in an organization, and this account is a personal one, so the behavior is undocumented and has to +be observed. + +The measurement reads two independent signals from the review this pull request draws. + +- sets `use_images_and_animations = false`. Qodo + applies that itself rather than asking the model to comply, so the divider and severity images + disappearing from the review is the structural signal. +- The same repository's `best_practices.md` reserves the `qodo-global-probe` file name prefix, so a + reported rule violation naming that rule is the second signal. This file carries that prefix. + +The two signals test different halves of the same claim and can disagree, which is why both are +read. + +This pull request is closed unmerged once the review lands. Nothing here is intended to be kept, +and the measurement is recorded on . From f0d7649573ceb22043f22b09afa619c3ccb96a89 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Tue, 8 Sep 2026 19:59:01 -0700 Subject: [PATCH 2/3] Re-trigger the review with the image setting at its default The first arm observed no images, which the setting predicts, but Qodo changed its review layout in the same window and one observation cannot separate the two causes. This commit draws a second review while the global setting reads true. Co-Authored-By: Claude Opus 5 (1M context) --- qodo-global-probe.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/qodo-global-probe.md b/qodo-global-probe.md index 76c3dcc3..8cb1445b 100644 --- a/qodo-global-probe.md +++ b/qodo-global-probe.md @@ -16,5 +16,13 @@ The measurement reads two independent signals from the review this pull request The two signals test different halves of the same claim and can disagree, which is why both are read. -This pull request is closed unmerged once the review lands. Nothing here is intended to be kept, +This pull request is closed unmerged once the review lands. + +## Control arm + +The first review carried no images, which is what the setting predicts, but Qodo changed its own +review layout in the same two-day window, so that one observation cannot separate the setting being +honored from a product change. The setting is now flipped to its default of `true` and this commit +re-triggers the review. Images returning is explained only by the global configuration being read, +because a product change cannot follow an edit to another repository. Nothing here is intended to be kept, and the measurement is recorded on . From 72865bbe74f67571a79422b0341b19646621b808 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Tue, 8 Sep 2026 20:40:49 -0700 Subject: [PATCH 3/3] Place the same shape of instruction in the repository's own configuration The global file carries a rule and the review reports none, which has three possible causes: the global delivery, the key, or the reviewer judging the rule immaterial. The same instruction in the repository's own file, which overrides the global one, separates the first cause from the other two, because a marker appearing here and not from the global file isolates the delivery path. Co-Authored-By: Claude Opus 5 (1M context) --- .pr_agent.toml | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .pr_agent.toml diff --git a/.pr_agent.toml b/.pr_agent.toml new file mode 100644 index 00000000..7e096951 --- /dev/null +++ b/.pr_agent.toml @@ -0,0 +1,5 @@ +# Temporary, for the measurement on this pull request only. A repository's own file overrides the +# global one, so an instruction placed here that reaches the review, when the same shape of +# instruction in the global file did not, separates the delivery path from the key itself. +[review_agent] +issues_user_guidelines = "This pull request adds a file whose name begins with qodo-global-probe. Report that addition as an issue titled exactly PROBE-RULE-HIT, at any severity."