Skip to content

feat(agent-toolkit): document the virtual column ids for creation time, update time and item id - #482

Merged
stasshw merged 7 commits into
masterfrom
fix/filter-guidelines-magic-column-ids
Sep 7, 2026
Merged

feat(agent-toolkit): document the virtual column ids for creation time, update time and item id#482
stasshw merged 7 commits into
masterfrom
fix/filter-guidelines-magic-column-ids

Conversation

@stasshw

@stasshw stasshw commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

What

Document the virtual column ids __creation_log__, __last_updated__ and __item_id__, and correct the last_updated guideline, which taught the wrong id.

Detail lives in the per-type get_column_type_info guidelines. The always-visible text is a pointer, not a copy of it — the get_board_items_page description trades its GROUP FILTERING paragraph for one [VIRTUAL COLUMNS] line covering all four ids, so getDescription() is 2,249 chars against 2,126 on master while documenting four ids instead of one.

Descriptions only — no schema shape, no filtering behavior, no error codes touched.

Why

These ids are required to filter or sort by when an item was created or updated, and get_board_info never returns them. Models used the bare ColumnType enum names instead — and our own last_updated guideline used "columnId": "last_updated" in its ✅ examples, observed being copied verbatim in production (RequestId = fbf038d1-6264-462d-ba45-aab0521588c8).

Measured over 7 days on get_board_items_page, against a baseline ResourceNotFoundException rate of 3.0% in filters and 2.3% in orderBy:

columnId calls errors error rate
__creation_log__ (filters + orderBy) 9,928 16 0.2%
__last_updated__ (filters + orderBy) 6,218 19 0.3%
bare creation_log 1,931 513 26.6%
bare last_updated 1,420 427 30.1%
bare item_id 504 49 9.7%
pulse_id 42 40 95.2%

Effect

~1,000 avoidable errors a week across ~900 accounts. orderBy carries 752 of them — "sort newest first" is the common phrasing, and orderBy[].columnId read only "The id of the column to order by". That was the largest single gap.

Notes

Why the two columnId descriptions still mention the ids, rather than the guidelines alone. Two reasons, both concrete:

  • Discovery. get_column_type_info takes a columnType the model must already name. All four are valid ColumnType enum members, so the entries are reachable — but get_board_info never lists creation_log on a board, so a model asked to sort newest-first has no reason to look up a type it doesn't know exists. Reachable is not discoverable, and that gap is the bug.
  • orderBy has no guidelines channel. buildFilterGuidelinesForColumnType returns text titled "Filtering Guidelines" and nothing routes it to the sort path — which is where most of the errors are.

Verified against platform source. filter_rules_service.rb:241-271 (filters) and sort_settings_service.rb:31-49 (orderBy) hold the same case statement accepting exactly group, __item_id__, __creation_log__, __last_updated__, raising ResourceNotFoundException otherwise. compareAttribute selects a facet because PulseLogColumn/PulseUpdatedColumn hold a person as well as a date (SETTING_MODE_PERSON_AND_DATE).

Three claims were checked and dropped as wrong or unproven, so they appear nowhere in the text:

  • compareAttribute is not required — 3,899 __creation_log__ calls omit it with 2 errors.
  • pulse_updated is not rejected — monday auto-generates real columns as pulse_updated_<suffix> (76 calls, 13.2%, mostly valid).
  • Bare id rejection is unproven (n=1).

The original plan's step 1 was dropped after measuring. It proposed replacing bare type-name example ids (status, date, email, …) with <your_column_id> placeholders, assuming agents copy them. Adding the missing denominator reverses that: monday names the first column of a type after the type, so those are frequently real ids — status 1.4% over 3,535 calls, date 1.6% over 790, both below baseline. Two of the eight entries already used a placeholder, so their traffic couldn't have come from this file at all. Only last_updated was genuinely wrong.

__group__ is a separate pre-existing failure (~300 calls/day, 98% error, 72 accounts), flat over 8 days, so it predates #481 rather than regressing from it. Documented as a ❌ in the group guideline because text saying "double underscores" would otherwise make it worse. Not repaired here.

Every example in this PR was then run against a real board (~45 read-only calls, 2026-09-07). Acceptance is not correctness, so each needed a discriminating result rather than "did not error". All four ids work in filters, orderBy accepts __creation_log__ and group, and [UNIT, AMOUNT] is exact for all 4 units x both operators — WORKDAYS 5 correctly skips the weekend, DAYS 30 lands on the right boundary. The hard failures reproduce too: bare creation_log / item_id / pulse_id / __group__ all 404, and a bare 7 as a window returns 500 INTERNAL_SERVER_ERROR, which is what the [UNIT, AMOUNT] line exists to prevent.

That testing found two defects in text this PR was introducing, fixed in the last commit:

  • LAST_WEEK and LAST_MONTH silently match nothing. Pre-existing in last_updated, and this PR had copied the list verbatim into the new creation_log entry. Proven against ground truth: the board has Aug 9 creations and Sept 1 updates, LAST_WEEK/LAST_MONTH return zero either way — even with the correct compareAttribute — while TODAY, YESTERDAY, THIS_WEEK and THIS_MONTH return the right items. Both keywords are gone, replaced by within_the_last with ["WEEKS", 1] / ["MONTHS", 1], which returns exactly the items LAST_WEEK missed. A regression test now blocks either keyword from returning as a correct example. This is a server-side defect worth its own ticket — two keywords the filter accepts without error and matches nothing.
  • "The latter three are also valid in orderBy" excluded group. All four are, per sort_settings_service.rb and a live orderBy: group call.

Two scalar compareValue examples marked ❌ were deliberately kept even though my account tolerates them. filter_rules_service.rb:152-167 validate_date_values requires the array form and raises ArgumentError otherwise, but only behind the use_shared_filter_rules_service flag, with an unvalidated fallback. My account lacks the flag — dropping the ❌ would over-generalise from one account, the same mistake that produced __group__.

email, numbers, long_text and checkbox are untested: no reachable board has those column types. All four are untouched here.

Out of scope

  • No new groupIds parameter and no server-side repair. Per the investigation method: ship the description alone, re-measure, then decide.
  • __group__ documented but not fixed.

Follow-up

Re-measure on the MCP usage dashboard (panel 15) after release, especially the orderBy split. Note this PR also shortens #481's group paragraph, so the group numbers reflect both changes together.

🤖 Generated with Claude Code

stasshw and others added 2 commits September 7, 2026 11:32
….69.0)

Filtering or sorting items by creation/update time needs the virtual ids
__creation_log__ and __last_updated__, which get_board_info never returns.
Models reached for the bare ColumnType enum names instead, and our own
last_updated filter guideline used "columnId": "last_updated" verbatim in its
correct examples.

Measured over 7 days on get_board_items_page against a 3.0% baseline for
filters and 2.3% for orderBy: bare creation_log 26.6% over 1,931 calls, bare
last_updated 30.1% over 1,420, bare item_id 9.7% over 504, pulse_id 95.2%.
The underscored forms sit at 0.2-0.3% over ~16,000 calls. 752 of the ~1,038
weekly errors are in orderBy, whose columnId was documented only as "The id
of the column to order by".

Descriptions only, no schema or behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four virtual column ids were documented in three places at once: the tool
description, the two columnId schema descriptions, and the per-type guidelines.
Detail now lives only in the guidelines, which get_column_type_info already
serves per column type. The always-visible text is a pointer instead of a
tutorial.

Replaces the GROUP FILTERING paragraph with one VIRTUAL COLUMNS line covering
all four ids, so getDescription is 2249 chars against 2126 on master while
documenting four ids instead of one. orderBy[].columnId keeps its own mention,
since guidelines are filter-only and never reach the orderBy path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stasshw stasshw changed the title feat(agent-toolkit): teach the double-underscore virtual column ids feat(agent-toolkit): document the virtual column ids for creation time, update time and item id Sep 7, 2026
stasshw and others added 5 commits September 7, 2026 12:21
…e_last and within_the_next

The two rolling-window operators were documented nowhere, so models invented
compareValue shapes and the monolith answered INTERNAL_SERVER_ERROR: 176 calls
over 12h, 106 of them 500s (60.2%) across 65 accounts, ~45% of all 500s on
get_board_items_page. Every other filter rule sits at 0.01%.

Descriptions only, no schema or behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…w figures

Re-measured with shape-accurate bucketing: 240 of 251 window calls in 12h
sent a compareValue that was not [UNIT, AMOUNT], and the 9 well-formed ones
did not error. States that a bad window can also surface as
InvalidArgumentException/no_operator_config rather than a 500.

Adds counter-examples for the two mistakes seen in production: the unit in
compareAttribute, and within_the_last on the virtual time columns without
compareAttribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…behavior

Hand-tested every claim on production. WORKDAYS works and is honored, not
just tolerated. compareAttribute is optional on __creation_log__ and
__last_updated__ for within_the_last, so the counter-examples claiming it is
required were wrong and are replaced with the PAST_DATETIME shape that does
fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Verified every example in this PR against a real board. Three were wrong.

- LAST_WEEK and LAST_MONTH silently return zero items, even with the right
  compareAttribute, on boards that do have matching items. TODAY, YESTERDAY,
  THIS_WEEK and THIS_MONTH all return the right ones. Removed both keywords
  from creation_log and last_updated and pointed models at within_the_last
  with ["WEEKS", 1] or ["MONTHS", 1], which returns exactly the items
  LAST_WEEK missed.
- compareAttribute is not required. Omitting it returns identical results,
  matching production where 3,899 __creation_log__ calls omit it with 2
  errors. Reworded as optional, with what it actually selects.
- "The latter three are also valid in orderBy" excluded group. All four are,
  per sort_settings_service.rb and a live orderBy: group call.

Added a regression test so neither keyword can come back as a correct example.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stasshw
stasshw merged commit 4fbbf69 into master Sep 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants