You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
isNativeBenchmarkConfig → isKeplerBenchmarkConfig, and the private NATIVE_BENCHMARK_ID_SET / NativeModelBenchmarkConfig get the same rename
run-by-id renames nativeBenchmark → keplerBenchmark, and the injected-id error messages now say "Kepler benchmark"
package.json description updated
Not changed, because these uses of "native" mean something else:
WebSearchEngine.Native and the draco/search "native" engine values, which are OpenRouter's native web search
@typescript/native
bench-worker-native in the logger tests, which is the deployed Cloud Run resource name and will be renamed separately if at all
Why?
"Native" was too generic a name for the harness, so it now has a proper noun (Kepler), alongside Harbor.
How to test
rg -n 'NativeBenchmarkRunConfig|isNativeBenchmarkConfig' src returns nothing. There is no behavior change.
Reviewer focus
This is a breaking rename of public exports with no aliases. Nothing in the openrouter-web subtree consumer imports these names outside packages/bench-harness, so the next subtree pull needs no follow-up edits.
Checklist
Tests cover changed behavior
Public API or configuration changes are backward compatible, or the break is documented
Benchmark changes document dataset provenance and licensing (n/a)
No credentials, private results, or restricted dataset contents are included
I'll fix CI failures and address comments from users with write access that start with 'DevinAI' or '@devin'.
Disable automatic comment, CI, and merge conflict monitoring
Original prompt from Abhinav
SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1790381041.747209]: @Devin instead of "native" we need to call our TS benchmark harness something else. come up with a cool name
</latest_message>
The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.
devin-ai-integrationBot
changed the title
refactor: rename the native benchmark set to Mendel
refactor: rename the native benchmark set to Kepler
Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
The TS benchmark harness is now called Kepler. This PR renames the harness-specific "native" identifiers to match.
What changed?
NativeBenchmarkRunConfigSchema/NativeBenchmarkRunConfig→KeplerBenchmarkRunConfigSchema/KeplerBenchmarkRunConfigisNativeBenchmarkConfig→isKeplerBenchmarkConfig, and the privateNATIVE_BENCHMARK_ID_SET/NativeModelBenchmarkConfigget the same renamerun-by-idrenamesnativeBenchmark→keplerBenchmark, and the injected-id error messages now say "Kepler benchmark"package.jsondescription updatedNot changed, because these uses of "native" mean something else:
WebSearchEngine.Nativeand the draco/search"native"engine values, which are OpenRouter's native web search@typescript/nativebench-worker-nativein the logger tests, which is the deployed Cloud Run resource name and will be renamed separately if at allWhy?
"Native" was too generic a name for the harness, so it now has a proper noun (Kepler), alongside Harbor.
How to test
rg -n 'NativeBenchmarkRunConfig|isNativeBenchmarkConfig' srcreturns nothing. There is no behavior change.Reviewer focus
packages/bench-harness, so the next subtree pull needs no follow-up edits.Checklist
Link to Devin session: https://openrouter.devinenterprise.com/sessions/26aed72e72b94648bcafa700baf71832
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/26aed72e72b94648bcafa700baf71832?variant=devin
Requested by: @abhinav-pola