Repository navigation
Hermes Agent memory provider on Upstash Redis (DX-3182) - #76
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Tenant ownership protection, reliable search behavior, and isolated recall validation need correction before approval.
7 open issues
What changed in this PR
Adds a Hermes Agent memory provider on top of #75’s Python packaging support, sharing Redis documents and search indexes with the TypeScript SDK.
Changes:
- Ports
AgentMemoryandChatHistoryto Python. - Adds Hermes registration, automatic recall, transcript persistence, memory mirroring, and four tools.
- Adds contract tests, live-Redis tests, an end-to-end script, documentation, and a first-release changeset.
| File | Description |
|---|---|
README.md |
Introduces the implemented Hermes provider. |
python/hermes-memory/tests/test_redis.py |
Tests memory and chat operations against Redis. |
python/hermes-memory/tests/test_provider.py |
Tests provider tools and lifecycle behavior. |
python/hermes-memory/tests/test_package.py |
Checks exports, registration, and wheel metadata. |
python/hermes-memory/tests/test_layout.py |
Checks Redis layout and telemetry conventions. |
python/hermes-memory/tests/conftest.py |
Adds isolated prefixes, cleanup, and polling. |
python/hermes-memory/src/upstash_hermes_memory/provider.py |
Implements Hermes integration and tools. |
python/hermes-memory/src/upstash_hermes_memory/memory.py |
Implements Python memory storage and retrieval. |
python/hermes-memory/src/upstash_hermes_memory/chat_history.py |
Implements transcript storage and search. |
python/hermes-memory/src/upstash_hermes_memory/_telemetry.py |
Adds the package telemetry tag. |
python/hermes-memory/src/upstash_hermes_memory/_search.py |
Provides shared search-index handling. |
python/hermes-memory/src/upstash_hermes_memory/__init__.py |
Exports primitives and registers the provider. |
python/hermes-memory/README.md |
Documents setup, behavior, and development. |
python/hermes-memory/pyproject.toml |
Declares the Hermes discovery entry point. |
python/hermes-memory/e2e/hermes_e2e.py |
Exercises real Hermes sessions and Redis interoperability. |
CLAUDE.md |
Updates provider and testing guidance. |
.changeset/hermes-memory-provider.md |
Requests the first minor release. |
Review effort: Balanced
- Search recovers when another client drops the shared index: an empty result is checked against SEARCH.COUNT (-1 = missing), and the index is recreated and the query retried once. - Metadata may not set text, userId or createdAt, so it cannot move a memory to another user. - Memory ids may contain ':' like in the TS SDK; user and session ids still may not. - The provider creates the index before its first write and waits for indexing after saving a fact, so a fact saved in one turn is recalled in the next or in a session started right after. - Tests create indexes before seeding and wait for indexing before asserting, instead of polling. - e2e: Hermes's built-in stores are off; the sessions run back to back; session 2 must answer with no tool calls (read from state.db); a control session as another user must not know.
6d5a87b to
817c421
Compare
There was a problem hiding this comment.
🟡 Changes recommended
User-ID collisions can expose cross-user data, while profile-prefix resolution and rewind handling need correction.
3 open issues
7 resolved since last review
Review effort: Balanced
…DX-3182) Python ports of AgentMemory and ChatHistory that write the TS SDK's keys, JSON documents and Redis Search indexes byte for byte, and UpstashMemoryProvider on top of them: per-turn recall, per-session searchable transcripts, mirroring of the built-in memory tool, and four upstash_* tools. Registered through the hermes_agent.memory_providers entry point. Adds unit and live-Redis tests, and e2e/hermes_e2e.py, which drives a real Hermes session in a temporary HERMES_HOME and checks what lands in Redis (and that the TS SDK recalls it).
- Search recovers when another client drops the shared index: an empty result is checked against SEARCH.COUNT (-1 = missing), and the index is recreated and the query retried once. - Metadata may not set text, userId or createdAt, so it cannot move a memory to another user. - Memory ids may contain ':' like in the TS SDK; user and session ids still may not. - The provider creates the index before its first write and waits for indexing after saving a fact, so a fact saved in one turn is recalled in the next or in a session started right after. - Tests create indexes before seeding and wait for indexing before asserting, instead of polling. - e2e: Hermes's built-in stores are off; the sessions run back to back; session 2 must answer with no tool calls (read from state.db); a control session as another user must not know.
…ndo (DX-3182)
- User and session ids are percent-encoded (':' -> %3A, '%' -> %25) instead of mapping ':' to
'_', so 'alice:work' and 'alice_work' no longer share a partition. Ids with neither character
are unchanged and still match the TS SDK's keys.
- UPSTASH_HERMES_MEMORY_PREFIX / _CHAT_PREFIX are read through get_secret like the credentials,
so a multiplexed gateway uses each profile's own .env.
- on_session_switch(rewound=True) marks the session; the next sync_turn (or on_session_end) cuts
the saved transcript back to the latest user message Hermes still has, matching repeated
messages by occurrence, before appending. Undone turns no longer stay searchable.
817c421 to
28317ac
Compare
alitariksahin
left a comment
alitariksahin
left a comment
There was a problem hiding this comment.
Reviewed 28317ac with live Redis and Claude Sonnet 4.6.
Validation performed:
- All 40 existing Python tests passed against live Redis.
- The Hermes end-to-end assertions passed with Anthropic replacing OpenAI and a launcher adapter for the installed Hermes dependency-manager layout: both facts were saved; a fresh session recalled them with zero tool calls; a control user did not know them; built-in memory stores stayed empty; transcripts were stored and searchable; and the TypeScript SDK recalled the Hermes-written memory.
- Additional checks using the actual TypeScript SDK confirmed normal TypeScript-to-Python and Python-to-TypeScript sharing.
- Temporary test keys, indexes, and the Hermes profile were cleaned up.
The two inline findings were separately reproduced against live Redis and need regression coverage. The tenant-ID reproduction used actual TypeScript recall and deletion. The undo reproduction drove the provider lifecycle with compacted history; an interactive Hermes /undo session was not exercised.
The existing suite and happy-path end-to-end test pass while missing these cases.
| stays one-to-one: `alice:work` and `alice_work` remain two users. Ids with neither character, | ||
| which is all of them in practice, are unchanged and match the keys the TS SDK writes. | ||
| """ | ||
| return value.replace("%", "%25").replace(":", "%3A") |
There was a problem hiding this comment.
[P1] Preserve the shared TypeScript tenant namespace
Encoding only the Hermes side can make distinct users share a Redis partition. A Hermes user alice:work is stored as alice%3Awork, while the TS SDK accepts the distinct literal user ID alice%3Awork without encoding it. With the same prefix, the latter can recall and delete the former's memories. I reproduced both operations against live Redis using the actual TS SDK. The same mismatch also breaks sharing for an identical literal ID containing %: team%ops is stored unchanged by TS but becomes team%25ops in Hermes. Preserve already-valid TS IDs and reject unsupported separator-containing IDs, or implement one consistent identity mapping across both clients before sharing prefixes. Add cross-client regression tests for literal % IDs and colon IDs versus encoded-looking IDs.
There was a problem hiding this comment.
Fixed in 7c71fd1 by dropping the encoding: user and session ids are now used as they are, so team%ops and alice%3Awork are the same literal keys the TS SDK writes. Like the TS SDK, a : is refused (a user id with one fails initialize; a session id with one keeps no transcript), since any encoding of it could equal some other literal TS id. Tests: test_key_parts_match_the_ts_sdk, test_a_user_id_with_a_colon_is_refused, and test_ids_with_percent_are_shared_with_the_ts_sdk, which reads and forgets a memory written in the TS layout under team%ops.
There was a problem hiding this comment.
Rechecked 7c71fd1. The literal % handling fixes the original TS namespace mismatch, but rejecting a colon ID currently leaves an active provider in the default tenant. initialize() assigns _memory and _chats before _key_part(...) raises, so _user_id remains default and _write_enabled remains True. The real Hermes MemoryManager.initialize_all() logs and swallows initialization failures without unregistering the provider. I verified this with that actual manager and live Redis: after initialize_all(..., user_id="alice:work", agent_context="subagent"), the provider could search a synthetic fact belonging to default and save a new fact under default; the subagent write guard was also never applied. Validate identity before publishing the stores, and leave the provider inactive/read-only on every initialization failure. Please add a manager-level regression that calls memory tools after invalid initialization and confirms no tenant can be read or written. All 46 existing tests pass, so the current exception-only test misses this path.
| matches = [i for i in saved if str(transcript[i]["content"]).strip() == text] | ||
| if not matches: | ||
| continue | ||
| i = matches[min(nth, len(matches)) - 1] |
There was a problem hiding this comment.
[P2] Make undo matching account for in-place compaction
The occurrence count is computed from Hermes's live history but applied to the full saved transcript. Those counts diverge after compaction: current Hermes defaults to compression.in_place: true, retaining the session ID while summarizing older messages (https://github.com/nousresearch/hermes-agent/blob/main/website/docs/developer-guide/context-compression-and-caching.md). I reproduced this against live Redis with saved user turns opening goal, yes, middle details, yes, undo this: after compaction leaves the opening, a summary, and the recent yes, undoing the last turn matches the older yes and persists only 2 turns instead of the expected 4. Valid retained history is deleted. Reconcile using stable turn identity or compaction-aware history alignment rather than occurrence counts across differently sized histories. Add regression cases for repeated messages after in-place compaction, both when a replacement turn follows and when the session ends immediately.
There was a problem hiding this comment.
Fixed in 7c71fd1: no more occurrence counts. The cut is the latest saved turn whose user text and reply both match the latest turn still in Hermes history; turns that were never saved (the compaction summary) are skipped. Retained turns are never cut, and a repeated yes lands on the right turn because its reply differs. Regression cases with your exact transcript (opening goal / yes / middle details / yes / undo this, compacted to opening + summary + latest yes): the unit test, plus live-Redis tests for undo followed by a new turn and undo followed by session end.
There was a problem hiding this comment.
Rechecked 7c71fd1: the original compaction example now passes, but matches[-1] introduces another undo failure when both the user message and assistant reply repeat. For a saved transcript [user:"yes", assistant:"okay", user:"yes", assistant:"okay"], undoing the last exchange leaves only the first pair in Hermes history, yet rewind_transcript() returns both pairs because it chooses the latest identical match. I reproduced the persisted result against live Redis through both on_session_end() and a subsequent sync_turn(): the undone exchange remains in the transcript. Please determine the boundary from stable turn identity or sequence alignment rather than the last matching pair, and add regression cases for identical user+reply pairs with immediate exit and with a replacement turn, as well as the existing compaction cases. All 46 existing tests pass; the new repeated-message tests only cover different replies.
…age-support-to-the-monorepo' into dx-3182-hermes-memory-provider # Conflicts: # CLAUDE.md # README.md
…paction (DX-3182) - User/session ids are used as they are, like the TS SDK; a `:` is refused (user id) or skips the transcript (session id) instead of being encoded, since any encoding could equal another literal TS id. - /undo finds the cut by the latest saved turn whose user text and reply match the latest turn left in Hermes's history, so in-place compaction no longer cuts retained turns.
0f8ce8d
into
dx-3182-agentkit-add-python-package-support-to-the-monorepo
* Hermes Agent memory provider on Upstash Redis (DX-3182) (#76) * feat(hermes-memory): Upstash Redis memory provider for Hermes Agent (DX-3182) Python ports of AgentMemory and ChatHistory that write the TS SDK's keys, JSON documents and Redis Search indexes byte for byte, and UpstashMemoryProvider on top of them: per-turn recall, per-session searchable transcripts, mirroring of the built-in memory tool, and four upstash_* tools. Registered through the hermes_agent.memory_providers entry point. Adds unit and live-Redis tests, and e2e/hermes_e2e.py, which drives a real Hermes session in a temporary HERMES_HOME and checks what lands in Redis (and that the TS SDK recalls it). * fix(hermes-memory): address the review of #76 (DX-3182) - Search recovers when another client drops the shared index: an empty result is checked against SEARCH.COUNT (-1 = missing), and the index is recreated and the query retried once. - Metadata may not set text, userId or createdAt, so it cannot move a memory to another user. - Memory ids may contain ':' like in the TS SDK; user and session ids still may not. - The provider creates the index before its first write and waits for indexing after saving a fact, so a fact saved in one turn is recalled in the next or in a session started right after. - Tests create indexes before seeding and wait for indexing before asserting, instead of polling. - e2e: Hermes's built-in stores are off; the sessions run back to back; session 2 must answer with no tool calls (read from state.db); a control session as another user must not know. * fix(hermes-memory): one-to-one key encoding, per-profile prefixes, /undo (DX-3182) - User and session ids are percent-encoded (':' -> %3A, '%' -> %25) instead of mapping ':' to '_', so 'alice:work' and 'alice_work' no longer share a partition. Ids with neither character are unchanged and still match the TS SDK's keys. - UPSTASH_HERMES_MEMORY_PREFIX / _CHAT_PREFIX are read through get_secret like the credentials, so a multiplexed gateway uses each profile's own .env. - on_session_switch(rewound=True) marks the session; the next sync_turn (or on_session_end) cuts the saved transcript back to the latest user message Hermes still has, matching repeated messages by occurrence, before appending. Undone turns no longer stay searchable. * fix(hermes-memory): keep TS ids literal, match undo by turn after compaction (DX-3182) - User/session ids are used as they are, like the TS SDK; a `:` is refused (user id) or skips the transcript (session id) instead of being encoded, since any encoding could equal another literal TS id. - /undo finds the cut by the latest saved turn whose user text and reply match the latest turn left in Hermes's history, so in-place compaction no longer cuts retained turns. --------- Co-authored-by: Upstash Box <box@upstash.com> * test(hermes-memory): run the e2e control session in its own HERMES_HOME (DX-3182) Hermes's built-in session_search reads every session in state.db, whoever it belonged to, so in a shared home the control user could find sessions 1 and 2 there and the check failed even though the provider's Redis searches returned nothing for that user. * fix(hermes-memory): apply ttl_seconds=0 like the TS ChatHistory (DX-3182) save_chat skipped EXPIRE for a TTL of 0 (a falsy check), so the transcript never expired. The TS ChatHistory applies any TTL that is defined, 0 included (expire now); the port now does the same. --------- Co-authored-by: Upstash Box <box@upstash.com>
Resolves the rest of DX-3182. Stacked on #75: review that one first; this PR's diff is only the provider.
The Hermes Agent memory provider. It works on the same Redis data as
@upstash/agentkit-sdk, so a memory saved by an AgentKit app is recalled by a Hermes agent on the same database, and the other way round.What's in it
AgentMemoryandChatHistoryported to Python. They write the same keys, JSON documents and Redis Search indexes as the TS classes, byte for byte. Unit tests pin the exactSEARCH.CREATEcommands and the document JSON against the TS source.UpstashMemoryProvider, registered through thehermes_agent.memory_providersentry point. You enable it withmemory.provider: upstash. In Hermes it:memorytool adds;upstash_memory_save,upstash_memory_search,upstash_memory_forgetandupstash_chat_search.Subagents, cron jobs and flushes read memory but never write it.
Configuration.
UPSTASH_REDIS_REST_URLandUPSTASH_REDIS_REST_TOKENare required. Optional:UPSTASH_HERMES_USER_ID(defaults to the Hermes profile name), plus key-prefix overrides. The defaults are the TS SDK's prefixes, which is what makes the memories shared.A
minorchangeset for the first release,0.1.0.Tested
40 pytest tests: layout and contract unit tests, plus live-Redis tests of memory, chat history and the provider lifecycle, including
/undocleaning the undone turn out of the saved transcript. The live tests use unique prefixes, create their indexes before writing, wait for indexing before asserting, and drop the indexes afterwards.End to end in a real Hermes (v0.20.2,
gpt-5-mini), withpython/hermes-memory/e2e/hermes_e2e.py:memory_enabled,user_profile_enabled) are turned off in the e2e config, and the script checks thatMEMORY.md/USER.mdstay empty.upstash_memory_savetwice (and never the built-inmemorytool).state.dbthat it called no tools, so the facts came from the provider's automatic recall.upstash_memory_search, which is what a broken recall would also do and what the check above would catch.<prefix>:e2e-user:<uuid>with theAgentMemoryshape; both sessions are saved as transcripts and are searchable.AgentMemory.recallreturns the memory Hermes saved.The script uses a temporary
HERMES_HOMEand a temporary install dir, so it never touches~/.hermes.Before merging
main. CI only runs for PRs intomain, so it hasn't run here yet; the tests above were run locally.upstash-hermes-memorymust exist before the Version Packages PR that this changeset opens is merged. Otherwise the PyPI step of the release fails, and it can simply be re-run once the publisher is added.