Skip to content

Support Python packages in the monorepo (DX-3182) - #75

Merged
CahidArda merged 6 commits into
mainfrom
dx-3182-agentkit-add-python-package-support-to-the-monorepo
Oct 28, 2026
Merged

CahidArda merged 6 commits into
mainfrom
dx-3182-agentkit-add-python-package-support-to-the-monorepo

Conversation

@CahidArda

@CahidArda CahidArda commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Resolves part of DX-3182.

This PR lets the monorepo hold Python packages that release through the existing Changesets flow, with no second versioning system. The first package is a scaffold for a Hermes Agent memory provider on Redis Search. It stays unreleased until the Python port lands.

How it works

  • uv workspace at the root: pyproject.toml (members python/*) and a single uv.lock.
  • Changesets still owns versions. Each Python package has a private package.json that only holds its version. python/* is added to pnpm-workspace.yaml, so pnpm changeset lists the package and changeset version bumps it and writes its CHANGELOG. With privatePackages: { version: true, tag: false }, changesets never tags or publishes it.
  • scripts/sync-version.mjs now also finds every python/* package and copies its version into pyproject.toml and src/<module>/_version.py (the VERSION constant used for telemetry). --check still catches drift. ci:version runs uv lock afterwards, so the Version Packages PR carries a matching lockfile.
  • scripts/publish-python.mjs (pnpm ci:publish:python) does for PyPI what changeset publish does for npm:
    • For any version PyPI doesn't have yet, it runs uv build and uv publish with trusted publishing, then creates a <name>@<version> tag and a GitHub release.
    • It decides from PyPI's contents, not from the last commit, so re-running it is safe.
    • 0.0.0 means unreleased and is skipped, which is why this scaffold can merge without reaching PyPI.
    • --dry-run prints the plan without changing anything.
  • Adding a future package means adding a folder under python/ and registering a trusted publisher for it on PyPI. Nothing else changes.

Also in this PR

  • Scaffold python/hermes-memory: upstash-hermes-memory at 0.0.0, requires Python 3.11+, depends on upstash-redis>=1.8.0 (which already supports Redis Search).
  • New root scripts py:lint (ruff) and py:test (pytest); .gitignore entries for the virtualenv and caches; a fetch global for scripts/*.mjs in the eslint config.
  • A "Python packages" section in CLAUDE.md and a README row.
  • ci.yml and release.yml (pushed from a local checkout, since the sandbox token could not edit workflow files): CI gets a python · lint · test job (uv sync --locked, ruff, pytest) that runs after the Node job because both share the test Redis. The release job installs uv before changesets (ci:version runs uv lock) and runs pnpm ci:publish:python in publish mode, using trusted publishing through the existing id-token: write.
  • Python tests for upstash-hermes-memory: the three version carriers agree, the built wheel holds the module with the right metadata, and a Redis round trip that skips without secrets.

Not in this PR

  • A PyPI trusted publisher for upstash-hermes-memory, which must exist before its first non-zero version merges.

Verified (in an Upstash Box)

  • pnpm lint, pnpm build, pnpm typecheck and node scripts/sync-version.mjs --check pass.
  • uv sync --locked --all-packages, pnpm py:lint and pnpm py:test pass.
  • Release dry run: I added a temporary upstash-hermes-memory: minor changeset and ran pnpm ci:version. It bumped package.json, pyproject.toml, _version.py and uv.lock to 0.1.0 and wrote the CHANGELOG. publish-python.mjs --dry-run then reported would build, publish and tag. I reverted all of it before committing.
  • Not exercised: a real PyPI upload.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JTGdTkxgRFnNpNZow3diPo

Add a uv workspace under python/* that releases through the existing
Changesets flow. Each Python package carries a private package.json that
holds its version. sync-version.mjs copies that version into pyproject.toml
and _version.py, and publish-python.mjs uploads any version PyPI doesn't
have yet. Scaffold upstash-hermes-memory at 0.0.0, which means unreleased.

The CI and release workflow changes come in a separate commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTGdTkxgRFnNpNZow3diPo
@linear-code

linear-code Bot commented Oct 22, 2026

Copy link
Copy Markdown

DX-3182

@CahidArda

Copy link
Copy Markdown
Collaborator Author

The workflow part is in.

  • c7da5fa ci.yml: a python · lint · test job (uv sync --locked, ruff, pytest). It runs after the Node job because both use the shared test Redis. release.yml: installs uv before changesets (ci:version runs uv lock), then runs pnpm ci:publish:python in publish mode. It uses trusted publishing through the existing id-token: write, with no PyPI token.
  • 435b1f0 Python tests for upstash-hermes-memory: the three version carriers agree, the built wheel holds the module with the right metadata, and a Redis round trip that skips without secrets. In CI all 8 pass, including the Redis round trip.

Both CI jobs are green. Before the first non-zero version, someone with PyPI access still needs to add a trusted publisher for upstash-hermes-memory (owner upstash, repo agentkit, workflow release.yml, no environment).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Publication retries cannot recover unfinished tags or releases, and local Redis tests skip credentials stored in the documented .env file.

2 open issues
1 file not reviewed
  • pnpm-lock.yaml: Generated file
What changed in this PR

Adds Python package support alongside the existing pnpm monorepo, using Changesets as the single versioning system. The Hermes memory provider remains an unreleased scaffold.

Changes:

  • Adds a uv workspace, Python lint/test commands, and CI integration.
  • Extends version synchronization and adds trusted PyPI publishing with GitHub release creation.
  • Introduces the Hermes package scaffold with version, wheel-content, and Redis smoke tests.
File Description
uv.lock Locks Python workspace dependencies.
scripts/​sync-version.mjs Synchronizes Python version files from Changesets.
scripts/​publish-python.mjs Adds PyPI publishing, tags, and GitHub releases.
README.md Lists the Python package and development commands.
python/​hermes-memory/​tests/​test_version.py Checks version and package-name consistency.
python/​hermes-memory/​tests/​test_redis.py Adds a Redis round-trip smoke test.
python/​hermes-memory/​tests/​test_package.py Checks public exports and wheel contents.
python/​hermes-memory/​tests/​conftest.py Provides a credential-gated Redis fixture.
python/​hermes-memory/​src/​upstash_hermes_memory/​_version.py Stores the generated scaffold version.
python/​hermes-memory/​src/​upstash_hermes_memory/​__init__.py Defines the scaffold’s public export.
python/​hermes-memory/​README.md Documents the unreleased scaffold.
python/​hermes-memory/​pyproject.toml Defines package metadata and build configuration.
python/​hermes-memory/​package.json Carries the Changesets-managed version.
pyproject.toml Configures the uv workspace, Ruff, and pytest.
pnpm-workspace.yaml Includes Python version-carrier packages.
pnpm-lock.yaml Registers the Python workspace member.
package.json Adds Python commands and release integration.
eslint.config.js Allows fetch in Node scripts.
CLAUDE.md Documents Python development and release conventions.
.gitignore Excludes Python environments and caches.
.github/​workflows/​release.yml Installs uv and invokes Python publishing.
.github/​workflows/​ci.yml Runs Python checks after Node checks.
.changeset/​config.json Enables private-package versioning without tagging.

Review effort: Balanced

Comment thread python/hermes-memory/tests/conftest.py Resolved
Comment thread scripts/publish-python.mjs Resolved
…tests (DX-3182)

publish-python.mjs checked PyPI only, so a run that uploaded but failed to push the tag or create
the GitHub release was skipped forever after. Upload, tag and release are now reconciled one by
one; an existing tag is never moved.

conftest.py loads the repo-root .env like the TS test-support does, without overriding variables
already set (CI's secrets).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Publishing retries treat partially uploaded PyPI versions as complete, leaving missing distribution files unrepaired.

1 open issue
2 resolved since last review
1 file not reviewed
  • pnpm-lock.yaml: Generated file

Review effort: Balanced

Comment thread scripts/publish-python.mjs Resolved
A version counts as published only when PyPI has both its wheel and its sdist. Otherwise the
release is built and only the files PyPI does not list are uploaded, with --check-url as a second
guard against a lagging JSON API. Before, a run that uploaded one file and then failed made every
retry skip the upload.
@alitariksahin

Copy link
Copy Markdown

Reviewed 1127ccc. I found no additional blocking issue in this PR.

Validation performed:

  • Python scaffold tests: 7 passed, 1 Redis test initially skipped. That Redis smoke test subsequently passed as part of the full 40-test suite on stacked PR Hermes Agent memory provider on Upstash Redis (DX-3182) #76, against a live Redis database.
  • Ruff, Python formatting, and version synchronization checks passed.
  • Nine isolated publisher checks passed: new release; wheel-only and sdist-only partial releases; completed PyPI upload with missing tag/release; already-complete release; existing tag with missing GitHub release; unreleased 0.0.0; dry run; and a PyPI HTTP 503 response.

I recommend checking those publisher cases into the repository as automated regression tests for scripts/publish-python.mjs, since the package/wheel tests do not exercise release reconciliation.

A real PyPI upload and GitHub Actions OIDC/trusted-publishing authentication remain unverified. The PyPI trusted publisher for upstash/agentkit / release.yml must be configured before the first non-zero release.

@CahidArda
CahidArda merged commit 4a7fc36 into main Oct 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants