From 8e145f2dcbb6188fc37b1221d5ecd27275c1880d Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:36:00 -0400 Subject: [PATCH 01/15] =?UTF-8?q?docs:=20design=20for=20simplifying=20veti?= =?UTF-8?q?ver=E2=86=94Connect=20integration=20testing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adopt posit-dev/with-connect in both repos, move vetiver's Connect tests fully into vetiver, strip vetiver-specific test baggage from rsconnect-python (re-homing its own system-caches integration test), and relabel the deploy_python_fastapi compatibility shim rather than removing it. --- ...iver-connect-test-simplification-design.md | 223 ++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md diff --git a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md new file mode 100644 index 00000000..feda0813 --- /dev/null +++ b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md @@ -0,0 +1,223 @@ +# Simplifying the vetiver ↔ Connect integration-testing relationship + +**Date:** 2026-06-30 +**Status:** Approved design, pending implementation plan +**Repos affected:** `vetiver-python`, `rsconnect-python` + +## Problem + +The integration-testing relationship between `vetiver-python` and +`rsconnect-python` is both inverted and duplicated: + +- The real dependency is one-way: `vetiver-python` declares + `rsconnect-python>=1.11.0` as a production dependency and calls + `rsconnect.actions.deploy_python_fastapi` / `rsconnect.api.RSConnectServer` / + `RSConnectClient` directly. +- Despite not depending on vetiver, `rsconnect-python` carries vetiver-specific + test baggage: a `vetiver-testing/` harness, `tests/test_vetiver_pins.py`, a + `--vetiver` pytest marker, and a nightly `test-dev-connect` CI job that installs + vetiver and deploys a model. +- Both repos independently maintain a near-identical bespoke Connect-in-Docker + harness (`docker-compose.yml`, `setup-rsconnect/` with `dump_api_keys.py`, + `add-users.sh`, `users.txt`, `rstudio-connect.gcfg`). + +## Goal + +vetiver owns and runs all vetiver↔Connect integration tests in its own repo; +`rsconnect-python` carries no vetiver-specific test code. Both repos use +[`with-connect`](https://github.com/posit-dev/with-connect) — the preferred +shared tool for running Connect in Docker during tests — instead of a hand-rolled +harness. + +## Target architecture + +``` +rsconnect-python vetiver-python + ├ tests/test_main_system_caches.py ├ vetiver/tests/test_rsconnect.py + │ (own integration test; kept) │ (reads env creds, single admin user) + ├ CI: own with-connect job; ├ CI: posit-dev/with-connect Action (pinned) + │ creates publisher user via admin API ├ local: `make test-rsc` → uvx with-connect + └ actions.py compatibility shim └ deletes docker-compose.yml, + (deploy_python_fastapi for vetiver) script/setup-rsconnect/, dump_api_keys.py + +Deleted from rsconnect-python: vetiver-testing/, tests/test_vetiver_pins.py, +the --vetiver marker, the nightly test-dev-connect job, root docker-compose.yml, +and the dev/dev-stop Justfile recipes. +``` + +The reverse-dependency canary ("does new rsconnect break vetiver?") lives in +vetiver's weekly CI matrix, which already installs `rsconnect` from `main` and +runs the integration tests. rsconnect PRs do not get that signal directly; a +failure notification on the weekly canary is recommended so rsconnect maintainers +learn of breakage without watching the schedule. + +### `with-connect` behavior the design relies on + +Verified against `posit-dev/with-connect@main`: + +- Exports exactly two env vars to the wrapped command: `CONNECT_SERVER` and + `CONNECT_API_KEY`. +- Bootstraps a single admin-privileged API key via JWT; it does **not** create + named PAM users. The bootstrapped username is assigned by Connect — derive it + at runtime, never hardcode it. +- CLI `--license` takes a file path (default `./rstudio-connect.lic`). The + Action's `license` input takes license *contents* and writes the file itself. +- Does nothing with `requirements.txt` / manifests. +- Default `version` is `release`; supports `--config`/`config-file`, `-e`/`env`, + `--port`. +- Tool runtime needs Python 3.13 / `uv`, satisfied via `uvx` or the Action. This + is independent of the test matrix: the wrapped `pytest` runs in the CI job's + own Python (3.9/3.10), so there is no Python-version regression. + +## vetiver-python changes + +### Test fixture — `vetiver/tests/test_rsconnect.py` + +- Read `CONNECT_SERVER` and `CONNECT_API_KEY` from `os.environ` instead of the + hardcoded `http://localhost:3939` and the `rsconnect_api_keys.json` file. +- Use the single `with-connect` admin key. Remove `get_key`, `rsc_from_key`, + `rsc_fs_from_key`, and the JSON helpers; the `rsc_short` fixture builds its + `BoardRsConnect` from the one env-provided key. The current test already uses + only one user (`susan`), so a single user covers everything it verifies. +- Do not hardcode a username. Derive it at runtime via + `rsc.get_user()["username"]` and build the pin name as `f"{username}/model"`; + update the corresponding `pin_read`, `deploy_connect(pin_name=...)`, and + `content_search` title lookup. +- The `Authorization: Key ...` header and prediction assertions + (`response.iloc[0, 0] == 44.47`, `len(response) == 100`) are unchanged. + +### Local dev — `Makefile` + +- Remove `dev-start`, `dev`, the `RSC_API_KEYS` variable, and the `dump_api_keys` + invocation. +- `make test-rsc` runs the `with-connect` CLI, e.g. + `uvx --from git+https://github.com/posit-dev/with-connect@ with-connect -- pytest vetiver -m 'rsc_test'`. + License resolves from `./rstudio-connect.lic` per the CLI default. +- `make test` (the `not rsc_test and not docker` run) is unchanged. + +### CI — `.github/workflows/tests.yml` and `weekly.yml` + +- In each Connect job, replace the `docker compose up --build -d` + `make dev` + block with the Action, pinned to a release tag or commit SHA: + + ```yaml + - uses: posit-dev/with-connect@ + with: + version: # release, or a pinned version in the weekly matrix + license: ${{ secrets.RSC_LICENSE }} # existing secret = license CONTENTS + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' + ``` + + The `pip freeze > requirements.txt` step stays because the test deploys with + `extra_files=["requirements.txt"]` so Connect can rebuild the model env; this is + unrelated to `with-connect`. +- The weekly matrix keeps its version combinations: vetiver `pypi` × rsconnect + `main`, and vetiver `main` × rsconnect `main`. Only the harness invocation + changes; the rsconnect-version install steps stay. These jobs are the canary. +- Add a failure notification (issue/Slack) on the weekly canary. + +### Deletions + +- `docker-compose.yml` +- `script/setup-rsconnect/` (`dump_api_keys.py`, `add-users.sh`, `users.txt`, + `rstudio-connect.gcfg`) +- `vetiver/tests/rsconnect_api_keys.json` + +## rsconnect-python changes + +### Keep and re-home `tests/test_main_system_caches.py` + +This is rsconnect's own integration test for the `system caches` CLI, unrelated +to vetiver, that currently freeloads on the vetiver harness (it reads +`vetiver-testing/rsconnect_api_keys.json` and runs inside the `test-dev-connect` +job). It must survive the harness removal: + +- It needs two privilege levels — an admin (can list/delete caches) and a + non-admin publisher (must be denied) — so it cannot use only the `with-connect` + admin key. +- Refactor it to read `CONNECT_SERVER`/`CONNECT_API_KEY` from env, and add a + fixture that uses the admin key to create a publisher user via the Connect API + and mint that user's key at runtime (replacing static `get_key("susan")`). +- Give rsconnect its own `with-connect`-based CI job (Action, pinned) that runs + `pytest tests/test_main_system_caches.py`. Confirm during implementation that + the cache add/remove host commands (`ADD_CACHE_COMMAND`/`RM_CACHE_COMMAND`) can + still target the `with-connect`-managed container. + +### Relabel the `actions.py` compatibility shim + +`actions.py` lines 281–442 (`validate_extra_files`, `validate_entry_point`, +`deploy_app`) are kept and reframed as a supported compatibility entry point used +by vetiver: + +- `deploy_python_fastapi` (line 446, called by vetiver's `deploy_connect`) does + `return deploy_app(...)`, and `deploy_app` calls the two `validate_*` functions + — all within this block. `deploy_app` has no other caller. These functions are + load-bearing for vetiver and have no `DeprecationWarning` or changelog notice, + so they are treated as a maintained shim, not dead code. +- Rewrite the block-marker comments (lines 281–285, 441–443) to describe this as + a supported compatibility entry point for `vetiver-python`, dropping the + "deprecated … will be removed in the future" language. +- Optional cleanup: have `deploy_app` call the `bundle.py` `validate_*` functions + instead of the local copies, so vetiver deploys stop emitting spurious "This + method has been moved and will be deprecated" warnings on every call. + +The identically-named `validate_extra_files` / `validate_entry_point` in +`bundle.py` (lines ~1774, ~1845) are the active versions used by the `main.py` +CLI and must not be altered. The `actions.py` copies are reachable only through +`deploy_app`. + +### Deletions + +- `vetiver-testing/` (entire directory). +- `tests/test_vetiver_pins.py`. +- `conftest.py`: the `--vetiver` option and `pytest.mark.vetiver` skip logic. +- `pyproject.toml`: the `vetiver` marker definition and the `vetiver-testing/` + ruff exclude. +- `.github/workflows/main.yml`: the `test-dev-connect` job (replaced by the + rsconnect-own `system caches` job above, which no longer installs vetiver). +- Root `docker-compose.yml` and the `dev` / `dev-stop` recipes in `Justfile` + (lines ~62–71). Their only consumers are the vetiver dev/test path and the + `test-dev-connect` job. The separate `integration-testing/docker-compose.yml` + is unrelated and stays. + +## Sequencing + +The two repos' changes are independent and may land in either order; vetiver +keeps working against rsconnect `main` throughout. Suggested intra-repo order for +rsconnect to keep CI green at every commit: + +1. Update `conftest.py` to drop the `--vetiver` option/marker logic. +2. Refactor and re-home `test_main_system_caches.py` onto the new `with-connect` + job (create publisher user via admin API; env-var creds). +3. Delete `tests/test_vetiver_pins.py`. +4. Update `pyproject.toml` (drop `vetiver` marker + ruff exclude). +5. Replace the `test-dev-connect` job in `main.yml`. +6. Delete the `dev`/`dev-stop` Justfile recipes and root `docker-compose.yml`. +7. Delete `vetiver-testing/`. +8. Relabel the `actions.py` shim comments. + +## Verification items + +- Confirm `with-connect`'s default Connect image/config enables Python content + execution (vetiver's test deploys a FastAPI app). If not, enable via the + Action's `config-file` or `-e`. +- Confirm the bootstrapped admin user can write pins to its own namespace and that + `rsc.get_user()["username"]` returns the expected value. +- Confirm creating a non-admin publisher user via the Connect API with the + bootstrap admin key works, for `test_main_system_caches.py`. +- Confirm the `system caches` host commands can still add/remove caches in the + `with-connect`-managed container. +- Pin a specific `with-connect` Action ref (tag or SHA) and CLI ref; record an + update strategy. + +## Risks / accepted tradeoffs + +- rsconnect PRs lose direct vetiver-breakage signal; it moves to vetiver's weekly + CI, with a failure notification to compensate. +- New external dependency on `posit-dev/with-connect`, mitigated by pinning to a + tag/SHA. Both repos rely on Docker either way. +- vetiver remains coupled to rsconnect's internal `deploy_python_fastapi` shim. + Removing that coupling is out of scope and would require a real deprecation + cycle plus migrating vetiver to the `rsconnect` CLI. From c6691829c16c1e5211f3f4130cc25349343f772f Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:44:35 -0400 Subject: [PATCH 02/15] docs: implementation plan for with-connect adoption --- .../2026-06-30-vetiver-with-connect-tests.md | 447 ++++++++++++++++++ 1 file changed, 447 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md diff --git a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md new file mode 100644 index 00000000..460dc2f6 --- /dev/null +++ b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md @@ -0,0 +1,447 @@ +# vetiver-python: adopt with-connect for Connect tests — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace vetiver's bespoke Connect-in-Docker test harness with `posit-dev/with-connect`, so the Connect integration test reads credentials from the environment and the repo no longer carries its own docker-compose/user-bootstrap scripts. + +**Architecture:** `with-connect` boots a licensed Connect container, bootstraps a single admin API key, and exports `CONNECT_SERVER` + `CONNECT_API_KEY` to the wrapped command. The test fixture reads those env vars and derives the pin namespace from the authenticated user. CI uses the `with-connect` GitHub Action; local dev uses the `with-connect` CLI via `uvx`. + +**Tech Stack:** Python, pytest, pins, rsconnect-python, Docker, `uv`/`uvx`, GitHub Actions. + +## Global Constraints + +- vetiver supports Python 3.9+; the Connect test job runs on Python 3.9. (`with-connect`'s own 3.13 requirement is satisfied by `uvx`/the Action and does not affect the test interpreter.) +- `with-connect` exposes **only** `CONNECT_SERVER` and `CONNECT_API_KEY` to a wrapped command. Do not rely on any other injected env var, on named users (`susan`/`derek`/`admin`), or on a JSON keys file. +- Never hardcode a Connect username; derive it at runtime via the pins API. +- Pin `with-connect` to commit `0783dabdd24e360e985a4588ce1239c3dc31c542` (no release tags exist yet). Verify this is still the desired ref at execution time with `gh api repos/posit-dev/with-connect/commits/main -q .sha`. +- A valid `rstudio-connect.lic` must be present in the repo root for local runs (gitignored). CI passes the license via the `RSC_LICENSE` secret. +- The Connect test deploys with `extra_files=["requirements.txt"]`, so a `requirements.txt` snapshot of the test env must exist in the working directory before the test runs. + +--- + +### Task 1: Refactor `test_rsconnect.py` to env-var credentials + single user + +**Files:** +- Modify: `vetiver/tests/test_rsconnect.py` (full rewrite of fixtures/helpers) + +**Interfaces:** +- Consumes: `CONNECT_SERVER`, `CONNECT_API_KEY` from the environment (provided by `with-connect`). +- Produces: a single `rsc_admin` board fixture and a `username` fixture; no JSON-key helpers remain. + +- [ ] **Step 1: Replace the module so credentials are read lazily and the user is derived at runtime** + +Rationale for lazy reads: pytest imports every test module during collection even for `make test` (which deselects `rsc_test` by marker). Module-level `os.environ["CONNECT_SERVER"]` would raise `KeyError` during that collection. Use `os.environ.get(...)` at module scope and `pytest.skip(...)` inside the fixtures. + +Replace the entire contents of `vetiver/tests/test_rsconnect.py` with: + +```python +import os + +import numpy as np +import pandas as pd +import pytest +import sklearn + +import pins +from pins.boards import BoardRsConnect +from pins.rsconnect.api import RsConnectApi +from pins.rsconnect.fs import RsConnectFs +from rsconnect.api import RSConnectClient, RSConnectServer + +import vetiver + +RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") +RSC_API_KEY = os.environ.get("CONNECT_API_KEY") + +pytestmark = pytest.mark.rsc_test # noqa + + +def _require_connect(): + if not RSC_SERVER_URL or not RSC_API_KEY: + pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") + + +def _api(): + return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) + + +def rsc_delete_user_content(rsc): + guid = rsc.get_user()["guid"] + content = rsc.get_content(owner_guid=guid) + for entry in content: + rsc.delete_content_item(entry["guid"]) + + +@pytest.fixture(scope="function") +def username(): + _require_connect() + return _api().get_user()["username"] + + +@pytest.fixture(scope="function") +def rsc_admin(): + # tears down content after each test + _require_connect() + fs = RsConnectFs(_api()) + rsc_delete_user_content(fs.api) + yield BoardRsConnect("", fs, allow_pickle_read=True) + rsc_delete_user_content(fs.api) + + +def test_deploy(rsc_admin, username): + np.random.seed(500) + + # Load data, model + X_df, y = vetiver.mock.get_mock_data() + model = vetiver.mock.get_mock_model().fit(X_df, y) + + pin_name = f"{username}/model" + v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) + + board = pins.board_connect( + server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True + ) + + vetiver.vetiver_pin_write(board=board, model=v) + connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) + assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) + + vetiver.deploy_connect( + connect_server=connect_server, + board=board, + pin_name=pin_name, + title="testapi", + extra_files=["requirements.txt"], + ) + + # get url of where content lives + client = RSConnectClient(connect_server) + dicts = client.content_search() + rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) + content_url = rsc_api[0].get("content_url") + + h = {"Authorization": f"Key {RSC_API_KEY}"} + + endpoint = vetiver.vetiver_endpoint(content_url + "/predict") + response = vetiver.predict(endpoint, X_df, headers=h) + + assert isinstance(response, pd.DataFrame), response + assert response.iloc[0, 0] == 44.47 + assert len(response) == 100 +``` + +- [ ] **Step 2: Verify the normal (non-Connect) test suite still collects and runs** + +This confirms the lazy env reads didn't break collection of the module for the default run. + +Run: `pytest -m 'not rsc_test and not docker' -q` +Expected: PASS (no `KeyError` during collection; `test_rsconnect.py::test_deploy` is deselected by the marker). + +- [ ] **Step 3: Verify the Connect test passes under with-connect** + +Requires Docker + `rstudio-connect.lic` in the repo root. + +Run: +```bash +pip freeze > requirements.txt +uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' -v +``` +Expected: PASS — `test_deploy` deploys the model and asserts `response.iloc[0, 0] == 44.47`. + +- [ ] **Step 4: Commit** + +```bash +git add vetiver/tests/test_rsconnect.py +git commit -m "test: read Connect creds from env, derive pin user at runtime" +``` + +--- + +### Task 2: Point `make test-rsc` at with-connect and remove the dev harness targets + +**Files:** +- Modify: `Makefile` + +**Interfaces:** +- Consumes: nothing new. +- Produces: a `test-rsc` target that no longer depends on `dev`/`dev-start`/`dev-stop` or `RSC_API_KEYS`. + +- [ ] **Step 1: Remove the `RSC_API_KEYS` variable** + +Delete this line near the top of `Makefile`: + +```make +RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json +``` + +- [ ] **Step 2: Replace the `test-rsc` target** + +Change: + +```make +test-rsc: clean-test + pytest +``` + +to: + +```make +test-rsc: clean-test + pip freeze > requirements.txt + uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- pytest vetiver -m 'rsc_test' +``` + +- [ ] **Step 3: Remove the `dev`, `dev-start`, `dev-stop`, and `$(RSC_API_KEYS)` targets** + +Delete these blocks from `Makefile`: + +```make +dev: vetiver/tests/rsconnect_api_keys.json + +dev-start: + docker compose up -d + docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh + # curl fails with error 52 without a short sleep.... + sleep 5 + curl -s --retry 10 --retry-connrefused http://localhost:3939 + +dev-stop: + docker compose down + rm -f $(RSC_API_KEYS) +``` + +and + +```make +$(RSC_API_KEYS): dev-start + python script/setup-rsconnect/dump_api_keys.py $@ +``` + +- [ ] **Step 4: Update the `help` target text** + +Delete these three lines from the `help` recipe: + +```make + @echo "dev - generate Connect API keys" + @echo "dev-start - start up development Connect in Docker" + @echo "dev-stop - stop Connect dev container" +``` + +- [ ] **Step 5: Verify the Makefile is well-formed and `test-rsc` runs** + +Run: `make -n test-rsc` +Expected: prints the `pip freeze` + `uvx ... with-connect -- pytest` commands with no `make` errors. + +Run (Docker + license present): `make test-rsc` +Expected: PASS (same green result as Task 1, Step 3). + +- [ ] **Step 6: Commit** + +```bash +git add Makefile +git commit -m "build: run rsc tests via with-connect, drop bespoke dev targets" +``` + +--- + +### Task 3: Convert the `test-connect` CI job to the with-connect Action + +**Files:** +- Modify: `.github/workflows/tests.yml` (the `test-connect` job) + +**Interfaces:** +- Consumes: `secrets.RSC_LICENSE` (license contents). +- Produces: a `test-connect` job that needs no `docker-compose.yml` or `make dev`. + +- [ ] **Step 1: Replace the `run Connect` + `Run tests` steps** + +In the `test-connect` job, replace these two steps: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + pip freeze > requirements.txt + make dev + cat requirements.txt + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + + # NOTE: edited to run checks for python package + - name: Run tests + run: | + pytest vetiver -m 'rsc_test' +``` + +with: + +```yaml + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' +``` + +Note: keep the existing `Install dependencies` step (it still installs `.[dev]` and the vetiver build) so `pip freeze` captures the deployable environment. + +- [ ] **Step 2: Lint the workflow YAML** + +Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/tests.yml'))"` +Expected: no output (valid YAML). + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/tests.yml +git commit -m "ci: run Connect tests via with-connect action" +``` + +--- + +### Task 4: Convert the weekly workflow jobs + +**Files:** +- Modify: `.github/workflows/weekly.yml` + +**Interfaces:** +- Consumes: `secrets.RSC_LICENSE`. +- Produces: the two rsconnect jobs run via the Action; the two pins jobs no longer start Connect. + +- [ ] **Step 1: Remove Connect setup from the two `*_pins_main` jobs** + +`vetiver_main_pins_main` and `vetiver_pypi_pins_main` run `make test` (which is `-m 'not rsc_test and not docker'`), so they do not need Connect. In **both** jobs, delete the `run Connect` step: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + make dev + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} +``` + +Leave their final `make test` step intact. + +- [ ] **Step 2: Convert the two `*_rsconnect_latest` jobs to the Action** + +In `vetiver_pypi_rsconnect_latest` and `vetiver_latest_rsconnect_latest`, replace the `run Connect` + `Run Tests` steps: + +```yaml + - name: run Connect + run: | + docker compose up --build -d + make dev + env: + RSC_LICENSE: ${{ secrets.RSC_LICENSE }} + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + + - name: Run Tests + run: | + pytest -m 'rsc_test' +``` + +with: + +```yaml + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver -m 'rsc_test' +``` + +These jobs already run `pip freeze > requirements.txt` in their `Install dependencies` step; the line inside `command:` re-snapshots after all installs and is harmless. The rsconnect-from-`main` install steps stay unchanged — these jobs remain the reverse-dependency canary. + +- [ ] **Step 3: Add a failure notification to the canary jobs** + +Append this step to **both** `*_rsconnect_latest` jobs so rsconnect breakage is surfaced: + +```yaml + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) +``` + +- [ ] **Step 4: Lint the workflow YAML** + +Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/weekly.yml'))"` +Expected: no output (valid YAML). + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/weekly.yml +git commit -m "ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs" +``` + +--- + +### Task 5: Delete the bespoke harness files + +**Files:** +- Delete: `docker-compose.yml` +- Delete: `script/setup-rsconnect/dump_api_keys.py`, `add-users.sh`, `users.txt`, `rstudio-connect.gcfg` (the whole `script/setup-rsconnect/` directory) +- Delete: `vetiver/tests/rsconnect_api_keys.json` (if present / untracked) + +- [ ] **Step 1: Confirm nothing else references the harness paths** + +Run: +```bash +grep -rn "setup-rsconnect\|rsconnect_api_keys\|docker-compose\|docker compose" \ + --include='*.py' --include='*.yml' --include='*.yaml' --include='Makefile' . \ + | grep -v 'docs/superpowers' +``` +Expected: no hits outside `docs/superpowers/` (the design/plan docs). If `script/setup-docker/` appears, that is the *Docker* test path (`test-docker` job) and is unrelated — leave it. + +- [ ] **Step 2: Delete the files** + +Run: +```bash +git rm docker-compose.yml +git rm -r script/setup-rsconnect +rm -f vetiver/tests/rsconnect_api_keys.json +``` + +- [ ] **Step 3: Verify the default test suite still passes** + +Run: `pytest -m 'not rsc_test and not docker' -q` +Expected: PASS. + +- [ ] **Step 4: Verify the Connect test still passes end-to-end** + +Run: `make test-rsc` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "chore: remove bespoke Connect test harness (replaced by with-connect)" +``` + +--- + +## Self-Review notes + +- Spec "vetiver-python changes" → Tasks 1–5 cover the fixture, Makefile, `tests.yml`, `weekly.yml`, and all deletions. +- The spec's `requirements.txt` requirement is honored in Task 1 Step 3, Task 2 Step 2, and the `command:` blocks in Tasks 3–4. +- `with-connect` ref is pinned to one SHA used identically in Tasks 1–4 (`0783dabdd24e360e985a4588ce1239c3dc31c542`). +- The lazy-env-read detail (Task 1) prevents a collection-time regression in the default `pytest` run. From f262ceb059960802a5e69634a7e6184565da9a41 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 14:53:57 -0400 Subject: [PATCH 03/15] chore: gitignore local rstudio-connect.lic --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index c546a16a..884efd2c 100644 --- a/.gitignore +++ b/.gitignore @@ -149,3 +149,6 @@ NOTES.md /.quarto/ /.luarc.json + +# Local Connect license for with-connect integration tests +rstudio-connect.lic From f0be51c2c7c3a7f3e5023be241288573b06448d6 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:19:34 -0400 Subject: [PATCH 04/15] test: read Connect creds from env, derive pin user at runtime Replaces the bespoke docker-compose credential setup (hardcoded localhost:3939 + JSON key file with named users) with env-var credentials provided by with-connect. Credentials are read lazily at module scope via os.environ.get() to avoid KeyError during normal test collection when CONNECT_SERVER / CONNECT_API_KEY are not set. Additional fixes needed for compatibility with rsconnect-python 1.29.x and FastAPI 0.138.x / pydantic v2: - content_search() renamed to content_list() - new=True required to avoid stale AppStore GUID across fresh containers - content_url may include a trailing slash; strip before appending /predict - Content-Type: application/json required; modern FastAPI/pydantic v2 no longer accepts JSON body without explicit content-type header --- vetiver/tests/test_rsconnect.py | 76 +++++++++++++++------------------ 1 file changed, 35 insertions(+), 41 deletions(-) diff --git a/vetiver/tests/test_rsconnect.py b/vetiver/tests/test_rsconnect.py index adeafae3..81897193 100644 --- a/vetiver/tests/test_rsconnect.py +++ b/vetiver/tests/test_rsconnect.py @@ -1,40 +1,31 @@ +import os + +import numpy as np +import pandas as pd import pytest -import json import sklearn -import pins -import pandas as pd -import numpy as np +import pins from pins.boards import BoardRsConnect from pins.rsconnect.api import RsConnectApi from pins.rsconnect.fs import RsConnectFs -from rsconnect.api import RSConnectServer, RSConnectClient +from rsconnect.api import RSConnectClient, RSConnectServer import vetiver -RSC_SERVER_URL = "http://localhost:3939" -RSC_KEYS_FNAME = "vetiver/tests/rsconnect_api_keys.json" +RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") +RSC_API_KEY = os.environ.get("CONNECT_API_KEY") pytestmark = pytest.mark.rsc_test # noqa -def get_key(name): - with open(RSC_KEYS_FNAME) as f: - api_key = json.load(f)[name] - return api_key - +def _require_connect(): + if not RSC_SERVER_URL or not RSC_API_KEY: + pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") -def rsc_from_key(name): - with open(RSC_KEYS_FNAME) as f: - api_key = json.load(f)[name] - return RsConnectApi(RSC_SERVER_URL, api_key) - -def rsc_fs_from_key(name): - - rsc = rsc_from_key(name) - - return RsConnectFs(rsc) +def _api(): + return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) def rsc_delete_user_content(rsc): @@ -45,52 +36,55 @@ def rsc_delete_user_content(rsc): @pytest.fixture(scope="function") -def rsc_short(): - # tears down content after each test - fs_susan = rsc_fs_from_key("susan") +def username(): + _require_connect() + return _api().get_user()["username"] - # delete any content that might already exist - rsc_delete_user_content(fs_susan.api) - yield BoardRsConnect( - "", fs_susan, allow_pickle_read=True - ) # fs_susan.ls to list content - - rsc_delete_user_content(fs_susan.api) +@pytest.fixture(scope="function") +def rsc_admin(): + # tears down content after each test + _require_connect() + fs = RsConnectFs(_api()) + rsc_delete_user_content(fs.api) + yield BoardRsConnect("", fs, allow_pickle_read=True) + rsc_delete_user_content(fs.api) -def test_deploy(rsc_short): +def test_deploy(rsc_admin, username): np.random.seed(500) # Load data, model X_df, y = vetiver.mock.get_mock_data() model = vetiver.mock.get_mock_model().fit(X_df, y) - v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name="susan/model") + pin_name = f"{username}/model" + v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) board = pins.board_connect( - server_url=RSC_SERVER_URL, api_key=get_key("susan"), allow_pickle_read=True + server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True ) vetiver.vetiver_pin_write(board=board, model=v) - connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=get_key("susan")) - assert isinstance(board.pin_read("susan/model"), sklearn.dummy.DummyRegressor) + connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) + assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) vetiver.deploy_connect( connect_server=connect_server, board=board, - pin_name="susan/model", + pin_name=pin_name, title="testapi", extra_files=["requirements.txt"], + new=True, ) # get url of where content lives client = RSConnectClient(connect_server) - dicts = client.content_search() + dicts = client.content_list() rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) - content_url = rsc_api[0].get("content_url") + content_url = rsc_api[0].get("content_url").rstrip("/") - h = {"Authorization": f'Key {get_key("susan")}'} + h = {"Authorization": f"Key {RSC_API_KEY}", "Content-Type": "application/json"} endpoint = vetiver.vetiver_endpoint(content_url + "/predict") response = vetiver.predict(endpoint, X_df, headers=h) From 563b498b788df78b7c5e5eead0cc9c4955d7efae Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:30:15 -0400 Subject: [PATCH 05/15] fix: set Content-Type in predict() for DataFrame input; unmask integration test --- vetiver/server.py | 3 ++- vetiver/tests/test_rsconnect.py | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/vetiver/server.py b/vetiver/server.py index a5be4207..2c6d5679 100644 --- a/vetiver/server.py +++ b/vetiver/server.py @@ -394,8 +394,9 @@ def predict(endpoint, data: Union[dict, pd.DataFrame, pd.Series], **kw) -> pd.Da # TO DO: dispatch if isinstance(data, pd.DataFrame): + headers = {"Content-Type": "application/json", **kw.pop("headers", {})} response = requester.post( - endpoint, data=data.to_json(orient="records"), **kw + endpoint, data=data.to_json(orient="records"), headers=headers, **kw ) # TO DO: httpx deprecating data in favor of content for TestClient elif isinstance(data, pd.Series): response = requester.post(endpoint, json=[data.to_dict()], **kw) diff --git a/vetiver/tests/test_rsconnect.py b/vetiver/tests/test_rsconnect.py index 81897193..520b2b45 100644 --- a/vetiver/tests/test_rsconnect.py +++ b/vetiver/tests/test_rsconnect.py @@ -84,7 +84,7 @@ def test_deploy(rsc_admin, username): rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) content_url = rsc_api[0].get("content_url").rstrip("/") - h = {"Authorization": f"Key {RSC_API_KEY}", "Content-Type": "application/json"} + h = {"Authorization": f"Key {RSC_API_KEY}"} endpoint = vetiver.vetiver_endpoint(content_url + "/predict") response = vetiver.predict(endpoint, X_df, headers=h) From bb2f894bc857f4a163d557d7d0f9e8c3e15cf5f1 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:36:28 -0400 Subject: [PATCH 06/15] build: run rsc tests via with-connect, drop bespoke dev targets - remove RSC_API_KEYS variable and dev/dev-start/dev-stop/$(RSC_API_KEYS) targets - test-rsc now boots Connect via pinned with-connect (0783dab) - generate requirements.txt via uv run pip freeze, filtering editable/local vetiver entries and appending plain 'vetiver' so Connect can pip-install it - invoke pytest via uv run --with pytest on the specific test file to avoid collection errors in unrelated test files --- Makefile | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index 2350c8c2..bc5a2b01 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,6 @@ .PHONY: clean-pyc clean-build clean docs UNAME := $(shell uname) -RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json ifeq ($(UNAME), Darwin) BROWSER := open @@ -19,9 +18,6 @@ help: @echo "coverage - check code coverage quickly with the default Python" @echo "docs - generate HTML documentation, including API docs" @echo "release - package and upload a release" - @echo "dev - generate Connect API keys" - @echo "dev-start - start up development Connect in Docker" - @echo "dev-stop - stop Connect dev container" clean: clean-build clean-pyc clean-test @@ -47,7 +43,9 @@ test-pdb: clean-test pytest -m 'not rsc_test and not docker' --pdb test-rsc: clean-test - pytest + uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt && echo 'vetiver' >> requirements.txt + uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ + with-connect -- uv run --with pytest pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' coverage: coverage report -m @@ -60,21 +58,6 @@ docs doc documentation: release: dist twine upload dist/* -dev: vetiver/tests/rsconnect_api_keys.json - -dev-start: - docker compose up -d - docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh - # curl fails with error 52 without a short sleep.... - sleep 5 - curl -s --retry 10 --retry-connrefused http://localhost:3939 - -dev-stop: - docker compose down - rm -f $(RSC_API_KEYS) - typecheck: pyright -$(RSC_API_KEYS): dev-start - python script/setup-rsconnect/dump_api_keys.py $@ From 66b8b87cd4528c8bd6e250be4b405f2374c9d8d8 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:39:33 -0400 Subject: [PATCH 07/15] ci: run Connect tests via with-connect action --- .github/workflows/tests.yml | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 087b5621..e4858e68 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -58,20 +58,13 @@ jobs: uv pip install ".[dev]" uv pip install --upgrade git+https://github.com/rstudio/vetiver-python@${{ github.sha }} echo {{ github.sha }} - - name: run Connect - run: | - docker compose up --build -d - pip freeze > requirements.txt - make dev - cat requirements.txt - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - # NOTE: edited to run checks for python package - - name: Run tests - run: | - pytest vetiver -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze > requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' test-docker: name: "Test Docker" From 03c90d0cc0778d39a5aa6a7ddeb977820034c5a5 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:42:34 -0400 Subject: [PATCH 08/15] ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs --- .github/workflows/weekly.yml | 82 ++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 40 deletions(-) diff --git a/.github/workflows/weekly.yml b/.github/workflows/weekly.yml index a2c89ce6..01d72bf7 100644 --- a/.github/workflows/weekly.yml +++ b/.github/workflows/weekly.yml @@ -37,14 +37,6 @@ jobs: python -m pip install '.[all]' python -m pip install --upgrade git+https://github.com/rstudio/pins-python - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run tests run: | make test @@ -84,14 +76,6 @@ jobs: python -m pip install .[dev] python -m pip install --upgrade git+https://github.com/rstudio/pins-python - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run Tests run: | make test @@ -129,19 +113,28 @@ jobs: python -m pip install --upgrade pip python -m pip install .[dev] python -m pip install --upgrade git+https://github.com/rstudio/rsconnect-python - pip freeze > requirements.txt - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - - name: Run Tests - run: | - pytest -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' + + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) vetiver_latest_rsconnect_latest: name: 'vetiver latest, rsconnect latest' @@ -164,16 +157,25 @@ jobs: python -m pip install --upgrade pip python -m pip install '.[all]' python -m pip install --upgrade git+https://github.com/rstudio/rsconnect-python - pip freeze > requirements.txt - - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - name: Run Tests - run: | - pytest -m 'rsc_test' + - name: Run Connect integration tests + uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 + with: + license: ${{ secrets.RSC_LICENSE }} + command: | + pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt + pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' + + - name: Open issue on failure + if: failure() && github.event_name == 'schedule' + uses: actions/github-script@v7 + with: + script: | + github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Weekly canary failed: ${context.job}`, + body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + labels: ["ci-failure"] + }) From 5d343365e944c33f7478560cdb98524e889d59b3 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:46:34 -0400 Subject: [PATCH 09/15] chore: remove bespoke Connect test harness (replaced by with-connect) --- docker-compose.yml | 15 -------------- script/setup-rsconnect/add-users.sh | 1 - script/setup-rsconnect/dump_api_keys.py | 21 -------------------- script/setup-rsconnect/rstudio-connect.gcfg | 22 --------------------- script/setup-rsconnect/users.txt | 4 ---- vetiver/tests/rsconnect_api_keys.json | 1 - 6 files changed, 64 deletions(-) delete mode 100644 docker-compose.yml delete mode 100644 script/setup-rsconnect/add-users.sh delete mode 100644 script/setup-rsconnect/dump_api_keys.py delete mode 100644 script/setup-rsconnect/rstudio-connect.gcfg delete mode 100644 script/setup-rsconnect/users.txt delete mode 100644 vetiver/tests/rsconnect_api_keys.json diff --git a/docker-compose.yml b/docker-compose.yml deleted file mode 100644 index 4262845c..00000000 --- a/docker-compose.yml +++ /dev/null @@ -1,15 +0,0 @@ -services: - - rsconnect: - image: rstudio/rstudio-connect:latest - restart: always - ports: - - 3939:3939 - volumes: - - $PWD/script/setup-rsconnect/users.txt:/etc/users.txt - - $PWD/script/setup-rsconnect/rstudio-connect.gcfg:/etc/rstudio-connect/rstudio-connect.gcfg - # by default, mysql rounds to 4 decimals, but tests require more precision - privileged: true - environment: - RSTUDIO_CONNECT_HASTE: "enabled" - RSC_LICENSE: ${RSC_LICENSE} diff --git a/script/setup-rsconnect/add-users.sh b/script/setup-rsconnect/add-users.sh deleted file mode 100644 index 1df8c7f4..00000000 --- a/script/setup-rsconnect/add-users.sh +++ /dev/null @@ -1 +0,0 @@ -awk ' { system("useradd -m -s /bin/bash "$1); system("echo \""$1":"$2"\" | chpasswd"); system("id "$1) } ' /etc/users.txt diff --git a/script/setup-rsconnect/dump_api_keys.py b/script/setup-rsconnect/dump_api_keys.py deleted file mode 100644 index eebef59f..00000000 --- a/script/setup-rsconnect/dump_api_keys.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import sys - -from pins.rsconnect.api import _HackyConnect - -OUT_FILE = sys.argv[1] - - -def get_api_key(user, password, email): - rsc = _HackyConnect("http://localhost:3939") - - return rsc.create_first_admin(user, password, email).api_key - - -api_keys = { - "admin": get_api_key("admin", "admin0", "admin@example.com"), - "susan": get_api_key("susan", "susan", "susan@example.com"), - "derek": get_api_key("derek", "derek", "derek@example.com"), -} - -json.dump(api_keys, open(OUT_FILE, "w")) diff --git a/script/setup-rsconnect/rstudio-connect.gcfg b/script/setup-rsconnect/rstudio-connect.gcfg deleted file mode 100644 index 98d0add0..00000000 --- a/script/setup-rsconnect/rstudio-connect.gcfg +++ /dev/null @@ -1,22 +0,0 @@ -[Server] -DataDir = /data -Address = http://localhost:3939 - -[HTTP] -Listen = :3939 - -[Authentication] -Provider = pam - -[Authorization] -DefaultUserRole = publisher - -[Python] -Enabled = true -Executable = /opt/python/3.9.5/bin/python - -[RPackageRepository "CRAN"] -URL = https://packagemanager.rstudio.com/cran/__linux__/bionic/latest - -[RPackageRepository "RSPM"] -URL = https://packagemanager.rstudio.com/cran/__linux__/bionic/latest diff --git a/script/setup-rsconnect/users.txt b/script/setup-rsconnect/users.txt deleted file mode 100644 index dd4ec359..00000000 --- a/script/setup-rsconnect/users.txt +++ /dev/null @@ -1,4 +0,0 @@ -admin admin0 -test test -susan susan -derek derek diff --git a/vetiver/tests/rsconnect_api_keys.json b/vetiver/tests/rsconnect_api_keys.json deleted file mode 100644 index 594a58c3..00000000 --- a/vetiver/tests/rsconnect_api_keys.json +++ /dev/null @@ -1 +0,0 @@ -{"admin": "iv6yzMB4GwhQIXjI3lOWG2T21N02C9SU", "susan": "llQqii6bHqV5R9eMN7qpTvJqr3xtnRmy", "derek": "FkakZvdVk2ApxDVkbEn6u2g0eSul4tE6"} From 2962e3beb73da15a6f3423e0f4b3f85215db7b39 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 15:53:13 -0400 Subject: [PATCH 10/15] ci/build: document tests.yml freeze asymmetry; harden Makefile requirements.txt generation --- .github/workflows/tests.yml | 4 ++++ Makefile | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e4858e68..d91b54a1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -62,6 +62,10 @@ jobs: uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 with: license: ${{ secrets.RSC_LICENSE }} + # Plain freeze (unlike the Makefile/weekly jobs, which filter): here + # vetiver is installed from git+...@, so the freeze yields an + # installable git URL and Connect deploys THIS PR's vetiver. Filtering + # would deploy released vetiver and miss PR-level serving regressions. command: | pip freeze > requirements.txt pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' diff --git a/Makefile b/Makefile index bc5a2b01..1def8eb3 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,6 @@ .PHONY: clean-pyc clean-build clean docs UNAME := $(shell uname) - ifeq ($(UNAME), Darwin) BROWSER := open else @@ -43,7 +42,8 @@ test-pdb: clean-test pytest -m 'not rsc_test and not docker' --pdb test-rsc: clean-test - uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt && echo 'vetiver' >> requirements.txt + uv run pip freeze | grep -v '^vetiver' | grep -v '^-e ' > requirements.txt + echo 'vetiver' >> requirements.txt uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ with-connect -- uv run --with pytest pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' From 8f6042d03e645c05cfd617c3bc019012f87321ee Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 16:30:37 -0400 Subject: [PATCH 11/15] rm superpowers --- .../2026-06-30-vetiver-with-connect-tests.md | 447 ------------------ ...iver-connect-test-simplification-design.md | 223 --------- 2 files changed, 670 deletions(-) delete mode 100644 docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md delete mode 100644 docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md diff --git a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md b/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md deleted file mode 100644 index 460dc2f6..00000000 --- a/docs/superpowers/plans/2026-06-30-vetiver-with-connect-tests.md +++ /dev/null @@ -1,447 +0,0 @@ -# vetiver-python: adopt with-connect for Connect tests — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace vetiver's bespoke Connect-in-Docker test harness with `posit-dev/with-connect`, so the Connect integration test reads credentials from the environment and the repo no longer carries its own docker-compose/user-bootstrap scripts. - -**Architecture:** `with-connect` boots a licensed Connect container, bootstraps a single admin API key, and exports `CONNECT_SERVER` + `CONNECT_API_KEY` to the wrapped command. The test fixture reads those env vars and derives the pin namespace from the authenticated user. CI uses the `with-connect` GitHub Action; local dev uses the `with-connect` CLI via `uvx`. - -**Tech Stack:** Python, pytest, pins, rsconnect-python, Docker, `uv`/`uvx`, GitHub Actions. - -## Global Constraints - -- vetiver supports Python 3.9+; the Connect test job runs on Python 3.9. (`with-connect`'s own 3.13 requirement is satisfied by `uvx`/the Action and does not affect the test interpreter.) -- `with-connect` exposes **only** `CONNECT_SERVER` and `CONNECT_API_KEY` to a wrapped command. Do not rely on any other injected env var, on named users (`susan`/`derek`/`admin`), or on a JSON keys file. -- Never hardcode a Connect username; derive it at runtime via the pins API. -- Pin `with-connect` to commit `0783dabdd24e360e985a4588ce1239c3dc31c542` (no release tags exist yet). Verify this is still the desired ref at execution time with `gh api repos/posit-dev/with-connect/commits/main -q .sha`. -- A valid `rstudio-connect.lic` must be present in the repo root for local runs (gitignored). CI passes the license via the `RSC_LICENSE` secret. -- The Connect test deploys with `extra_files=["requirements.txt"]`, so a `requirements.txt` snapshot of the test env must exist in the working directory before the test runs. - ---- - -### Task 1: Refactor `test_rsconnect.py` to env-var credentials + single user - -**Files:** -- Modify: `vetiver/tests/test_rsconnect.py` (full rewrite of fixtures/helpers) - -**Interfaces:** -- Consumes: `CONNECT_SERVER`, `CONNECT_API_KEY` from the environment (provided by `with-connect`). -- Produces: a single `rsc_admin` board fixture and a `username` fixture; no JSON-key helpers remain. - -- [ ] **Step 1: Replace the module so credentials are read lazily and the user is derived at runtime** - -Rationale for lazy reads: pytest imports every test module during collection even for `make test` (which deselects `rsc_test` by marker). Module-level `os.environ["CONNECT_SERVER"]` would raise `KeyError` during that collection. Use `os.environ.get(...)` at module scope and `pytest.skip(...)` inside the fixtures. - -Replace the entire contents of `vetiver/tests/test_rsconnect.py` with: - -```python -import os - -import numpy as np -import pandas as pd -import pytest -import sklearn - -import pins -from pins.boards import BoardRsConnect -from pins.rsconnect.api import RsConnectApi -from pins.rsconnect.fs import RsConnectFs -from rsconnect.api import RSConnectClient, RSConnectServer - -import vetiver - -RSC_SERVER_URL = os.environ.get("CONNECT_SERVER") -RSC_API_KEY = os.environ.get("CONNECT_API_KEY") - -pytestmark = pytest.mark.rsc_test # noqa - - -def _require_connect(): - if not RSC_SERVER_URL or not RSC_API_KEY: - pytest.skip("CONNECT_SERVER and CONNECT_API_KEY must be set (run via with-connect)") - - -def _api(): - return RsConnectApi(RSC_SERVER_URL, RSC_API_KEY) - - -def rsc_delete_user_content(rsc): - guid = rsc.get_user()["guid"] - content = rsc.get_content(owner_guid=guid) - for entry in content: - rsc.delete_content_item(entry["guid"]) - - -@pytest.fixture(scope="function") -def username(): - _require_connect() - return _api().get_user()["username"] - - -@pytest.fixture(scope="function") -def rsc_admin(): - # tears down content after each test - _require_connect() - fs = RsConnectFs(_api()) - rsc_delete_user_content(fs.api) - yield BoardRsConnect("", fs, allow_pickle_read=True) - rsc_delete_user_content(fs.api) - - -def test_deploy(rsc_admin, username): - np.random.seed(500) - - # Load data, model - X_df, y = vetiver.mock.get_mock_data() - model = vetiver.mock.get_mock_model().fit(X_df, y) - - pin_name = f"{username}/model" - v = vetiver.VetiverModel(model=model, prototype_data=X_df, model_name=pin_name) - - board = pins.board_connect( - server_url=RSC_SERVER_URL, api_key=RSC_API_KEY, allow_pickle_read=True - ) - - vetiver.vetiver_pin_write(board=board, model=v) - connect_server = RSConnectServer(url=RSC_SERVER_URL, api_key=RSC_API_KEY) - assert isinstance(board.pin_read(pin_name), sklearn.dummy.DummyRegressor) - - vetiver.deploy_connect( - connect_server=connect_server, - board=board, - pin_name=pin_name, - title="testapi", - extra_files=["requirements.txt"], - ) - - # get url of where content lives - client = RSConnectClient(connect_server) - dicts = client.content_search() - rsc_api = list(filter(lambda x: x["title"] == "testapi", dicts)) - content_url = rsc_api[0].get("content_url") - - h = {"Authorization": f"Key {RSC_API_KEY}"} - - endpoint = vetiver.vetiver_endpoint(content_url + "/predict") - response = vetiver.predict(endpoint, X_df, headers=h) - - assert isinstance(response, pd.DataFrame), response - assert response.iloc[0, 0] == 44.47 - assert len(response) == 100 -``` - -- [ ] **Step 2: Verify the normal (non-Connect) test suite still collects and runs** - -This confirms the lazy env reads didn't break collection of the module for the default run. - -Run: `pytest -m 'not rsc_test and not docker' -q` -Expected: PASS (no `KeyError` during collection; `test_rsconnect.py::test_deploy` is deselected by the marker). - -- [ ] **Step 3: Verify the Connect test passes under with-connect** - -Requires Docker + `rstudio-connect.lic` in the repo root. - -Run: -```bash -pip freeze > requirements.txt -uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ - with-connect -- pytest vetiver/tests/test_rsconnect.py -m 'rsc_test' -v -``` -Expected: PASS — `test_deploy` deploys the model and asserts `response.iloc[0, 0] == 44.47`. - -- [ ] **Step 4: Commit** - -```bash -git add vetiver/tests/test_rsconnect.py -git commit -m "test: read Connect creds from env, derive pin user at runtime" -``` - ---- - -### Task 2: Point `make test-rsc` at with-connect and remove the dev harness targets - -**Files:** -- Modify: `Makefile` - -**Interfaces:** -- Consumes: nothing new. -- Produces: a `test-rsc` target that no longer depends on `dev`/`dev-start`/`dev-stop` or `RSC_API_KEYS`. - -- [ ] **Step 1: Remove the `RSC_API_KEYS` variable** - -Delete this line near the top of `Makefile`: - -```make -RSC_API_KEYS=vetiver/tests/rsconnect_api_keys.json -``` - -- [ ] **Step 2: Replace the `test-rsc` target** - -Change: - -```make -test-rsc: clean-test - pytest -``` - -to: - -```make -test-rsc: clean-test - pip freeze > requirements.txt - uvx --from git+https://github.com/posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 \ - with-connect -- pytest vetiver -m 'rsc_test' -``` - -- [ ] **Step 3: Remove the `dev`, `dev-start`, `dev-stop`, and `$(RSC_API_KEYS)` targets** - -Delete these blocks from `Makefile`: - -```make -dev: vetiver/tests/rsconnect_api_keys.json - -dev-start: - docker compose up -d - docker compose exec -T rsconnect bash < script/setup-rsconnect/add-users.sh - # curl fails with error 52 without a short sleep.... - sleep 5 - curl -s --retry 10 --retry-connrefused http://localhost:3939 - -dev-stop: - docker compose down - rm -f $(RSC_API_KEYS) -``` - -and - -```make -$(RSC_API_KEYS): dev-start - python script/setup-rsconnect/dump_api_keys.py $@ -``` - -- [ ] **Step 4: Update the `help` target text** - -Delete these three lines from the `help` recipe: - -```make - @echo "dev - generate Connect API keys" - @echo "dev-start - start up development Connect in Docker" - @echo "dev-stop - stop Connect dev container" -``` - -- [ ] **Step 5: Verify the Makefile is well-formed and `test-rsc` runs** - -Run: `make -n test-rsc` -Expected: prints the `pip freeze` + `uvx ... with-connect -- pytest` commands with no `make` errors. - -Run (Docker + license present): `make test-rsc` -Expected: PASS (same green result as Task 1, Step 3). - -- [ ] **Step 6: Commit** - -```bash -git add Makefile -git commit -m "build: run rsc tests via with-connect, drop bespoke dev targets" -``` - ---- - -### Task 3: Convert the `test-connect` CI job to the with-connect Action - -**Files:** -- Modify: `.github/workflows/tests.yml` (the `test-connect` job) - -**Interfaces:** -- Consumes: `secrets.RSC_LICENSE` (license contents). -- Produces: a `test-connect` job that needs no `docker-compose.yml` or `make dev`. - -- [ ] **Step 1: Replace the `run Connect` + `Run tests` steps** - -In the `test-connect` job, replace these two steps: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - pip freeze > requirements.txt - make dev - cat requirements.txt - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - # NOTE: edited to run checks for python package - - name: Run tests - run: | - pytest vetiver -m 'rsc_test' -``` - -with: - -```yaml - - name: Run Connect integration tests - uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 - with: - license: ${{ secrets.RSC_LICENSE }} - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' -``` - -Note: keep the existing `Install dependencies` step (it still installs `.[dev]` and the vetiver build) so `pip freeze` captures the deployable environment. - -- [ ] **Step 2: Lint the workflow YAML** - -Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/tests.yml'))"` -Expected: no output (valid YAML). - -- [ ] **Step 3: Commit** - -```bash -git add .github/workflows/tests.yml -git commit -m "ci: run Connect tests via with-connect action" -``` - ---- - -### Task 4: Convert the weekly workflow jobs - -**Files:** -- Modify: `.github/workflows/weekly.yml` - -**Interfaces:** -- Consumes: `secrets.RSC_LICENSE`. -- Produces: the two rsconnect jobs run via the Action; the two pins jobs no longer start Connect. - -- [ ] **Step 1: Remove Connect setup from the two `*_pins_main` jobs** - -`vetiver_main_pins_main` and `vetiver_pypi_pins_main` run `make test` (which is `-m 'not rsc_test and not docker'`), so they do not need Connect. In **both** jobs, delete the `run Connect` step: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} -``` - -Leave their final `make test` step intact. - -- [ ] **Step 2: Convert the two `*_rsconnect_latest` jobs to the Action** - -In `vetiver_pypi_rsconnect_latest` and `vetiver_latest_rsconnect_latest`, replace the `run Connect` + `Run Tests` steps: - -```yaml - - name: run Connect - run: | - docker compose up --build -d - make dev - env: - RSC_LICENSE: ${{ secrets.RSC_LICENSE }} - GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} - - - name: Run Tests - run: | - pytest -m 'rsc_test' -``` - -with: - -```yaml - - name: Run Connect integration tests - uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 - with: - license: ${{ secrets.RSC_LICENSE }} - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' -``` - -These jobs already run `pip freeze > requirements.txt` in their `Install dependencies` step; the line inside `command:` re-snapshots after all installs and is harmless. The rsconnect-from-`main` install steps stay unchanged — these jobs remain the reverse-dependency canary. - -- [ ] **Step 3: Add a failure notification to the canary jobs** - -Append this step to **both** `*_rsconnect_latest` jobs so rsconnect breakage is surfaced: - -```yaml - - name: Open issue on failure - if: failure() && github.event_name == 'schedule' - uses: actions/github-script@v7 - with: - script: | - github.rest.issues.create({ - owner: context.repo.owner, - repo: context.repo.repo, - title: `Weekly canary failed: ${context.job}`, - body: `The weekly Connect canary failed on run ${context.runId}. This often means a change in rsconnect-python \`main\` broke vetiver. See https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, - labels: ["ci-failure"] - }) -``` - -- [ ] **Step 4: Lint the workflow YAML** - -Run: `python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/weekly.yml'))"` -Expected: no output (valid YAML). - -- [ ] **Step 5: Commit** - -```bash -git add .github/workflows/weekly.yml -git commit -m "ci: weekly Connect jobs via with-connect; drop Connect from pins-only jobs" -``` - ---- - -### Task 5: Delete the bespoke harness files - -**Files:** -- Delete: `docker-compose.yml` -- Delete: `script/setup-rsconnect/dump_api_keys.py`, `add-users.sh`, `users.txt`, `rstudio-connect.gcfg` (the whole `script/setup-rsconnect/` directory) -- Delete: `vetiver/tests/rsconnect_api_keys.json` (if present / untracked) - -- [ ] **Step 1: Confirm nothing else references the harness paths** - -Run: -```bash -grep -rn "setup-rsconnect\|rsconnect_api_keys\|docker-compose\|docker compose" \ - --include='*.py' --include='*.yml' --include='*.yaml' --include='Makefile' . \ - | grep -v 'docs/superpowers' -``` -Expected: no hits outside `docs/superpowers/` (the design/plan docs). If `script/setup-docker/` appears, that is the *Docker* test path (`test-docker` job) and is unrelated — leave it. - -- [ ] **Step 2: Delete the files** - -Run: -```bash -git rm docker-compose.yml -git rm -r script/setup-rsconnect -rm -f vetiver/tests/rsconnect_api_keys.json -``` - -- [ ] **Step 3: Verify the default test suite still passes** - -Run: `pytest -m 'not rsc_test and not docker' -q` -Expected: PASS. - -- [ ] **Step 4: Verify the Connect test still passes end-to-end** - -Run: `make test-rsc` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "chore: remove bespoke Connect test harness (replaced by with-connect)" -``` - ---- - -## Self-Review notes - -- Spec "vetiver-python changes" → Tasks 1–5 cover the fixture, Makefile, `tests.yml`, `weekly.yml`, and all deletions. -- The spec's `requirements.txt` requirement is honored in Task 1 Step 3, Task 2 Step 2, and the `command:` blocks in Tasks 3–4. -- `with-connect` ref is pinned to one SHA used identically in Tasks 1–4 (`0783dabdd24e360e985a4588ce1239c3dc31c542`). -- The lazy-env-read detail (Task 1) prevents a collection-time regression in the default `pytest` run. diff --git a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md b/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md deleted file mode 100644 index feda0813..00000000 --- a/docs/superpowers/specs/2026-06-30-vetiver-connect-test-simplification-design.md +++ /dev/null @@ -1,223 +0,0 @@ -# Simplifying the vetiver ↔ Connect integration-testing relationship - -**Date:** 2026-06-30 -**Status:** Approved design, pending implementation plan -**Repos affected:** `vetiver-python`, `rsconnect-python` - -## Problem - -The integration-testing relationship between `vetiver-python` and -`rsconnect-python` is both inverted and duplicated: - -- The real dependency is one-way: `vetiver-python` declares - `rsconnect-python>=1.11.0` as a production dependency and calls - `rsconnect.actions.deploy_python_fastapi` / `rsconnect.api.RSConnectServer` / - `RSConnectClient` directly. -- Despite not depending on vetiver, `rsconnect-python` carries vetiver-specific - test baggage: a `vetiver-testing/` harness, `tests/test_vetiver_pins.py`, a - `--vetiver` pytest marker, and a nightly `test-dev-connect` CI job that installs - vetiver and deploys a model. -- Both repos independently maintain a near-identical bespoke Connect-in-Docker - harness (`docker-compose.yml`, `setup-rsconnect/` with `dump_api_keys.py`, - `add-users.sh`, `users.txt`, `rstudio-connect.gcfg`). - -## Goal - -vetiver owns and runs all vetiver↔Connect integration tests in its own repo; -`rsconnect-python` carries no vetiver-specific test code. Both repos use -[`with-connect`](https://github.com/posit-dev/with-connect) — the preferred -shared tool for running Connect in Docker during tests — instead of a hand-rolled -harness. - -## Target architecture - -``` -rsconnect-python vetiver-python - ├ tests/test_main_system_caches.py ├ vetiver/tests/test_rsconnect.py - │ (own integration test; kept) │ (reads env creds, single admin user) - ├ CI: own with-connect job; ├ CI: posit-dev/with-connect Action (pinned) - │ creates publisher user via admin API ├ local: `make test-rsc` → uvx with-connect - └ actions.py compatibility shim └ deletes docker-compose.yml, - (deploy_python_fastapi for vetiver) script/setup-rsconnect/, dump_api_keys.py - -Deleted from rsconnect-python: vetiver-testing/, tests/test_vetiver_pins.py, -the --vetiver marker, the nightly test-dev-connect job, root docker-compose.yml, -and the dev/dev-stop Justfile recipes. -``` - -The reverse-dependency canary ("does new rsconnect break vetiver?") lives in -vetiver's weekly CI matrix, which already installs `rsconnect` from `main` and -runs the integration tests. rsconnect PRs do not get that signal directly; a -failure notification on the weekly canary is recommended so rsconnect maintainers -learn of breakage without watching the schedule. - -### `with-connect` behavior the design relies on - -Verified against `posit-dev/with-connect@main`: - -- Exports exactly two env vars to the wrapped command: `CONNECT_SERVER` and - `CONNECT_API_KEY`. -- Bootstraps a single admin-privileged API key via JWT; it does **not** create - named PAM users. The bootstrapped username is assigned by Connect — derive it - at runtime, never hardcode it. -- CLI `--license` takes a file path (default `./rstudio-connect.lic`). The - Action's `license` input takes license *contents* and writes the file itself. -- Does nothing with `requirements.txt` / manifests. -- Default `version` is `release`; supports `--config`/`config-file`, `-e`/`env`, - `--port`. -- Tool runtime needs Python 3.13 / `uv`, satisfied via `uvx` or the Action. This - is independent of the test matrix: the wrapped `pytest` runs in the CI job's - own Python (3.9/3.10), so there is no Python-version regression. - -## vetiver-python changes - -### Test fixture — `vetiver/tests/test_rsconnect.py` - -- Read `CONNECT_SERVER` and `CONNECT_API_KEY` from `os.environ` instead of the - hardcoded `http://localhost:3939` and the `rsconnect_api_keys.json` file. -- Use the single `with-connect` admin key. Remove `get_key`, `rsc_from_key`, - `rsc_fs_from_key`, and the JSON helpers; the `rsc_short` fixture builds its - `BoardRsConnect` from the one env-provided key. The current test already uses - only one user (`susan`), so a single user covers everything it verifies. -- Do not hardcode a username. Derive it at runtime via - `rsc.get_user()["username"]` and build the pin name as `f"{username}/model"`; - update the corresponding `pin_read`, `deploy_connect(pin_name=...)`, and - `content_search` title lookup. -- The `Authorization: Key ...` header and prediction assertions - (`response.iloc[0, 0] == 44.47`, `len(response) == 100`) are unchanged. - -### Local dev — `Makefile` - -- Remove `dev-start`, `dev`, the `RSC_API_KEYS` variable, and the `dump_api_keys` - invocation. -- `make test-rsc` runs the `with-connect` CLI, e.g. - `uvx --from git+https://github.com/posit-dev/with-connect@ with-connect -- pytest vetiver -m 'rsc_test'`. - License resolves from `./rstudio-connect.lic` per the CLI default. -- `make test` (the `not rsc_test and not docker` run) is unchanged. - -### CI — `.github/workflows/tests.yml` and `weekly.yml` - -- In each Connect job, replace the `docker compose up --build -d` + `make dev` - block with the Action, pinned to a release tag or commit SHA: - - ```yaml - - uses: posit-dev/with-connect@ - with: - version: # release, or a pinned version in the weekly matrix - license: ${{ secrets.RSC_LICENSE }} # existing secret = license CONTENTS - command: | - pip freeze > requirements.txt - pytest vetiver -m 'rsc_test' - ``` - - The `pip freeze > requirements.txt` step stays because the test deploys with - `extra_files=["requirements.txt"]` so Connect can rebuild the model env; this is - unrelated to `with-connect`. -- The weekly matrix keeps its version combinations: vetiver `pypi` × rsconnect - `main`, and vetiver `main` × rsconnect `main`. Only the harness invocation - changes; the rsconnect-version install steps stay. These jobs are the canary. -- Add a failure notification (issue/Slack) on the weekly canary. - -### Deletions - -- `docker-compose.yml` -- `script/setup-rsconnect/` (`dump_api_keys.py`, `add-users.sh`, `users.txt`, - `rstudio-connect.gcfg`) -- `vetiver/tests/rsconnect_api_keys.json` - -## rsconnect-python changes - -### Keep and re-home `tests/test_main_system_caches.py` - -This is rsconnect's own integration test for the `system caches` CLI, unrelated -to vetiver, that currently freeloads on the vetiver harness (it reads -`vetiver-testing/rsconnect_api_keys.json` and runs inside the `test-dev-connect` -job). It must survive the harness removal: - -- It needs two privilege levels — an admin (can list/delete caches) and a - non-admin publisher (must be denied) — so it cannot use only the `with-connect` - admin key. -- Refactor it to read `CONNECT_SERVER`/`CONNECT_API_KEY` from env, and add a - fixture that uses the admin key to create a publisher user via the Connect API - and mint that user's key at runtime (replacing static `get_key("susan")`). -- Give rsconnect its own `with-connect`-based CI job (Action, pinned) that runs - `pytest tests/test_main_system_caches.py`. Confirm during implementation that - the cache add/remove host commands (`ADD_CACHE_COMMAND`/`RM_CACHE_COMMAND`) can - still target the `with-connect`-managed container. - -### Relabel the `actions.py` compatibility shim - -`actions.py` lines 281–442 (`validate_extra_files`, `validate_entry_point`, -`deploy_app`) are kept and reframed as a supported compatibility entry point used -by vetiver: - -- `deploy_python_fastapi` (line 446, called by vetiver's `deploy_connect`) does - `return deploy_app(...)`, and `deploy_app` calls the two `validate_*` functions - — all within this block. `deploy_app` has no other caller. These functions are - load-bearing for vetiver and have no `DeprecationWarning` or changelog notice, - so they are treated as a maintained shim, not dead code. -- Rewrite the block-marker comments (lines 281–285, 441–443) to describe this as - a supported compatibility entry point for `vetiver-python`, dropping the - "deprecated … will be removed in the future" language. -- Optional cleanup: have `deploy_app` call the `bundle.py` `validate_*` functions - instead of the local copies, so vetiver deploys stop emitting spurious "This - method has been moved and will be deprecated" warnings on every call. - -The identically-named `validate_extra_files` / `validate_entry_point` in -`bundle.py` (lines ~1774, ~1845) are the active versions used by the `main.py` -CLI and must not be altered. The `actions.py` copies are reachable only through -`deploy_app`. - -### Deletions - -- `vetiver-testing/` (entire directory). -- `tests/test_vetiver_pins.py`. -- `conftest.py`: the `--vetiver` option and `pytest.mark.vetiver` skip logic. -- `pyproject.toml`: the `vetiver` marker definition and the `vetiver-testing/` - ruff exclude. -- `.github/workflows/main.yml`: the `test-dev-connect` job (replaced by the - rsconnect-own `system caches` job above, which no longer installs vetiver). -- Root `docker-compose.yml` and the `dev` / `dev-stop` recipes in `Justfile` - (lines ~62–71). Their only consumers are the vetiver dev/test path and the - `test-dev-connect` job. The separate `integration-testing/docker-compose.yml` - is unrelated and stays. - -## Sequencing - -The two repos' changes are independent and may land in either order; vetiver -keeps working against rsconnect `main` throughout. Suggested intra-repo order for -rsconnect to keep CI green at every commit: - -1. Update `conftest.py` to drop the `--vetiver` option/marker logic. -2. Refactor and re-home `test_main_system_caches.py` onto the new `with-connect` - job (create publisher user via admin API; env-var creds). -3. Delete `tests/test_vetiver_pins.py`. -4. Update `pyproject.toml` (drop `vetiver` marker + ruff exclude). -5. Replace the `test-dev-connect` job in `main.yml`. -6. Delete the `dev`/`dev-stop` Justfile recipes and root `docker-compose.yml`. -7. Delete `vetiver-testing/`. -8. Relabel the `actions.py` shim comments. - -## Verification items - -- Confirm `with-connect`'s default Connect image/config enables Python content - execution (vetiver's test deploys a FastAPI app). If not, enable via the - Action's `config-file` or `-e`. -- Confirm the bootstrapped admin user can write pins to its own namespace and that - `rsc.get_user()["username"]` returns the expected value. -- Confirm creating a non-admin publisher user via the Connect API with the - bootstrap admin key works, for `test_main_system_caches.py`. -- Confirm the `system caches` host commands can still add/remove caches in the - `with-connect`-managed container. -- Pin a specific `with-connect` Action ref (tag or SHA) and CLI ref; record an - update strategy. - -## Risks / accepted tradeoffs - -- rsconnect PRs lose direct vetiver-breakage signal; it moves to vetiver's weekly - CI, with a failure notification to compensate. -- New external dependency on `posit-dev/with-connect`, mitigated by pinning to a - tag/SHA. Both repos rely on Docker either way. -- vetiver remains coupled to rsconnect's internal `deploy_python_fastapi` shim. - Removing that coupling is out of scope and would require a real deprecation - cycle plus migrating vetiver to the `rsconnect` CLI. From a045c8f50e6dbf1942af85783a4d01ff97e7b1b6 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 17:58:35 -0400 Subject: [PATCH 12/15] fix: repair pre-existing CI rot so the suite passes - data: replace removed pkg_resources with importlib.resources (unblocks Docker and no-extras jobs that import vetiver.data) - monitor: use pd.api.types.is_datetime64_any_dtype; np.issubdtype now raises on pandas string dtype. Drop the now-unused numpy import - test_monitor: pandas dropped the uppercase 'S' freq alias; use 's' - test_prepare_docker: the two test functions were decorated as fixtures (since 238e967) so they never ran and broke collection under newer pytest. Restore them as tests and add an autouse fixture that restores the shared fsspec LocalFileSystem protocol they mutate, preventing the leak that otherwise corrupted test_write_app - test_server: test_endpoint_adds must send Content-Type: application/json (same root cause as the predict() fix in this branch) - ci: pin fastapi<0.116 in the pydantic v1 job; fastapi>=0.116 unconditionally imports IncEx from pydantic v2 --- .github/workflows/tests.yml | 4 +++- vetiver/data/__init__.py | 7 ++++--- vetiver/monitor.py | 3 +-- vetiver/tests/test_monitor.py | 2 +- vetiver/tests/test_prepare_docker.py | 13 +++++++++++-- vetiver/tests/test_server.py | 6 +++++- 6 files changed, 25 insertions(+), 10 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index d91b54a1..b60002d1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -126,7 +126,9 @@ jobs: run: | python -m pip install --upgrade pip python -m pip install -e .[dev] - python -m pip install 'pydantic<2.0.0' + # FastAPI >=0.116 unconditionally imports `IncEx` from pydantic v2, + # so pin to the last release that still supports pydantic v1. + python -m pip install 'pydantic<2.0.0' 'fastapi<0.116' - name: Run tests run: | diff --git a/vetiver/data/__init__.py b/vetiver/data/__init__.py index 6e4bfd32..48ca60a7 100644 --- a/vetiver/data/__init__.py +++ b/vetiver/data/__init__.py @@ -11,10 +11,11 @@ def __dir__(): def _load_data_csv(name): import pandas as pd - import pkg_resources + from importlib.resources import files - fname = pkg_resources.resource_filename("vetiver.data", f"{name}.csv") - return pd.read_csv(fname) + fname = files("vetiver.data").joinpath(f"{name}.csv") + with fname.open("rb") as f: + return pd.read_csv(f) def __getattr__(name): diff --git a/vetiver/monitor.py b/vetiver/monitor.py index 24cd8a61..82a1505c 100644 --- a/vetiver/monitor.py +++ b/vetiver/monitor.py @@ -1,6 +1,5 @@ import plotly.express as px import pandas as pd -import numpy as np from datetime import timedelta @@ -55,7 +54,7 @@ def compute_metrics( df = data[[truth, estimate, date_var]].copy() - if not np.issubdtype(df[date_var], np.datetime64): + if not pd.api.types.is_datetime64_any_dtype(df[date_var]): df[date_var] = pd.to_datetime(df[date_var]) df = df.set_index(date_var).sort_index() diff --git a/vetiver/tests/test_monitor.py b/vetiver/tests/test_monitor.py index b0f875e0..4a9386a2 100644 --- a/vetiver/tests/test_monitor.py +++ b/vetiver/tests/test_monitor.py @@ -9,7 +9,7 @@ import pytest -rng = pd.date_range("1/1/2012", periods=10, freq="S") +rng = pd.date_range("1/1/2012", periods=10, freq="s") new = dict(x=range(len(rng)), y=range(len(rng))) df = pd.DataFrame(new, index=rng) td = timedelta(seconds=2) diff --git a/vetiver/tests/test_prepare_docker.py b/vetiver/tests/test_prepare_docker.py index 0b50f35f..97ed2f58 100644 --- a/vetiver/tests/test_prepare_docker.py +++ b/vetiver/tests/test_prepare_docker.py @@ -32,7 +32,17 @@ def create_vetiver_model(): return vetiver.VetiverModel(model.fit(X, y), "model", prototype_data=X) -@pytest.fixture(scope="module") +@pytest.fixture(autouse=True) +def _restore_fs_protocol(): + # board.fs is a cached fsspec LocalFileSystem singleton; the tests below + # mutate its `protocol` to simulate cloud backends. Restore it afterwards + # so the mutation does not leak into other tests (e.g. test_write_app). + fs = pins.board_temp(allow_pickle_read=True).fs + original = fs.protocol + yield + fs.protocol = original + + def test_warning_if_no_protocol(create_vetiver_model): with pytest.warns(UserWarning): board = pins.board_temp(allow_pickle_read=True) @@ -49,7 +59,6 @@ def test_warning_if_no_protocol(create_vetiver_model): (("gcs", "gs"), "gcsfs"), ], ) -@pytest.fixture(scope="module") def test_get_board_pkgs(prot, output, create_vetiver_model): board = pins.board_temp(allow_pickle_read=True) board.fs.protocol = prot diff --git a/vetiver/tests/test_server.py b/vetiver/tests/test_server.py index 6d953787..74ada611 100644 --- a/vetiver/tests/test_server.py +++ b/vetiver/tests/test_server.py @@ -167,7 +167,11 @@ def data() -> pd.DataFrame: def test_endpoint_adds(client, data): - response = client.post("/sum/", data=data.to_json(orient="records")) + response = client.post( + "/sum/", + data=data.to_json(orient="records"), + headers={"Content-Type": "application/json"}, + ) assert response.status_code == 200 assert response.json() == {"sum": [40, 320, 220, 7.8, 5.24, 32.00, 0, 2, 8, 8]} From 62d0e6a4f73b499880318132c78649e0bccfc518 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Tue, 30 Jun 2026 18:11:09 -0400 Subject: [PATCH 13/15] ci: unblock Connect (UV_PYTHON) and Docs (griffe<2) jobs - test-connect: the with-connect action runs `uv tool install` for a tool that requires Python >=3.13, but inherits UV_PYTHON=3.9 from setup-uv. Repoint UV_PYTHON=3.13 before the action so the tool installs; the freeze and pytest command still run in the 3.9 VIRTUAL_ENV (the deploy manifest must record a Python version Connect actually has) - docs: pin griffe<2; quartodoc 0.11.x breaks against the griffe 2.x rewrite (imports griffe.loader, passes the removed parse_numpy kwarg), which was failing `make docs` --- .github/workflows/tests.yml | 6 ++++++ pyproject.toml | 3 +++ 2 files changed, 9 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index b60002d1..254bcf32 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -58,6 +58,12 @@ jobs: uv pip install ".[dev]" uv pip install --upgrade git+https://github.com/rstudio/vetiver-python@${{ github.sha }} echo {{ github.sha }} + - name: Let with-connect use its own interpreter + # setup-uv pinned UV_PYTHON=3.9 for the vetiver env above, but the + # with-connect tool requires Python >=3.13 to install. Repoint UV_PYTHON + # so `uv tool install with-connect` resolves; the freeze/pytest command + # still runs in the 3.9 VIRTUAL_ENV created above. + run: echo "UV_PYTHON=3.13" >> "$GITHUB_ENV" - name: Run Connect integration tests uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 with: diff --git a/pyproject.toml b/pyproject.toml index 52365ba4..abf37965 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -67,6 +67,9 @@ dev = [ ] docs = [ "quartodoc", + # quartodoc 0.11.x is incompatible with the griffe 2.x rewrite + # (it imports griffe.loader / passes removed parse_numpy kwargs). + "griffe<2", # quarto render dependencies "jupyter" ] From fbf5db0f5b6b34a7a880904d81c4a3a100e6b0e3 Mon Sep 17 00:00:00 2001 From: edavidaja Date: Wed, 1 Jul 2026 09:35:37 -0400 Subject: [PATCH 14/15] ci(connect): run test-connect on Python 3.14 to match Connect image The with-connect Connect image provides only Python 3.14.6 as a Local environment. rsconnect records the client Python in the deploy manifest, so freezing under 3.9 made Connect fail to build the content ("no compatible Local environment with Python version 3.9.25"). Run the whole job on 3.14 so the freeze matches; 3.14 also satisfies with-connect's own requires-python (>=3.13), so the separate UV_PYTHON override is no longer needed. --- .github/workflows/tests.yml | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 254bcf32..abbee400 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -51,19 +51,18 @@ jobs: - name: Install uv and Python uses: astral-sh/setup-uv@v5 with: - python-version: "3.9" + # Match the Python the with-connect Connect image provides (3.14): + # rsconnect records the client Python in the deploy manifest and + # Connect must have a matching Local environment to build the + # content. 3.14 also satisfies the with-connect tool's own + # requires-python (>=3.13), so no separate UV_PYTHON override. + python-version: "3.14" - name: Install dependencies run: | uv pip install --upgrade pip uv pip install ".[dev]" uv pip install --upgrade git+https://github.com/rstudio/vetiver-python@${{ github.sha }} echo {{ github.sha }} - - name: Let with-connect use its own interpreter - # setup-uv pinned UV_PYTHON=3.9 for the vetiver env above, but the - # with-connect tool requires Python >=3.13 to install. Repoint UV_PYTHON - # so `uv tool install with-connect` resolves; the freeze/pytest command - # still runs in the 3.9 VIRTUAL_ENV created above. - run: echo "UV_PYTHON=3.13" >> "$GITHUB_ENV" - name: Run Connect integration tests uses: posit-dev/with-connect@0783dabdd24e360e985a4588ce1239c3dc31c542 with: From 6d52e761d680ea2c89d5060f82c012ae9ade53b3 Mon Sep 17 00:00:00 2001 From: "E. David Aja" Date: Wed, 1 Jul 2026 10:44:53 -0400 Subject: [PATCH 15/15] Update .github/workflows/weekly.yml Co-authored-by: Isabel Zimmerman <54685329+isabelizimm@users.noreply.github.com> --- .github/workflows/weekly.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/weekly.yml b/.github/workflows/weekly.yml index 01d72bf7..8f8acf6d 100644 --- a/.github/workflows/weekly.yml +++ b/.github/workflows/weekly.yml @@ -87,7 +87,7 @@ jobs: strategy: matrix: # Minimum and Maximum supported versions - python-version: ['3.9'] + python-version: ['3.14'] steps: - name: Get latest release with tag from GitHub API