From 2bedf4a70ac7533ecea96a142251392b5765c48a Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Sat, 31 Oct 2026 00:37:42 +0000 Subject: [PATCH 1/3] ref(devenv): Remove scripts/do.sh and scripts/lib.sh, slim .envrc .envrc mixed activation with diagnosis and repair: it sourced scripts/lib.sh, launched xcode-select, deleted shadowing executables, hard-failed on a node mismatch, and teed its own output to a tempfile. It now only sets environment variables and PATH, with one cheap `devenv sync` hint. - PATH order is explicit: global devenv, .venv/bin, .devenv/bin, node_modules/.bin, gcloud, homebrew. The venv outranks every other checkout directory, so the executable-deletion loop is gone; the package.json postinstall still drops node_modules/.bin/sentry for pnpm. - dotenv_if_exists watches .env while it's absent. - The node, node_modules, prek, and agent-skill checks move to devenv/checks/synced.py for `devenv doctor`. - The Make targets that dispatched through do.sh keep their names. The database ones go through tools/dev_db.py, which treats an existing database as success but surfaces every other createdb failure (the shell version ignored all of them). reset-db is one ordered invocation, so `make -j` can't interleave it. - devenv sync calls tools.dev_db directly instead of `make apply-migrations`. - CI: the dev env workflow now watches devenv/** and checks direnv activation; migrations-drift runs when tools/dev_db.py or the Makefile change. Co-Authored-By: Joshua Li <14209781+joshuarli@users.noreply.github.com> --- .agents/skills/setup-dev/SKILL.md | 22 +- .envrc | 233 ++---------------- .github/workflows/development-environment.yml | 9 + .github/workflows/migrations-drift.yml | 2 + Makefile | 44 +++- devenv/checks/synced.py | 59 +++++ devenv/sync.py | 2 +- scripts/do.sh | 29 --- scripts/lib.sh | 106 -------- tests/tools/test_dev_db.py | 102 ++++++++ tests/tools/test_devenv_checks.py | 70 ++++++ tools/dev_db.py | 86 +++++++ 12 files changed, 387 insertions(+), 377 deletions(-) create mode 100644 devenv/checks/synced.py delete mode 100755 scripts/do.sh delete mode 100755 scripts/lib.sh create mode 100644 tests/tools/test_dev_db.py create mode 100644 tests/tools/test_devenv_checks.py create mode 100644 tools/dev_db.py diff --git a/.agents/skills/setup-dev/SKILL.md b/.agents/skills/setup-dev/SKILL.md index d650a84e48f2..2b1286ce77f6 100644 --- a/.agents/skills/setup-dev/SKILL.md +++ b/.agents/skills/setup-dev/SKILL.md @@ -153,24 +153,9 @@ This is the longest step. Tell the user: > **This will take 10-20 minutes** on first run. It installs Python, Node, all pip/npm dependencies, and runs database migrations. Subsequent syncs are much faster (2-5 min). -### If direnv hangs +### Before the first sync -The `.envrc` runs Docker checks. If the Docker runtime isn't running, direnv will hang. Symptoms: - -``` -direnv: ([...]/direnv export zsh) is taking a while to execute. Use CTRL-C to give up. -``` - -**Fix**: Start the Docker runtime first: - -- OrbStack: Open OrbStack.app or `open -a OrbStack` -- Colima: `devenv colima start` - -Then `direnv allow` again. - -### Chicken-and-Egg Problem - -direnv checks node version and sentry installation. On a fresh setup, these don't exist yet, so direnv will fail. This is normal. **Bypass direnv and run devenv sync directly:** +`.envrc` only sets environment variables and PATH. On a fresh checkout it still loads and prints a `Run: devenv sync` hint. Run sync directly: ```bash ~/.local/share/sentry-devenv/bin/devenv sync @@ -182,7 +167,7 @@ Or if devenv is on PATH: devenv sync ``` -After sync completes, `direnv allow` should succeed. +After sync completes, `direnv allow` activates the environment. ### If devenv sync fails @@ -314,7 +299,6 @@ Each worktree has its own `.venv`. When you create a worktree with `git worktree | Symptom | Likely Cause | Fix | | ----------------------------------------------- | ----------------------------------------- | ---------------------------------------------------- | -| direnv hangs | Docker runtime not running | Start OrbStack / Colima | | `devenv sync` fails with version error | Outdated devenv | `devenv update` or reinstall | | `devservices up` says "colima not running" | Using OrbStack, script only checks Colima | Patch `devservices.py` — see Step 6 | | Server 500s on every page | DB not seeded | `.venv/bin/sentry upgrade --noinput` | diff --git a/.envrc b/.envrc index 274704dc0ac3..927fb69c1e5b 100644 --- a/.envrc +++ b/.envrc @@ -1,95 +1,10 @@ #!/not/executable/bash -# This is the .envrc for sentry, for use with direnv. -# It's responsible for enforcing a standard dev environment by checking as much state as possible, and either performing -# initialization (e.g. activating the venv) or giving recommendations on how to reach the desired state. -# It also sets useful environment variables. -# If you'd like to override or set any custom environment variables, this .envrc will read a .env file at the end. +# direnv activation for sentry: environment variables and PATH, nothing else. +# `devenv sync` installs and repairs the environment; `devenv doctor` diagnoses it. +# Put local overrides in a .env file. set -eu -# Upgrading Mac can uninstall the Command Line Tools, thus, removing our access to git -# The message talks about xcrun, however, we can use the lack of git as a way to know that we need this -# xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), -# missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun -if [ "$(uname -s)" == "Darwin" ] && [ ! -f "/Library/Developer/CommandLineTools/usr/bin/git" ]; then - echo -e "$(tput setaf 1)\nERROR: Complete the interactive installation (10+ mins) of the Command Line Tools.$(tput sgr0)" - xcode-select --install - return 1 -fi - -source "scripts/lib.sh" - -bold="$(tput bold)" -red="$(tput setaf 1)" -green="$(tput setaf 2)" -yellow="$(tput setaf 3)" -reset="$(tput sgr0)" - -# XXX: we can't trap bash EXIT, because it'll override direnv's finalizing routines. -# consequently, using "exit" anywhere will skip this notice from showing. -# so need to use set -e, and return 1. -trap notice ERR - -complete_success="yup" - - - -help_message() { - cat <&2 - complete_success="nope" -} - -die() { - echo -e "${red}${bold}FATAL: ${*}${reset}" >&2 - return 1 -} - -show_commands_info() { - echo -e "\n${red}Run the following commands to bring your environment up-to-date:" - for cmd in "${commands_to_run[@]}"; do - warn " ${red}$cmd" - done - echo "" -} - -### Environment ### - -commands_to_run=() - -# Always write stdout immediately. Very helpful for debugging export PYTHONUNBUFFERED=1 - -# make sure we don't have any conflicting PYTHONPATH unset PYTHONPATH # increase node's memory limit, required for our webpacking @@ -99,140 +14,32 @@ export NODE_OPTIONS="--max-old-space-size=5120" # Enable this by default for development envs (CI/deploys do not use envrc) export SENTRY_UI_HOT_RELOAD=1 -### You can override the exported variables with a .env file -# All exports should happen before here unless they're safeguarded (see devenv error reporting below) -if [ -f ".env" ]; then - info "Loading variables from .env" - dotenv -fi +# Watches .env even while it's absent, so creating it triggers a reload. +dotenv_if_exists -## Notify of reporting to Sentry -if [ -n "${SENTRY_DEVENV_NO_REPORT+x}" ]; then - debug "No development environment errors will be reported (since you've defined SENTRY_DEVENV_NO_REPORT)." - _SENTRY_LOG_FILE='' -else - # Since direnv traps the EXIT signal we place the temp file under /tmp for the odd time - # the script will use the EXIT path - _SENTRY_LOG_FILE=$(mktemp /tmp/sentry.envrc.out.$$.XXXXXX) - exec > >(tee "$_SENTRY_LOG_FILE") - exec 2>&1 - debug "Development errors will be reported to Sentry.io. If you wish to opt-out, set SENTRY_DEVENV_NO_REPORT as an env variable." +# Development environment errors are reported to Sentry.io. Set SENTRY_DEVENV_NO_REPORT to opt out. +if [ -z "${SENTRY_DEVENV_NO_REPORT+x}" ]; then export SENTRY_DEVSERVICES_DSN=https://23670f54c6254bfd9b7de106637808e9@o1.ingest.sentry.io/1492057 fi -# for deduplicating -PATH_rm "/opt/homebrew/bin" - -# homebrew must not be searched first, we need venv and devenv to take precedence -PATH_add "/opt/homebrew/bin" - -# PATH_add prepends, so entries listed earlier rank lower. npm ships a binary -# named `sentry`, which must not shadow the venv's. +# PATH_add prepends, so later entries take precedence. Homebrew is moved ahead +# of the system paths, but the checkout's tools must win over it. +PATH_rm /opt/homebrew/bin +PATH_add /opt/homebrew/bin +PATH_add /opt/homebrew/share/google-cloud-sdk/bin +# npm ships a binary named `sentry`, which must not shadow the venv's. PATH_add node_modules/.bin - -### Python ### +PATH_add .devenv/bin export VIRTUAL_ENV="${PWD}/.venv" PATH_add "${VIRTUAL_ENV}/bin" -if [ -f "${VIRTUAL_ENV}/bin/devenv" ]; then - DEVENV="${VIRTUAL_ENV}/bin/devenv" -else - DEVENV=devenv -fi - -export SENTRY_DEVENV_HOME="${SENTRY_DEVENV_HOME:-$XDG_DATA_HOME/sentry-devenv}" +# The global devenv, not the copy pinned in the venv. +export SENTRY_DEVENV_HOME="${SENTRY_DEVENV_HOME:-${XDG_DATA_HOME:-$HOME/.local/share}/sentry-devenv}" PATH_add "${SENTRY_DEVENV_HOME}/bin" -if ! command -v "$DEVENV" >/dev/null; then - die ' -Please install the devenv tool: -https://github.com/getsentry/devenv#install -' -fi - -if ! [ -x "${VIRTUAL_ENV}/bin/sentry" ]; then - warn "Your virtualenv is activated, but sentry doesn't seem to be installed." - commands_to_run+=("devenv sync") -fi - -# this is the standard repo-local bin root -PATH_add "${PWD}/.devenv/bin" - -# gcloud and friends -PATH_add "/opt/homebrew/share/google-cloud-sdk/bin" - -# pnpm prepends node_modules/.bin to PATH, even when the venv comes first here. -rm -f node_modules/.bin/sentry -hash -r -# Only remove shadowing executables once the expected command is installed. -if [ -x "${VIRTUAL_ENV}/bin/sentry" ]; then - while [ "$(command -v sentry)" != "${VIRTUAL_ENV}/bin/sentry" ]; do - sentry_path="$(command -v sentry || true)" - if [ "$(type -t sentry)" != "file" ] || [ ! -f "$sentry_path" ] || [ "$sentry_path" -ef "${VIRTUAL_ENV}/bin/sentry" ]; then - die "Expected sentry to resolve to ${VIRTUAL_ENV}/bin/sentry, got: ${sentry_path:-not found}" - fi - # Resolve parent directories, not the executable: unlinking a local shim - # is safe, but following a directory symlink could delete a global tool. - sentry_dir="$(cd -- "$(dirname -- "$sentry_path")" && pwd -P)" - case "$sentry_dir/" in - "$(pwd -P)/"*) ;; - *) die "Refusing to remove $sentry_path outside this checkout. Adjust PATH so ${VIRTUAL_ENV}/bin/sentry takes precedence." ;; - esac - info "Removing $sentry_path: it shadows ${VIRTUAL_ENV}/bin/sentry" - rm -f -- "$sentry_path" - hash -r - done +if ! has devenv; then + log_error "devenv isn't installed. See https://github.com/getsentry/devenv#install" +elif [ ! -x "${VIRTUAL_ENV}/bin/sentry" ]; then + log_error "sentry isn't installed in ${VIRTUAL_ENV}. Run: devenv sync" fi - -### prek ### - -debug "Checking prek..." - -if ! require prek; then - warn "Looks like you don't have prek installed." - commands_to_run+=("devenv sync") -fi - -### dotagents ### - -if [ -f "agents.toml" ] && { [ ! -s "agents.lock" ] || [ ! -d ".agents/skills" ] || [ -z "$(ls -A ".agents/skills" 2>/dev/null)" ]; }; then - warn "Agent skills not installed." - commands_to_run+=("devenv sync") -fi - -### Node ### - -if [ -f ".node-version" ]; then - debug "Checking node..." - - if [ "${SENTRY_DEVENV_SKIP_FRONTEND:-}" != "1" ]; then - if ! require node; then - die "You don't seem to have node installed. Please run devenv sync." - fi - - read -r node_version < ".node-version" - if [ "v${node_version}" != "$(node --version)" ]; then - die "Unexpected $(command -v node) version. Please run devenv sync." - fi - - if [ ! -d "node_modules" ]; then - die "You don't seem to have pnpm packages installed. Please run devenv sync." - fi - fi -fi - -# These are commands that can take a significant amount of time -if [ ${#commands_to_run[@]} -ne 0 ]; then - show_commands_info -fi - -if [ "${complete_success}" != "yup" ]; then - help_message - warn "\nPartial success. The virtualenv is active, however, you're not fully up-to-date (see messages above)." -else - echo "${green}${bold}SUCCESS!${reset}" -fi - -# Since we can't use an EXIT routine we need to guarantee we delete the file here -[ -z "$_SENTRY_LOG_FILE" ] || rm -f "$_SENTRY_LOG_FILE" diff --git a/.github/workflows/development-environment.yml b/.github/workflows/development-environment.yml index 72ffeead752a..75945b395f7d 100644 --- a/.github/workflows/development-environment.yml +++ b/.github/workflows/development-environment.yml @@ -12,6 +12,7 @@ on: - '.python-version' - '.envrc' - 'Brewfile' + - 'devenv/**' - 'scripts/**' - 'tools/**' - 'src/sentry/runner/commands/devserver.py' @@ -77,3 +78,11 @@ jobs: - name: devenv sync run: | devenv --nocoderoot sync + + - name: direnv activation + run: | + sudo apt-get update -qq + sudo apt-get install -y -qq direnv + direnv allow + direnv exec . bash -euc 'test "$(command -v sentry)" = "$PWD/.venv/bin/sentry"' 2>&1 | tee direnv.log + if grep -q "isn't installed" direnv.log; then exit 1; fi diff --git a/.github/workflows/migrations-drift.yml b/.github/workflows/migrations-drift.yml index f6fb2e81523d..53a93b2dbe1b 100644 --- a/.github/workflows/migrations-drift.yml +++ b/.github/workflows/migrations-drift.yml @@ -4,6 +4,8 @@ on: paths: - 'src/sentry/**/migrations/**.py' - 'tools/migrations/drop_pending_deletions.py' + - 'tools/dev_db.py' + - 'Makefile' - '.github/workflows/migrations-drift.yml' # Cancel in progress workflows on pull_requests. diff --git a/Makefile b/Makefile index 5ae653eb2238..b4df71e8b862 100644 --- a/Makefile +++ b/Makefile @@ -11,15 +11,41 @@ bootstrap: @echo "you probably want to run devenv sync to bring the" @echo "sentry dev environment up to date!" -build-platform-assets \ -clean \ -init-config \ -run-dependent-services \ -drop-db \ -create-db \ -apply-migrations \ -reset-db : - @./scripts/do.sh $@ +.PHONY: init-config run-dependent-services build-platform-assets clean create-db drop-db apply-migrations reset-db +init-config: + sentry init --dev --no-clobber + +run-dependent-services: + devservices up + +build-platform-assets: + @echo "--> Building platform assets" + python3 -m sentry.build._integration_docs + @# make sure this didn't silently do nothing + @test -f src/sentry/integration-docs/android.json + +clean: + @echo "--> Cleaning static cache" + rm -rf dist/* src/sentry/static/sentry/dist/* + @echo "--> Cleaning integration docs cache" + rm -rf src/sentry/integration-docs + @echo "--> Cleaning pyc files" + find . -name "*.pyc" -delete + @echo "--> Cleaning python build artifacts" + rm -rf build/ dist/ src/sentry/assets.json + +create-db: + @python3 -m tools.dev_db create + +drop-db: + @python3 -m tools.dev_db drop + +apply-migrations: + @python3 -m tools.dev_db migrate + +# One invocation keeps drop, migrate and createuser ordered under `make -j`. +reset-db: + @python3 -m tools.dev_db reset develop \ install-js-dev \ diff --git a/devenv/checks/synced.py b/devenv/checks/synced.py new file mode 100644 index 000000000000..9c5781a6654a --- /dev/null +++ b/devenv/checks/synced.py @@ -0,0 +1,59 @@ +from __future__ import annotations + +import os +import subprocess +import sys + +from devenv.lib import proc + +tags: set[str] = {"sentry"} +name = "sentry environment is synced" + +reporoot = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + + +def _problems() -> list[str]: + problems = [] + + if not os.access(f"{reporoot}/.venv/bin/sentry", os.X_OK): + problems.append("sentry isn't installed in .venv") + if not os.access(f"{reporoot}/.venv/bin/prek", os.X_OK): + problems.append("prek isn't installed in .venv") + + if os.path.exists(f"{reporoot}/agents.toml"): + lock = f"{reporoot}/agents.lock" + skills = f"{reporoot}/.agents/skills" + locked = os.path.isfile(lock) and os.path.getsize(lock) > 0 + installed = os.path.isdir(skills) and bool(os.listdir(skills)) + if not (locked and installed): + problems.append("agent skills aren't installed") + + if os.environ.get("SENTRY_DEVENV_SKIP_FRONTEND") != "1": + with open(f"{reporoot}/.node-version") as f: + expected = f"v{f.read().strip()}" + node = f"{reporoot}/.devenv/bin/node" + if not os.access(node, os.X_OK): + problems.append("node isn't installed in .devenv/bin") + else: + actual = subprocess.run((node, "--version"), capture_output=True, text=True).stdout + if actual.strip() != expected: + problems.append(f"node is {actual.strip() or 'broken'}, expected {expected}") + if not os.path.isdir(f"{reporoot}/node_modules"): + problems.append("pnpm packages aren't installed") + + return problems + + +def check() -> tuple[bool, str]: + problems = _problems() + if problems: + return False, "; ".join(problems) + return True, "" + + +def fix() -> tuple[bool, str]: + try: + proc.run((sys.executable, "-P", "-m", "devenv", "sync"), cwd=reporoot) + except RuntimeError as e: + return False, f"devenv sync failed: {e}" + return check() diff --git a/devenv/sync.py b/devenv/sync.py index 3becdd58229b..62d56bdd0e60 100644 --- a/devenv/sync.py +++ b/devenv/sync.py @@ -360,7 +360,7 @@ def main(context: dict[str, str]) -> int: ( ( "python migrations", - ("make", "apply-migrations"), + (f"{venv_dir}/bin/python", "-m", "tools.dev_db", "migrate"), {}, ), ), diff --git a/scripts/do.sh b/scripts/do.sh deleted file mode 100755 index 1602ad047a8c..000000000000 --- a/scripts/do.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/bash -# This script is an interface to any of the methods of lib.sh -# Call this script as "do.sh method_from_lib" to execute any function from that library -set -eu - -HERE="$( - cd "$(dirname "${BASH_SOURCE[0]}")" - pwd -P -)" - -if ! command -v devenv >/dev/null 2>&1; then - echo ' -Please install the devenv tool: -https://github.com/getsentry/devenv#install -' - exit 1 -fi - -if ! [[ "$VIRTUAL_ENV" -ef "${HERE}/../.venv" ]]; then - echo " -Your sentry virtualenv isn't activated. You need to successfully run 'direnv allow'. -" - exit 1 -fi - -# shellcheck disable=SC1090 -source "${HERE}/lib.sh" - -"$@" diff --git a/scripts/lib.sh b/scripts/lib.sh deleted file mode 100755 index 521b260a4720..000000000000 --- a/scripts/lib.sh +++ /dev/null @@ -1,106 +0,0 @@ -#!/bin/bash -# NOTE: This file is sourced in CI across different repos (e.g. snuba), -# thus, renaming this file or any functions can break CI! -# -# Module containing code shared across various shell scripts -# Execute functions from this module via the script do.sh -# shellcheck disable=SC2034 # Unused variables -# shellcheck disable=SC2001 # https://github.com/koalaman/shellcheck/wiki/SC2001 - -POSTGRES_CONTAINER="postgres-postgres-1" - -venv_name=".venv" - -# XDG paths' standardized defaults: -# (see https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables ) -export XDG_DATA_HOME="${XDG_DATA_HOME:-$HOME/.local/share}" -export XDG_CONFIG_HOME="${XDG_CONFIG_HOME:-$HOME/.config}" -export XDG_STATE_HOME="${XDG_STATE_HOME:-$HOME/.local/state}" -export XDG_DATA_DIRS="${XDG_DATA_DIRS:-/usr/local/share/:/usr/share/}" -export XDG_CONFIG_DIRS="${XDG_CONFIG_DIRS:-/etc/xdg}" -export XDG_CACHE_HOME="${XDG_CACHE_HOME:-$HOME/.cache}" -export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/var/run}" - - -# Check if a command is available -require() { - command -v "$1" >/dev/null 2>&1 -} - -sudo-askpass() { - if [ -z "${sudo-askpass-x}" ]; then - sudo --askpass "$@" - else - sudo "$@" - fi -} - -init-config() { - sentry init --dev --no-clobber -} - -run-dependent-services() { - devservices up -} - -create-db() { - container_name=${POSTGRES_CONTAINER} - echo "--> Creating 'sentry' database" - docker exec "${container_name}" createdb -h 127.0.0.1 -U postgres -E utf-8 sentry || true - echo "--> Creating 'control', 'region' and 'secondary' database" - docker exec "${container_name}" createdb -h 127.0.0.1 -U postgres -E utf-8 control || true - docker exec "${container_name}" createdb -h 127.0.0.1 -U postgres -E utf-8 region || true - docker exec "${container_name}" createdb -h 127.0.0.1 -U postgres -E utf-8 secondary || true -} - -apply-migrations() { - create-db - echo "--> Applying migrations" - sentry upgrade --noinput -} - -create-superuser() { - echo "--> Creating a superuser account" - if [[ -n "${GITHUB_ACTIONS+x}" ]]; then - sentry createuser --superuser --email foo@tbd.com --no-password --no-input - else - sentry createuser --superuser --email admin@sentry.io --password admin --no-input - echo "Password is admin." - fi -} - -build-platform-assets() { - echo "--> Building platform assets" - python3 -m sentry.build._integration_docs - # make sure this didn't silently do nothing - test -f src/sentry/integration-docs/android.json -} - -clean() { - echo "--> Cleaning static cache" - rm -rf dist/* src/sentry/static/sentry/dist/* - echo "--> Cleaning integration docs cache" - rm -rf src/sentry/integration-docs - echo "--> Cleaning pyc files" - find . -name "*.pyc" -delete - echo "--> Cleaning python build artifacts" - rm -rf build/ dist/ src/sentry/assets.json - echo "" -} - -drop-db() { - container_name=${POSTGRES_CONTAINER} - echo "--> Dropping existing 'sentry' database" - docker exec "${container_name}" dropdb --if-exists -h 127.0.0.1 -U postgres sentry - echo "--> Dropping 'control' and 'region' database" - docker exec "${container_name}" dropdb --if-exists -h 127.0.0.1 -U postgres control - docker exec "${container_name}" dropdb --if-exists -h 127.0.0.1 -U postgres region - docker exec "${container_name}" dropdb --if-exists -h 127.0.0.1 -U postgres secondary -} - -reset-db() { - drop-db - apply-migrations - create-superuser - echo 'Finished resetting database. To load mock data, run `./bin/load-mocks`' -} diff --git a/tests/tools/test_dev_db.py b/tests/tools/test_dev_db.py new file mode 100644 index 000000000000..aafa6271e32c --- /dev/null +++ b/tests/tools/test_dev_db.py @@ -0,0 +1,102 @@ +from __future__ import annotations + +import os +import subprocess +import sys +from collections.abc import Generator, Sequence +from typing import Any +from unittest import mock + +import pytest + +from tools import dev_db + + +class FakeRun: + def __init__(self, fail: dict[str, tuple[int, str]] | None = None) -> None: + self.calls: list[tuple[str, ...]] = [] + self.fail = fail or {} + + def __call__(self, cmd: Sequence[str], **kwargs: Any) -> subprocess.CompletedProcess[str]: + cmd = tuple(cmd) + self.calls.append(cmd) + returncode, stderr = self.fail.get(cmd[-1], (0, "")) + if returncode and kwargs.get("check"): + raise subprocess.CalledProcessError(returncode, cmd) + return subprocess.CompletedProcess(cmd, returncode, "", stderr) + + +@pytest.fixture +def fake_run() -> Generator[FakeRun]: + fake = FakeRun() + with mock.patch.object(subprocess, "run", fake), mock.patch.dict(os.environ): + os.environ.pop("GITHUB_ACTIONS", None) + yield fake + + +def _summary(calls: list[tuple[str, ...]]) -> list[tuple[str, ...]]: + out = [] + for cmd in calls: + if cmd[0] == "docker": + out.append((cmd[3], cmd[-1])) + else: + assert cmd[:3] == (sys.executable, "-m", "sentry") + out.append(cmd[3:5]) + return out + + +def test_migrate_creates_every_database_then_upgrades(fake_run: FakeRun) -> None: + assert dev_db.main(("migrate",)) == 0 + assert _summary(fake_run.calls) == [ + ("createdb", "sentry"), + ("createdb", "control"), + ("createdb", "region"), + ("createdb", "secondary"), + ("upgrade", "--noinput"), + ] + + +def test_reset_is_ordered(fake_run: FakeRun) -> None: + assert dev_db.main(("reset",)) == 0 + assert _summary(fake_run.calls) == [ + ("dropdb", "sentry"), + ("dropdb", "control"), + ("dropdb", "region"), + ("dropdb", "secondary"), + ("createdb", "sentry"), + ("createdb", "control"), + ("createdb", "region"), + ("createdb", "secondary"), + ("upgrade", "--noinput"), + ("createuser", "--superuser"), + ] + assert "admin@sentry.io" in fake_run.calls[-1] + + +def test_reset_in_ci_creates_passwordless_user(fake_run: FakeRun) -> None: + os.environ["GITHUB_ACTIONS"] = "true" + assert dev_db.main(("reset",)) == 0 + assert fake_run.calls[-1][-4:] == ("--email", "foo@tbd.com", "--no-password", "--no-input") + + +def test_create_tolerates_existing_database(fake_run: FakeRun) -> None: + fake_run.fail["sentry"] = ( + 1, + 'createdb: error: database creation failed: ERROR: database "sentry" already exists\n', + ) + assert dev_db.main(("create",)) == 0 + assert len(fake_run.calls) == 4 + + +def test_create_propagates_other_failures( + fake_run: FakeRun, capsys: pytest.CaptureFixture[str] +) -> None: + fake_run.fail["control"] = (1, "Error response from daemon: container is not running\n") + assert dev_db.main(("migrate",)) == 1 + assert _summary(fake_run.calls) == [("createdb", "sentry"), ("createdb", "control")] + assert "container is not running" in capsys.readouterr().err + + +def test_migrate_propagates_upgrade_failure(fake_run: FakeRun) -> None: + fake_run.fail["--noinput"] = (3, "") + assert dev_db.main(("migrate",)) == 3 diff --git a/tests/tools/test_devenv_checks.py b/tests/tools/test_devenv_checks.py new file mode 100644 index 000000000000..905e94449231 --- /dev/null +++ b/tests/tools/test_devenv_checks.py @@ -0,0 +1,70 @@ +from __future__ import annotations + +import importlib.util +import os +from collections.abc import Generator +from pathlib import Path +from types import ModuleType +from unittest import mock + +import pytest + +CHECK = Path(__file__).parents[2] / "devenv" / "checks" / "synced.py" + + +@pytest.fixture +def synced(tmp_path: Path) -> Generator[ModuleType]: + spec = importlib.util.spec_from_file_location("synced", CHECK) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + module.reporoot = str(tmp_path) + (tmp_path / ".node-version").write_text("26.12.1\n") + with mock.patch.dict(os.environ): + os.environ.pop("SENTRY_DEVENV_SKIP_FRONTEND", None) + yield module + + +def _executable(path: Path, body: str = "") -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(f"#!/bin/sh\n{body}\n") + os.chmod(path, 0o755) + + +def test_unsynced_checkout(synced: ModuleType) -> None: + ok, msg = synced.check() + assert not ok + assert msg == ( + "sentry isn't installed in .venv; prek isn't installed in .venv; " + "node isn't installed in .devenv/bin; pnpm packages aren't installed" + ) + + +def test_synced_checkout(synced: ModuleType, tmp_path: Path) -> None: + _executable(tmp_path / ".venv/bin/sentry") + _executable(tmp_path / ".venv/bin/prek") + _executable(tmp_path / ".devenv/bin/node", "echo v26.12.1") + (tmp_path / "node_modules").mkdir() + (tmp_path / "agents.toml").touch() + (tmp_path / "agents.lock").write_text("lock\n") + (tmp_path / ".agents/skills/some-skill").mkdir(parents=True) + assert synced.check() == (True, "") + + +def test_wrong_node_and_missing_skills(synced: ModuleType, tmp_path: Path) -> None: + _executable(tmp_path / ".venv/bin/sentry") + _executable(tmp_path / ".venv/bin/prek") + _executable(tmp_path / ".devenv/bin/node", "echo v20.0.0") + (tmp_path / "node_modules").mkdir() + (tmp_path / "agents.toml").touch() + assert synced.check() == ( + False, + "agent skills aren't installed; node is v20.0.0, expected v26.12.1", + ) + + +def test_skip_frontend(synced: ModuleType, tmp_path: Path) -> None: + os.environ["SENTRY_DEVENV_SKIP_FRONTEND"] = "1" + _executable(tmp_path / ".venv/bin/sentry") + _executable(tmp_path / ".venv/bin/prek") + assert synced.check() == (True, "") diff --git a/tools/dev_db.py b/tools/dev_db.py new file mode 100644 index 000000000000..fc068d353974 --- /dev/null +++ b/tools/dev_db.py @@ -0,0 +1,86 @@ +"""Create, drop, migrate, and reset the local development databases.""" + +from __future__ import annotations + +import argparse +import os +import shlex +import subprocess +import sys +from collections.abc import Sequence + +POSTGRES_CONTAINER = "postgres-postgres-1" +DATABASES = ("sentry", "control", "region", "secondary") + + +def _pg(*cmd: str) -> list[str]: + return ["docker", "exec", POSTGRES_CONTAINER, *cmd, "-h", "127.0.0.1", "-U", "postgres"] + + +def _sentry(*args: str) -> None: + # sys.executable keeps every step on the interpreter that runs this module. + subprocess.run((sys.executable, "-m", "sentry", *args), check=True) + + +def create() -> None: + for name in DATABASES: + print(f"--> Creating '{name}' database", flush=True) + proc = subprocess.run( + (*_pg("createdb"), "-E", "utf-8", name), capture_output=True, text=True + ) + if proc.returncode != 0 and f'database "{name}" already exists' not in proc.stderr: + sys.stderr.write(proc.stderr) + raise subprocess.CalledProcessError(proc.returncode, proc.args) + + +def drop() -> None: + for name in DATABASES: + print(f"--> Dropping '{name}' database", flush=True) + subprocess.run((*_pg("dropdb"), "--if-exists", name), check=True) + + +def migrate() -> None: + create() + print("--> Applying migrations", flush=True) + _sentry("upgrade", "--noinput") + + +def reset() -> None: + drop() + migrate() + print("--> Creating a superuser account", flush=True) + if "GITHUB_ACTIONS" in os.environ: + _sentry( + "createuser", "--superuser", "--email", "foo@tbd.com", "--no-password", "--no-input" + ) + else: + _sentry( + "createuser", + "--superuser", + "--email", + "admin@sentry.io", + "--password", + "admin", + "--no-input", + ) + print("Password is admin.") + print("Finished resetting database. To load mock data, run `./bin/load-mocks`") + + +COMMANDS = {"create": create, "drop": drop, "migrate": migrate, "reset": reset} + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("command", choices=COMMANDS) + args = parser.parse_args(argv) + try: + COMMANDS[args.command]() + except subprocess.CalledProcessError as e: + print(f"--> Failed: {shlex.join(e.cmd)}", file=sys.stderr) + return e.returncode or 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 53293b4821366440fc62e0b3ba21385b3f8d62cd Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Sat, 31 Oct 2026 00:46:07 +0000 Subject: [PATCH 2/3] fix(devenv): Fix mypy errors in dev_db and doctor check tests --- tests/tools/test_dev_db.py | 2 +- tests/tools/test_devenv_checks.py | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/tools/test_dev_db.py b/tests/tools/test_dev_db.py index aafa6271e32c..0adf55b24ad4 100644 --- a/tests/tools/test_dev_db.py +++ b/tests/tools/test_dev_db.py @@ -35,7 +35,7 @@ def fake_run() -> Generator[FakeRun]: def _summary(calls: list[tuple[str, ...]]) -> list[tuple[str, ...]]: - out = [] + out: list[tuple[str, ...]] = [] for cmd in calls: if cmd[0] == "docker": out.append((cmd[3], cmd[-1])) diff --git a/tests/tools/test_devenv_checks.py b/tests/tools/test_devenv_checks.py index 905e94449231..6d9096f5b216 100644 --- a/tests/tools/test_devenv_checks.py +++ b/tests/tools/test_devenv_checks.py @@ -18,9 +18,8 @@ def synced(tmp_path: Path) -> Generator[ModuleType]: assert spec is not None and spec.loader is not None module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module) - module.reporoot = str(tmp_path) (tmp_path / ".node-version").write_text("26.12.1\n") - with mock.patch.dict(os.environ): + with mock.patch.object(module, "reporoot", str(tmp_path)), mock.patch.dict(os.environ): os.environ.pop("SENTRY_DEVENV_SKIP_FRONTEND", None) yield module From 1b8eeca4f980b3975aec6bed50542f8ac614ee1e Mon Sep 17 00:00:00 2001 From: Joshua Li Date: Fri, 30 Oct 2026 22:38:30 -0700 Subject: [PATCH 3/3] ref: bump sentry-devenv to 1.31.0 1.31.0 includes getsentry/devenv#251, so repo-defined doctor checks, including the one added in this PR, now load. Co-Authored-By: opencode (DeepSeek V4.1 Flash) --- pyproject.toml | 2 +- uv.lock | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 48545dd82b28..d643b2f697fe 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -193,7 +193,7 @@ dev = [ "selenium>=4.16.0", "sentry-cli>=3.3.0", "sentry-covdefaults-disable-branch-coverage>=1.0.2", - "sentry-devenv>=1.28.0", + "sentry-devenv>=1.31.0", "django-stubs>=5.2.9", "djangorestframework-stubs>=3.16.8", "time-machine>=2.16.0", diff --git a/uv.lock b/uv.lock index f41a276e5ba6..59bf1f34e174 100644 --- a/uv.lock +++ b/uv.lock @@ -2497,7 +2497,7 @@ dev = [ { name = "selenium", specifier = ">=4.16.0" }, { name = "sentry-cli", specifier = ">=3.3.0" }, { name = "sentry-covdefaults-disable-branch-coverage", specifier = ">=1.0.2" }, - { name = "sentry-devenv", specifier = ">=1.28.0" }, + { name = "sentry-devenv", specifier = ">=1.31.0" }, { name = "time-machine", specifier = ">=2.16.0" }, { name = "types-beautifulsoup4", specifier = ">=4.11.6" }, { name = "types-cachetools", specifier = ">=5.3.0.5" }, @@ -2561,14 +2561,14 @@ wheels = [ [[package]] name = "sentry-devenv" -version = "1.28.0" +version = "1.31.0" source = { registry = "https://pypi.devinfra.sentry.io/simple" } dependencies = [ { name = "sentry-sdk" }, { name = "typing-extensions" }, ] wheels = [ - { url = "https://pypi.devinfra.sentry.io/wheels/sentry_devenv-1.28.0-py3-none-any.whl", hash = "sha256:304b603c561c4a0a206c7d1346aebf8ec44e6175f44ecce0b9c1a5848fb3f7ca" }, + { url = "https://pypi.devinfra.sentry.io/wheels/sentry_devenv-1.31.0-py3-none-any.whl", hash = "sha256:aa2614e060b7793f2cd7aa73cedce02b53d3750dbebd7f9afe68a7c2d17fb5dc" }, ] [[package]]