Skip to content

Let stacklets ship migrations the framework runs once, in order #220

Open

Description

@arthware-dev

Story

As a stacklet author, I want to ship a change to existing data as a migration that the framework runs once per instance, in order, so that moving files, renaming fields or rewriting records happens exactly once and is recorded, the way database migrations work.

Context

Each data change today invents its own trigger and its own record of having run:

  • stack up memory moves each topic's old to-do file to the lists (move_todos_to_lists in stacklets/memory/lib.py, called from stacklets/memory/hooks/on_start_ready.py). It runs on every start and finds nothing to do after the first.
  • The archivist moves photos filed as bookmarks to notes (stacklets/docs/bot/migration_photos_to_notes.py, fix(archivist): keep what the family writes about a capture #219). It runs a minute after the bot starts and leaves a marker file in its session directory.
  • Moving documents to kind (Name the variant of every file type with kind #217) needs the same again.

None of them is ordered against another, none shows up in stack doctor, and a failure is a log line.

Proposal

  • A stacklet ships migrations as files: stacklets/<id>/migrations/NNNN_<name>.py, each with def run(ctx) and a docstring saying what it changes. Picked up by convention, like hooks.
  • stack up <id> runs the pending ones after the stacklet is healthy, in number order, and records each applied one in the instance's state (.stack/). The first failure stops the run and is reported; the rest wait for the next stack up.
  • A migration is idempotent, so a run cut short is safe to repeat.
  • stack doctor lists pending and failed migrations.
  • The two existing migrations move over and keep their behaviour.
  • Open question: a migration that needs a running service (the photo migration downloads uploads from Matrix) either runs in the stacklet's container through the context, or the context gives it what it needs on the host.

Out of scope: down migrations, versioned data formats.

Definition of done

  • A test: two migrations of one stacklet run once, in order, on stack up; a second stack up runs neither
  • A test: a failing migration stops the run, is reported, and runs again on the next stack up; the one after it waits
  • The to-do and photo migrations run through it, with their existing tests green
  • stack doctor names a pending or failed migration
  • docs/stack-reference.md describes the migration contract

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions