Conversation
Proposes a simple, explicit migration system for Campus storage: - Migration files with upgrade()/downgrade() functions - State tracking via _migrations table - CLI: `campus migrate` and `campus rollback` - Yapper integration for audit logging - Supports PostgreSQL tables and MongoDB documents Addresses requirements from issue #27: 1. Initialize app via auto-run on deploy 2. Data migration support with batching 3. Cross-database migration handlers 4. Rollback capability with downgrade() 5. Audit logging via Yapper events Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nycomp
left a comment
nycomp
left a comment
There was a problem hiding this comment.
What are the pros and cons of the following strategies:
- autocommit while upgrade in progress, with downgrade option
- atomic transaction, with all upgrade operations pending until explicitly committed
Assume database downtime while migration is in progress
|
|
||
| def upgrade(): | ||
| """Add created_at field to existing responses.""" | ||
| submissions = get_collection("submissions") |
There was a problem hiding this comment.
This seems fragile to typos in collection/table names. Would it be a better idea to import the relevant resource(s) from campus.auth or campus.api?
| 3. **upgrade()** - Forward transformation | ||
| 4. **downgrade()** - Reverse transformation |
There was a problem hiding this comment.
These are assuming upgrade from the previous schema version, e.g. Revision ID 002 right? So each upgrade/downgrade is only one revision step at a time?
| ### PostgreSQL Schema (for state tracking) | ||
|
|
||
| ```sql | ||
| CREATE TABLE IF NOT EXISTS "_migrations" ( | ||
| "id" TEXT PRIMARY KEY, -- Revision ID (e.g., "001", "002") | ||
| "description" TEXT NOT NULL, -- Human-readable description | ||
| "storage_type" TEXT NOT NULL, -- "table", "document", "object" | ||
| "applied_at" TIMESTAMP NOT NULL, -- When migration was applied | ||
| "rollback_at" TIMESTAMP NULL -- When migration was rolled back (if applicable) | ||
| ); | ||
| ``` |
There was a problem hiding this comment.
Would a postgres table or JSON store (e.g. using bucket storage) be more appropriate for archival?
Does schema need a field for migration status/outcome? E.g. successful/committed, unsuccessful/rollback
| ## CLI Interface | ||
|
|
||
| Add to `campus` CLI via `main.py`: | ||
|
|
||
| ```python | ||
| # main.py additions | ||
|
|
||
| def migrate(target: str | None = None): | ||
| """Run database migrations.""" | ||
| from campus.storage.migrations import MigrationRunner | ||
| from pathlib import Path | ||
|
|
||
| migrations_dir = Path(__file__).parent / "storage" / "migrations" / "migrations" | ||
| runner = MigrationRunner(migrations_dir) | ||
| runner.upgrade(target) | ||
| print("Migration complete.") | ||
|
|
||
|
|
||
| def rollback(target: str): | ||
| """Rollback to a specific migration.""" | ||
| from campus.storage.migrations import MigrationRunner | ||
| from pathlib import Path | ||
|
|
||
| migrations_dir = Path(__file__).parent / "storage" / "migrations" / "migrations" | ||
| runner = MigrationRunner(migrations_dir) | ||
| runner.downgrade(target) | ||
| print("Rollback complete.") |
There was a problem hiding this comment.
The current plan is to execute the migration upon PR to staging branch (on a backup copy of database), with failure/rollback being flagged in deployment status and fixed until successful. Is the CLI interface necessary?
|
|
||
| ### Pre-Deployment Checklist | ||
|
|
||
| 1. **Backup database** before running migrations in production |
There was a problem hiding this comment.
Should this be part of the migration runner?
2 participants
Summary
Adds documentation for a storage migration protocol to address issue #27.
Details
The proposed protocol includes:
upgrade()/downgrade()functions_migrationstable records applied migrationscampus migrateandcampus rollbackcommandsmigration.applied,migration.rolled_back)Design Philosophy
For ~2000 users, prioritizes simplicity and explicitness over framework complexity. Avoids heavy tools like Alembic/Flyway while still providing proper versioning, rollback, and audit capabilities.
Test Plan
Closes #27
🤖 Generated with Claude Code