docs: add database migration strategy guide
Document the 31 migration files, naming conventions, revision chain, circular FK workaround, NULL casting gotcha, and migration history table. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
163
docs/DATABASE-MIGRATIONS.md
Normal file
163
docs/DATABASE-MIGRATIONS.md
Normal file
@@ -0,0 +1,163 @@
|
|||||||
|
# Database Migrations Guide
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
ResolutionFlow uses **Alembic** for database migrations with PostgreSQL 16. As of February 2026, there are **31 migration files** (27 sequential + 4 hash-based).
|
||||||
|
|
||||||
|
## Migration Naming Conventions
|
||||||
|
|
||||||
|
The project uses **two naming styles** due to how migrations were created:
|
||||||
|
|
||||||
|
### Sequential (001–027)
|
||||||
|
Created with `alembic revision -m "description"` and manually named:
|
||||||
|
```
|
||||||
|
001_initial_schema.py
|
||||||
|
002_add_invite_codes.py
|
||||||
|
...
|
||||||
|
027_add_trees_fts_index.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Hash-based (Alembic auto-generated)
|
||||||
|
Created with `alembic revision --autogenerate -m "description"`:
|
||||||
|
```
|
||||||
|
4cdb5cba1aff_add_custom_steps_to_sessions.py
|
||||||
|
7e00fa3c75c9_fix_datetime_timezone.py
|
||||||
|
11c8abf7ef5b_add_session_search_indexes.py
|
||||||
|
25b001abd0f7_merge_tree_sharing_and_session_indexes.py
|
||||||
|
```
|
||||||
|
|
||||||
|
**Going forward:** Use sequential numeric naming (028, 029, etc.) for consistency. Only use `--autogenerate` when generating from model changes, then rename the file to the next sequential number.
|
||||||
|
|
||||||
|
## Revision Chain
|
||||||
|
|
||||||
|
Migrations form a linear chain via `down_revision`. The chain is **not purely numeric** — some hash-based migrations are interleaved. Always inspect `down_revision` in the latest file to find the current head:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Find current head
|
||||||
|
cd backend
|
||||||
|
alembic heads
|
||||||
|
|
||||||
|
# Show full chain
|
||||||
|
alembic history
|
||||||
|
```
|
||||||
|
|
||||||
|
## Creating New Migrations
|
||||||
|
|
||||||
|
### From model changes (auto-detect):
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
alembic revision --autogenerate -m "add_new_column"
|
||||||
|
# Rename file to next sequential number (e.g., 028_add_new_column.py)
|
||||||
|
# Review generated code — autogenerate misses some changes
|
||||||
|
alembic upgrade head
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual migration (no model change):
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
alembic revision -m "add_index_on_trees"
|
||||||
|
# Edit the file manually
|
||||||
|
alembic upgrade head
|
||||||
|
```
|
||||||
|
|
||||||
|
### Without a running database:
|
||||||
|
```bash
|
||||||
|
# Safe to create and commit without testing locally
|
||||||
|
alembic revision -m "description"
|
||||||
|
# Edit migration file manually
|
||||||
|
# Migration runs when DB is available (deploy, CI, or local Docker)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Known Issues
|
||||||
|
|
||||||
|
### Circular Foreign Keys: users ↔ invite_codes
|
||||||
|
|
||||||
|
The `users` and `invite_codes` tables have circular foreign keys (`users.invite_code_id → invite_codes.id` and `invite_codes.created_by → users.id`). This causes `CircularDependencyError` when using `Base.metadata.drop_all()`.
|
||||||
|
|
||||||
|
**Workaround (used in test fixtures):**
|
||||||
|
```python
|
||||||
|
# Instead of Base.metadata.drop_all(engine):
|
||||||
|
await conn.execute(sa.text("DROP SCHEMA public CASCADE"))
|
||||||
|
await conn.execute(sa.text("CREATE SCHEMA public"))
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: asyncpg rejects multi-statement strings, so these must be two separate `execute()` calls.
|
||||||
|
|
||||||
|
### NULL Casting in UUID Columns
|
||||||
|
|
||||||
|
PostgreSQL infers `NULL` as text type, which fails for UUID columns:
|
||||||
|
```sql
|
||||||
|
-- WRONG
|
||||||
|
INSERT INTO tree_tags (name, slug, team_id)
|
||||||
|
SELECT 'tag', 'slug', NULL as team_id -- Error!
|
||||||
|
|
||||||
|
-- CORRECT
|
||||||
|
INSERT INTO tree_tags (name, slug, team_id)
|
||||||
|
SELECT 'tag', 'slug', NULL::uuid as team_id
|
||||||
|
```
|
||||||
|
|
||||||
|
### Conditional Updates in Migrations
|
||||||
|
|
||||||
|
Always verify actual data values before writing `WHERE` clauses in data migrations. Migration 010 had `WHERE role = 'admin'` but the data had already changed to `role = 'engineer'`, so the UPDATE matched zero rows.
|
||||||
|
|
||||||
|
## Running Migrations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Apply all pending migrations
|
||||||
|
cd backend
|
||||||
|
alembic upgrade head
|
||||||
|
|
||||||
|
# Rollback one step
|
||||||
|
alembic downgrade -1
|
||||||
|
|
||||||
|
# Rollback to specific revision
|
||||||
|
alembic downgrade 025
|
||||||
|
|
||||||
|
# Check current state
|
||||||
|
alembic current
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration History
|
||||||
|
|
||||||
|
| # | Migration | Description |
|
||||||
|
|---|-----------|-------------|
|
||||||
|
| 001 | initial_schema | Users, teams, trees, sessions, attachments |
|
||||||
|
| 002 | add_invite_codes | Invite code system (circular FK with users) |
|
||||||
|
| 003 | add_tree_is_default | Default/system trees flag |
|
||||||
|
| 004 | add_tree_is_public | Public visibility flag |
|
||||||
|
| 005 | add_tree_organization | Categories and tags |
|
||||||
|
| 006 | add_folder_hierarchy | User folders with 3-level depth |
|
||||||
|
| 007 | add_step_categories | Step categories for library |
|
||||||
|
| 008 | add_step_library | Step library, ratings, usage tracking |
|
||||||
|
| 009 | add_scratchpad_to_sessions | Session scratchpad field |
|
||||||
|
| 010 | add_is_super_admin | Super admin boolean on users |
|
||||||
|
| 011 | add_role_check_constraint | CHECK constraint on role field |
|
||||||
|
| 012 | add_user_is_active | User activation/deactivation |
|
||||||
|
| 013 | add_refresh_tokens | JTI-based token revocation |
|
||||||
|
| 014 | add_audit_logs | Audit log table with JSONB details |
|
||||||
|
| 015 | add_deleted_at_to_trees | Soft delete for trees |
|
||||||
|
| 016 | add_subscription_tables | Subscriptions and plan limits |
|
||||||
|
| 017 | add_account_id_to_users | Account ID on user model |
|
||||||
|
| 018 | migrate_users_to_accounts | Data migration: users → accounts |
|
||||||
|
| 019 | migrate_team_fks_to_account | Remap team FKs to account FKs |
|
||||||
|
| 020 | finalize_account_migration | Clean up old team references |
|
||||||
|
| 021 | fix_owner_id_nullable | Fix account owner_id constraint |
|
||||||
|
| 022 | add_tree_forking | Fork tracking (parent_tree_id, root_tree_id) |
|
||||||
|
| 023 | add_session_sharing | Session share tokens |
|
||||||
|
| 024 | add_tree_sharing | Tree share tokens |
|
||||||
|
| 025 | add_tree_status_field | Draft/published status |
|
||||||
|
| 026 | add_admin_panel_tables | Feature flags, platform settings, plan limits |
|
||||||
|
| 027 | add_trees_fts_index | GIN index for full-text search |
|
||||||
|
| hash | add_custom_steps_to_sessions | Custom steps JSONB on sessions |
|
||||||
|
| hash | fix_datetime_timezone | Timezone-aware datetime columns |
|
||||||
|
| hash | add_session_search_indexes | Search indexes on sessions |
|
||||||
|
| hash | merge_tree_sharing_and_session_indexes | Merge head resolution |
|
||||||
|
|
||||||
|
## Squash Policy
|
||||||
|
|
||||||
|
Migration squashing is deferred until it becomes necessary (e.g., new developer onboarding takes too long, or migration chain causes issues). When squashing:
|
||||||
|
|
||||||
|
1. Create a fresh database dump as the "foundation" migration
|
||||||
|
2. Replace all existing migrations with a single `001_foundation.py`
|
||||||
|
3. Test on a clean database
|
||||||
|
4. Update this document
|
||||||
Reference in New Issue
Block a user