docs: add database migration strategy guide

Document the 31 migration files, naming conventions, revision chain,
circular FK workaround, NULL casting gotcha, and migration history table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
chihlasm
2026-02-08 17:48:38 -05:00
parent 298af2d6d7
commit d4acef5903

163
docs/DATABASE-MIGRATIONS.md Normal file
View File

@@ -0,0 +1,163 @@
# Database Migrations Guide
## Overview
ResolutionFlow uses **Alembic** for database migrations with PostgreSQL 16. As of February 2026, there are **31 migration files** (27 sequential + 4 hash-based).
## Migration Naming Conventions
The project uses **two naming styles** due to how migrations were created:
### Sequential (001027)
Created with `alembic revision -m "description"` and manually named:
```
001_initial_schema.py
002_add_invite_codes.py
...
027_add_trees_fts_index.py
```
### Hash-based (Alembic auto-generated)
Created with `alembic revision --autogenerate -m "description"`:
```
4cdb5cba1aff_add_custom_steps_to_sessions.py
7e00fa3c75c9_fix_datetime_timezone.py
11c8abf7ef5b_add_session_search_indexes.py
25b001abd0f7_merge_tree_sharing_and_session_indexes.py
```
**Going forward:** Use sequential numeric naming (028, 029, etc.) for consistency. Only use `--autogenerate` when generating from model changes, then rename the file to the next sequential number.
## Revision Chain
Migrations form a linear chain via `down_revision`. The chain is **not purely numeric** — some hash-based migrations are interleaved. Always inspect `down_revision` in the latest file to find the current head:
```bash
# Find current head
cd backend
alembic heads
# Show full chain
alembic history
```
## Creating New Migrations
### From model changes (auto-detect):
```bash
cd backend
alembic revision --autogenerate -m "add_new_column"
# Rename file to next sequential number (e.g., 028_add_new_column.py)
# Review generated code — autogenerate misses some changes
alembic upgrade head
```
### Manual migration (no model change):
```bash
cd backend
alembic revision -m "add_index_on_trees"
# Edit the file manually
alembic upgrade head
```
### Without a running database:
```bash
# Safe to create and commit without testing locally
alembic revision -m "description"
# Edit migration file manually
# Migration runs when DB is available (deploy, CI, or local Docker)
```
## Known Issues
### Circular Foreign Keys: users ↔ invite_codes
The `users` and `invite_codes` tables have circular foreign keys (`users.invite_code_id → invite_codes.id` and `invite_codes.created_by → users.id`). This causes `CircularDependencyError` when using `Base.metadata.drop_all()`.
**Workaround (used in test fixtures):**
```python
# Instead of Base.metadata.drop_all(engine):
await conn.execute(sa.text("DROP SCHEMA public CASCADE"))
await conn.execute(sa.text("CREATE SCHEMA public"))
```
Note: asyncpg rejects multi-statement strings, so these must be two separate `execute()` calls.
### NULL Casting in UUID Columns
PostgreSQL infers `NULL` as text type, which fails for UUID columns:
```sql
-- WRONG
INSERT INTO tree_tags (name, slug, team_id)
SELECT 'tag', 'slug', NULL as team_id -- Error!
-- CORRECT
INSERT INTO tree_tags (name, slug, team_id)
SELECT 'tag', 'slug', NULL::uuid as team_id
```
### Conditional Updates in Migrations
Always verify actual data values before writing `WHERE` clauses in data migrations. Migration 010 had `WHERE role = 'admin'` but the data had already changed to `role = 'engineer'`, so the UPDATE matched zero rows.
## Running Migrations
```bash
# Apply all pending migrations
cd backend
alembic upgrade head
# Rollback one step
alembic downgrade -1
# Rollback to specific revision
alembic downgrade 025
# Check current state
alembic current
```
## Migration History
| # | Migration | Description |
|---|-----------|-------------|
| 001 | initial_schema | Users, teams, trees, sessions, attachments |
| 002 | add_invite_codes | Invite code system (circular FK with users) |
| 003 | add_tree_is_default | Default/system trees flag |
| 004 | add_tree_is_public | Public visibility flag |
| 005 | add_tree_organization | Categories and tags |
| 006 | add_folder_hierarchy | User folders with 3-level depth |
| 007 | add_step_categories | Step categories for library |
| 008 | add_step_library | Step library, ratings, usage tracking |
| 009 | add_scratchpad_to_sessions | Session scratchpad field |
| 010 | add_is_super_admin | Super admin boolean on users |
| 011 | add_role_check_constraint | CHECK constraint on role field |
| 012 | add_user_is_active | User activation/deactivation |
| 013 | add_refresh_tokens | JTI-based token revocation |
| 014 | add_audit_logs | Audit log table with JSONB details |
| 015 | add_deleted_at_to_trees | Soft delete for trees |
| 016 | add_subscription_tables | Subscriptions and plan limits |
| 017 | add_account_id_to_users | Account ID on user model |
| 018 | migrate_users_to_accounts | Data migration: users → accounts |
| 019 | migrate_team_fks_to_account | Remap team FKs to account FKs |
| 020 | finalize_account_migration | Clean up old team references |
| 021 | fix_owner_id_nullable | Fix account owner_id constraint |
| 022 | add_tree_forking | Fork tracking (parent_tree_id, root_tree_id) |
| 023 | add_session_sharing | Session share tokens |
| 024 | add_tree_sharing | Tree share tokens |
| 025 | add_tree_status_field | Draft/published status |
| 026 | add_admin_panel_tables | Feature flags, platform settings, plan limits |
| 027 | add_trees_fts_index | GIN index for full-text search |
| hash | add_custom_steps_to_sessions | Custom steps JSONB on sessions |
| hash | fix_datetime_timezone | Timezone-aware datetime columns |
| hash | add_session_search_indexes | Search indexes on sessions |
| hash | merge_tree_sharing_and_session_indexes | Merge head resolution |
## Squash Policy
Migration squashing is deferred until it becomes necessary (e.g., new developer onboarding takes too long, or migration chain causes issues). When squashing:
1. Create a fresh database dump as the "foundation" migration
2. Replace all existing migrations with a single `001_foundation.py`
3. Test on a clean database
4. Update this document