diff --git a/docs/DATABASE-MIGRATIONS.md b/docs/DATABASE-MIGRATIONS.md new file mode 100644 index 00000000..c8f51b30 --- /dev/null +++ b/docs/DATABASE-MIGRATIONS.md @@ -0,0 +1,163 @@ +# Database Migrations Guide + +## Overview + +ResolutionFlow uses **Alembic** for database migrations with PostgreSQL 16. As of February 2026, there are **31 migration files** (27 sequential + 4 hash-based). + +## Migration Naming Conventions + +The project uses **two naming styles** due to how migrations were created: + +### Sequential (001–027) +Created with `alembic revision -m "description"` and manually named: +``` +001_initial_schema.py +002_add_invite_codes.py +... +027_add_trees_fts_index.py +``` + +### Hash-based (Alembic auto-generated) +Created with `alembic revision --autogenerate -m "description"`: +``` +4cdb5cba1aff_add_custom_steps_to_sessions.py +7e00fa3c75c9_fix_datetime_timezone.py +11c8abf7ef5b_add_session_search_indexes.py +25b001abd0f7_merge_tree_sharing_and_session_indexes.py +``` + +**Going forward:** Use sequential numeric naming (028, 029, etc.) for consistency. Only use `--autogenerate` when generating from model changes, then rename the file to the next sequential number. + +## Revision Chain + +Migrations form a linear chain via `down_revision`. The chain is **not purely numeric** — some hash-based migrations are interleaved. Always inspect `down_revision` in the latest file to find the current head: + +```bash +# Find current head +cd backend +alembic heads + +# Show full chain +alembic history +``` + +## Creating New Migrations + +### From model changes (auto-detect): +```bash +cd backend +alembic revision --autogenerate -m "add_new_column" +# Rename file to next sequential number (e.g., 028_add_new_column.py) +# Review generated code — autogenerate misses some changes +alembic upgrade head +``` + +### Manual migration (no model change): +```bash +cd backend +alembic revision -m "add_index_on_trees" +# Edit the file manually +alembic upgrade head +``` + +### Without a running database: +```bash +# Safe to create and commit without testing locally +alembic revision -m "description" +# Edit migration file manually +# Migration runs when DB is available (deploy, CI, or local Docker) +``` + +## Known Issues + +### Circular Foreign Keys: users ↔ invite_codes + +The `users` and `invite_codes` tables have circular foreign keys (`users.invite_code_id → invite_codes.id` and `invite_codes.created_by → users.id`). This causes `CircularDependencyError` when using `Base.metadata.drop_all()`. + +**Workaround (used in test fixtures):** +```python +# Instead of Base.metadata.drop_all(engine): +await conn.execute(sa.text("DROP SCHEMA public CASCADE")) +await conn.execute(sa.text("CREATE SCHEMA public")) +``` + +Note: asyncpg rejects multi-statement strings, so these must be two separate `execute()` calls. + +### NULL Casting in UUID Columns + +PostgreSQL infers `NULL` as text type, which fails for UUID columns: +```sql +-- WRONG +INSERT INTO tree_tags (name, slug, team_id) +SELECT 'tag', 'slug', NULL as team_id -- Error! + +-- CORRECT +INSERT INTO tree_tags (name, slug, team_id) +SELECT 'tag', 'slug', NULL::uuid as team_id +``` + +### Conditional Updates in Migrations + +Always verify actual data values before writing `WHERE` clauses in data migrations. Migration 010 had `WHERE role = 'admin'` but the data had already changed to `role = 'engineer'`, so the UPDATE matched zero rows. + +## Running Migrations + +```bash +# Apply all pending migrations +cd backend +alembic upgrade head + +# Rollback one step +alembic downgrade -1 + +# Rollback to specific revision +alembic downgrade 025 + +# Check current state +alembic current +``` + +## Migration History + +| # | Migration | Description | +|---|-----------|-------------| +| 001 | initial_schema | Users, teams, trees, sessions, attachments | +| 002 | add_invite_codes | Invite code system (circular FK with users) | +| 003 | add_tree_is_default | Default/system trees flag | +| 004 | add_tree_is_public | Public visibility flag | +| 005 | add_tree_organization | Categories and tags | +| 006 | add_folder_hierarchy | User folders with 3-level depth | +| 007 | add_step_categories | Step categories for library | +| 008 | add_step_library | Step library, ratings, usage tracking | +| 009 | add_scratchpad_to_sessions | Session scratchpad field | +| 010 | add_is_super_admin | Super admin boolean on users | +| 011 | add_role_check_constraint | CHECK constraint on role field | +| 012 | add_user_is_active | User activation/deactivation | +| 013 | add_refresh_tokens | JTI-based token revocation | +| 014 | add_audit_logs | Audit log table with JSONB details | +| 015 | add_deleted_at_to_trees | Soft delete for trees | +| 016 | add_subscription_tables | Subscriptions and plan limits | +| 017 | add_account_id_to_users | Account ID on user model | +| 018 | migrate_users_to_accounts | Data migration: users → accounts | +| 019 | migrate_team_fks_to_account | Remap team FKs to account FKs | +| 020 | finalize_account_migration | Clean up old team references | +| 021 | fix_owner_id_nullable | Fix account owner_id constraint | +| 022 | add_tree_forking | Fork tracking (parent_tree_id, root_tree_id) | +| 023 | add_session_sharing | Session share tokens | +| 024 | add_tree_sharing | Tree share tokens | +| 025 | add_tree_status_field | Draft/published status | +| 026 | add_admin_panel_tables | Feature flags, platform settings, plan limits | +| 027 | add_trees_fts_index | GIN index for full-text search | +| hash | add_custom_steps_to_sessions | Custom steps JSONB on sessions | +| hash | fix_datetime_timezone | Timezone-aware datetime columns | +| hash | add_session_search_indexes | Search indexes on sessions | +| hash | merge_tree_sharing_and_session_indexes | Merge head resolution | + +## Squash Policy + +Migration squashing is deferred until it becomes necessary (e.g., new developer onboarding takes too long, or migration chain causes issues). When squashing: + +1. Create a fresh database dump as the "foundation" migration +2. Replace all existing migrations with a single `001_foundation.py` +3. Test on a clean database +4. Update this document