resolutionflow/docs/DATABASE-MIGRATIONS.md

# Database Migrations Guide

## Overview

ResolutionFlow uses **Alembic** for database migrations with PostgreSQL 16. As of February 2026, there are **31 migration files** (27 sequential + 4 hash-based).

## Migration Naming Conventions

The project uses **two naming styles** due to how migrations were created:

### Sequential (001–027)
Created with `alembic revision -m "description"` and manually named:
```
001_initial_schema.py
002_add_invite_codes.py
...
027_add_trees_fts_index.py
```

### Hash-based (Alembic auto-generated)
Created with `alembic revision --autogenerate -m "description"`:
```
4cdb5cba1aff_add_custom_steps_to_sessions.py
7e00fa3c75c9_fix_datetime_timezone.py
11c8abf7ef5b_add_session_search_indexes.py
25b001abd0f7_merge_tree_sharing_and_session_indexes.py
```

**Going forward:** Use sequential numeric naming (028, 029, etc.) for consistency. Only use `--autogenerate` when generating from model changes, then rename the file to the next sequential number.

## Revision Chain

Migrations form a linear chain via `down_revision`. The chain is **not purely numeric** — some hash-based migrations are interleaved. Always inspect `down_revision` in the latest file to find the current head:

```bash
# Find current head
cd backend
alembic heads

# Show full chain
alembic history
```

## Creating New Migrations

### From model changes (auto-detect):
```bash
cd backend
alembic revision --autogenerate -m "add_new_column"
# Rename file to next sequential number (e.g., 028_add_new_column.py)
# Review generated code — autogenerate misses some changes
alembic upgrade head
```

### Manual migration (no model change):
```bash
cd backend
alembic revision -m "add_index_on_trees"
# Edit the file manually
alembic upgrade head
```

### Without a running database:
```bash
# Safe to create and commit without testing locally
alembic revision -m "description"
# Edit migration file manually
# Migration runs when DB is available (deploy, CI, or local Docker)
```

## Known Issues

### Circular Foreign Keys: users ↔ invite_codes

The `users` and `invite_codes` tables have circular foreign keys (`users.invite_code_id → invite_codes.id` and `invite_codes.created_by → users.id`). This causes `CircularDependencyError` when using `Base.metadata.drop_all()`.

**Workaround (used in test fixtures):**
```python
# Instead of Base.metadata.drop_all(engine):
await conn.execute(sa.text("DROP SCHEMA public CASCADE"))
await conn.execute(sa.text("CREATE SCHEMA public"))
```

Note: asyncpg rejects multi-statement strings, so these must be two separate `execute()` calls.

### NULL Casting in UUID Columns

PostgreSQL infers `NULL` as text type, which fails for UUID columns:
```sql
-- WRONG
INSERT INTO tree_tags (name, slug, team_id)
SELECT 'tag', 'slug', NULL as team_id  -- Error!

-- CORRECT
INSERT INTO tree_tags (name, slug, team_id)
SELECT 'tag', 'slug', NULL::uuid as team_id
```

### Conditional Updates in Migrations

Always verify actual data values before writing `WHERE` clauses in data migrations. Migration 010 had `WHERE role = 'admin'` but the data had already changed to `role = 'engineer'`, so the UPDATE matched zero rows.

## Running Migrations

```bash
# Apply all pending migrations
cd backend
alembic upgrade head

# Rollback one step
alembic downgrade -1

# Rollback to specific revision
alembic downgrade 025

# Check current state
alembic current
```

## Migration History

| # | Migration | Description |
|---|-----------|-------------|
| 001 | initial_schema | Users, teams, trees, sessions, attachments |
| 002 | add_invite_codes | Invite code system (circular FK with users) |
| 003 | add_tree_is_default | Default/system trees flag |
| 004 | add_tree_is_public | Public visibility flag |
| 005 | add_tree_organization | Categories and tags |
| 006 | add_folder_hierarchy | User folders with 3-level depth |
| 007 | add_step_categories | Step categories for library |
| 008 | add_step_library | Step library, ratings, usage tracking |
| 009 | add_scratchpad_to_sessions | Session scratchpad field |
| 010 | add_is_super_admin | Super admin boolean on users |
| 011 | add_role_check_constraint | CHECK constraint on role field |
| 012 | add_user_is_active | User activation/deactivation |
| 013 | add_refresh_tokens | JTI-based token revocation |
| 014 | add_audit_logs | Audit log table with JSONB details |
| 015 | add_deleted_at_to_trees | Soft delete for trees |
| 016 | add_subscription_tables | Subscriptions and plan limits |
| 017 | add_account_id_to_users | Account ID on user model |
| 018 | migrate_users_to_accounts | Data migration: users → accounts |
| 019 | migrate_team_fks_to_account | Remap team FKs to account FKs |
| 020 | finalize_account_migration | Clean up old team references |
| 021 | fix_owner_id_nullable | Fix account owner_id constraint |
| 022 | add_tree_forking | Fork tracking (parent_tree_id, root_tree_id) |
| 023 | add_session_sharing | Session share tokens |
| 024 | add_tree_sharing | Tree share tokens |
| 025 | add_tree_status_field | Draft/published status |
| 026 | add_admin_panel_tables | Feature flags, platform settings, plan limits |
| 027 | add_trees_fts_index | GIN index for full-text search |
| hash | add_custom_steps_to_sessions | Custom steps JSONB on sessions |
| hash | fix_datetime_timezone | Timezone-aware datetime columns |
| hash | add_session_search_indexes | Search indexes on sessions |
| hash | merge_tree_sharing_and_session_indexes | Merge head resolution |

## Squash Policy

Migration squashing is deferred until it becomes necessary (e.g., new developer onboarding takes too long, or migration chain causes issues). When squashing:

1. Create a fresh database dump as the "foundation" migration
2. Replace all existing migrations with a single `001_foundation.py`
3. Test on a clean database
4. Update this document