Files
resolutionflow/.gitea/workflows/ci.yml
Michael Chihlas 7f714363dd perf(ci): pytest-xdist with per-worker DBs — 22m → ~4m
Backend suite is the slow gate (1076 passed locally in 22m27s on
fix/ci-workflow-config). Adding pytest-xdist with per-worker DB
isolation drops it to ~4m20s on the 8-core homelab runner. Verified
locally: `pytest -n auto --no-cov` finished in 4m28s real time
(15m19s user — confirms ~5× parallelism).

How it works:
- conftest.py reads `PYTEST_XDIST_WORKER` (set per worker by xdist —
  'gw0', 'gw1', …). When set, derives a per-worker DB URL like
  `…/resolutionflow_test_gw0`. The base DB stays for serial / master
  runs.
- `_ensure_worker_db_exists` runs synchronously at conftest import,
  connects to the postgres maintenance DB, and `CREATE DATABASE`s the
  worker-suffixed DB if it doesn't exist. Idempotent across runs.
- The "test" safety guard still applies — every worker DB name
  contains "test" so the assertion holds.
- The per-test `DROP SCHEMA public CASCADE` now operates on the
  worker's isolated DB, no cross-worker race.

CI workflow: backend job switches to `pytest -n auto`. Coverage still
collected (pytest-cov has built-in xdist support).

Adds `pytest-xdist==3.6.1` to requirements-dev.txt.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 15:53:47 -04:00

212 lines
7.3 KiB
YAML

name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
backend:
runs-on: ubuntu-latest
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: resolutionflow_test
# No host port mapping. Tests connect to `postgres:5432` (the service
# container's docker-network DNS name), not `localhost:5432`. With
# multiple Gitea runners on the same homelab box, host-port mapping
# would race — two backend/e2e jobs both binding 0.0.0.0:5432 → the
# second fails with "port is already allocated".
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
DATABASE_URL: postgresql+asyncpg://postgres:postgres@postgres:5432/resolutionflow_test
DATABASE_URL_SYNC: postgresql://postgres:postgres@postgres:5432/resolutionflow_test
# conftest.py reads DATABASE_TEST_URL only (DATABASE_URL is intentionally
# not consulted after the dab740d test-isolation hardening). The CI test
# DB is the same postgres service, so point DATABASE_TEST_URL at it
# explicitly — without this, conftest falls back to localhost:5432 and
# all tests fail at fixture setup with "connection refused".
DATABASE_TEST_URL: postgresql+asyncpg://postgres:postgres@postgres:5432/resolutionflow_test
SECRET_KEY: ci-test-secret-key-not-for-production
DEBUG: "true"
APP_NAME: ResolutionFlow
TEST_DB_NAME: resolutionflow_test
DB_APP_ROLE_PASSWORD: app_secret_ci
steps:
- uses: actions/checkout@v4
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-${{ hashFiles('backend/requirements.txt', 'backend/requirements-dev.txt') }}
restore-keys: |
pip-${{ runner.os }}-
- name: Install system dependencies
run: |
apt-get update
apt-get install -y libpango1.0-dev libcairo2-dev libgdk-pixbuf-2.0-dev libffi-dev libjpeg-dev zlib1g-dev
- name: Install dependencies
run: pip install --break-system-packages -r backend/requirements.txt -r backend/requirements-dev.txt
- name: Run Alembic migrations
run: cd backend && alembic upgrade head
- name: Check tenant filter enforcement
run: cd backend && python scripts/check_tenant_filters.py
- name: Run tests with coverage
# `-n auto` parallelizes across all runner cores via pytest-xdist.
# conftest.py creates a per-worker DB (resolutionflow_test_gw0,
# resolutionflow_test_gw1, …) so the per-test DROP SCHEMA doesn't
# race across workers. Master/serial runs keep the base DB.
# term-missing dropped — the custom "Display coverage summary" step
# below parses coverage.json and prints the same info more concisely.
# --maxfail=10 short-circuits on structural breakage so we don't burn
# 25 minutes when a fixture explodes.
run: cd backend && python -m pytest --override-ini="addopts=" -n auto --maxfail=10 --cov=app --cov-report=json:coverage.json --cov-fail-under=50
- name: Display coverage summary
if: always()
run: |
cd backend
python -c "
import json
with open('coverage.json') as f:
data = json.load(f)
total = data['totals']['percent_covered_display']
print(f'Total coverage: {total}%')
print()
print('Module coverage:')
for fname, fdata in sorted(data['files'].items()):
pct = fdata['summary']['percent_covered_display']
if float(pct) < 80:
print(f' WARNING {fname}: {pct}%')
else:
print(f' OK {fname}: {pct}%')
"
frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Cache npm
uses: actions/cache@v3
with:
path: ~/.npm
key: npm-${{ runner.os }}-${{ hashFiles('frontend/package-lock.json') }}
restore-keys: |
npm-${{ runner.os }}-
- name: Install dependencies
run: cd frontend && npm ci
- name: Lint
run: cd frontend && npm run lint
- name: Test with coverage
run: cd frontend && npm run test:coverage
- name: Build
run: cd frontend && NODE_OPTIONS="--max-old-space-size=4096" npm run build
- name: Upload build artifact
uses: actions/upload-artifact@v3
with:
name: frontend-dist
path: frontend/dist
retention-days: 1
e2e:
needs: [frontend]
runs-on: ubuntu-latest
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: resolutionflow_test
# No host port mapping. Tests connect to `postgres:5432` (the service
# container's docker-network DNS name), not `localhost:5432`. With
# multiple Gitea runners on the same homelab box, host-port mapping
# would race — two backend/e2e jobs both binding 0.0.0.0:5432 → the
# second fails with "port is already allocated".
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
PLAYWRIGHT_DATABASE_URL: postgresql+asyncpg://postgres:postgres@postgres:5432/resolutionflow_test
PLAYWRIGHT_DATABASE_URL_SYNC: postgresql://postgres:postgres@postgres:5432/resolutionflow_test
PLAYWRIGHT_API_ORIGIN: http://127.0.0.1:8000
PLAYWRIGHT_BASE_URL: http://127.0.0.1:4173
PLAYWRIGHT_SECRET_KEY: ci-playwright-secret-key
PLAYWRIGHT_TEST_EMAIL: teamadmin@resolutionflow.example.com
PLAYWRIGHT_TEST_PASSWORD: TestPass123!
steps:
- uses: actions/checkout@v4
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-${{ hashFiles('backend/requirements.txt', 'backend/requirements-dev.txt') }}
restore-keys: |
pip-${{ runner.os }}-
- name: Cache npm
uses: actions/cache@v3
with:
path: ~/.npm
key: npm-${{ runner.os }}-${{ hashFiles('frontend/package-lock.json') }}
restore-keys: |
npm-${{ runner.os }}-
- name: Install backend dependencies
run: pip install --break-system-packages -r backend/requirements.txt -r backend/requirements-dev.txt
- name: Install frontend dependencies
run: cd frontend && npm ci
- name: Download frontend build
uses: actions/download-artifact@v3
with:
name: frontend-dist
path: frontend/dist
- name: Install Playwright browser
run: cd frontend && npx playwright install --with-deps chromium
- name: Run Playwright smoke tests
run: cd frontend && npm run test:e2e
- name: Upload Playwright report
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: |
frontend/playwright-report
frontend/test-results
if-no-files-found: ignore