Files
resolutionflow/docs/plans/2026-02-25-flow-to-library-sync.md
chihlasm e6a0c0549b feat: Step Library sync + service account for default tree ownership
* feat: maintenance flow UX redesign — batch status hub, context strip, detail page upgrades (#85)

- Add BatchStatusPage (/flows/:id/batches/:batchId): per-target Start/Resume/View cards, progress bar, 5s polling while in-progress, completion outcome summary
- Add BatchStatusCard: handles not-started/in-progress/complete states with step progress for in-progress targets
- Add ActiveBatchBanner: amber banner on detail page when a batch is running, links to BatchStatusPage
- Add MaintenanceContextStrip: amber strip in ProceduralNavigationPage for maintenance flows showing target name, batch progress (X/Y complete), and Back to Batch nav
- Update MaintenanceFlowDetailPage: active batch banner, clickable run history rows with mini progress dots and outcome summaries, Run button loading state, post-launch navigates to BatchStatusPage
- Update ProceduralNavigationPage: renders MaintenanceContextStrip between top bar and content when tree_type === 'maintenance'; fetches batch progress once on mount
- Add batch_id filter to GET /sessions backend endpoint and SessionListParams frontend type
- Add /flows/:id/batches/:batchId route to router

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: session detail page — completion action + outcome summary card

- In-progress sessions: amber banner with "Complete Session" button opens
  SessionOutcomeModal to set outcome/notes/next-steps and finalize
- Completed sessions: colored outcome summary card (icon + outcome label +
  duration + notes + next steps) replaces dense header metadata; "Copy for
  Ticket" promoted to primary action inside the card
- Export toolbar de-emphasized to secondary row of smaller controls below
  the summary card

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add library-page action props to StepCard (edit/delete/save)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: pass library-page action props through StepLibraryBrowser + refreshKey

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: StepFormModal wrapper + submitLabel/isSubmitting props on StepForm

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: Step Library page — create, edit, delete, save-to-library

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add RuntimeStep union type for procedural custom steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: StepChecklist accepts RuntimeStep[], renders amber Custom badge

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: StepDetail accepts RuntimeStep, renders Custom Step badge for custom steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: custom step insertion in procedural flow sessions

Engineers can add custom steps inline during execution. Steps are
persisted to session.custom_steps and restored on resume.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: suppress StepFeedback on custom steps, fix resume stepState seeding, functional updater for step index

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: add tree forking UI design doc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: add tree fork UI implementation plan

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add ForkInfo type and fork fields to Tree/TreeListItem

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: align ForkInfo type with backend schema, remove redundant fork fields

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: ForkInfo placement, required fork_info field, add JSDoc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add ForkModal component with name and reason fields

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: ForkModal accessibility and UX (escape, click-outside, labels, maxLength)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: open ForkModal on fork action in TreeLibraryPage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add ForkModal to MyTreesPage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: show Fork chip badge on forked tree cards

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: add flow-to-library step sync design doc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: add flow-to-library sync implementation plan

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add sync tracking columns to step_library

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add sync columns and source_tree relationship to StepLibrary model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add group_label to StepContent, is_flow_synced/source_tree_name to StepLibraryResponse

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: include is_flow_synced and source_tree_name in step list/detail responses

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add is_flow_synced and source_tree_name to step list response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add selectinload and sync fields to search and get_step endpoints

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add step_sync module with extraction and upsert logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: safe NOT IN placeholders for asyncpg, add deactivate docstring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: trigger step library sync on tree publish and deactivate on delete

- Call sync_steps_from_tree in update_tree whenever the tree is published
  (status transitions to 'published' or is already published and structure changes)
- Call deactivate_synced_steps_for_tree in delete_tree before db.commit()
  so the FK SET NULL does not nullify source_tree_id before the WHERE clause runs
- Fix ::jsonb cast syntax in step_sync.py (asyncpg rejects :: operator in text()
  queries; replaced with CAST(:content AS jsonb))
- Add UniqueConstraint('source_tree_id','source_node_id') to StepLibrary model
  so Base.metadata.create_all (used by tests) creates the constraint that the
  ON CONFLICT clause in sync_steps_from_tree depends on

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add is_flow_synced and source_tree_name to Step types

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: show From Flow badge and lock icon on flow-synced StepCard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: show source flow name in StepDetailModal for synced steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add Library Visibility select to procedural StepEditor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address code review issues in flow-to-library sync

- Fix sync trigger: only fire on publish transition, not every PUT
- Add TestSyncOnPublish integration tests (2 tests, 16 total passing)
- Add group_label to frontend StepContent interface
- Guard Library Visibility select to procedure_step nodes only
- Block API edits to flow-synced steps (400 read-only guard)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: handle None author_id in step sync to avoid invalid UUID error

When a system/default tree has no author (author_id is None),
str(None) produces the literal string 'None' which asyncpg
rejects as an invalid UUID for the created_by column.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add ResolutionFlow service account to own default tree steps in library

Default/system trees had no author_id (NULL), causing a NOT NULL violation
when syncing steps to step_library.created_by on publish.

- Add is_service_account flag to users table (migration 4f4137ce)
- Add service_account.py: idempotent ensure_service_account() creates
  noreply@resolutionflow.com with unusable password on startup
- Cache service account ID on app.state at lifespan startup
- Add get_service_account_id() FastAPI dep (returns None in tests)
- sync_steps_from_tree: resolve author_id or service_account_id as created_by
- create_tree: set author_id=service_account_id for is_default trees
- Migration 1490781700bc: backfill author_id on 31 existing default trees

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:17:29 -05:00

1066 lines
37 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Flow-to-Library Step Sync Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** When a flow is published, extract its steps (procedure_steps from procedural/maintenance flows; action/solution nodes from troubleshooting flows) and upsert them into the `step_library` table, keeping them in sync on subsequent publishes, and showing a "From Flow" read-only indicator in the library browser.
**Architecture:** New migration adds 4 columns to `step_library` + a unique constraint on `(source_tree_id, source_node_id)`. A new `backend/app/core/step_sync.py` module handles all extraction and upsert logic. The `update_tree` endpoint calls `sync_steps_from_tree()` after a successful publish. Frontend adds `is_flow_synced` + `source_tree_name` to the Step type, a read-only lock in `StepCard`, and a "Library Visibility" select in `StepEditor`.
**Tech Stack:** Python FastAPI, SQLAlchemy 2.0 async, PostgreSQL JSONB, Alembic, React 19 + TypeScript + Tailwind CSS.
---
## Codebase Context
- **Publish endpoint:** `backend/app/api/endpoints/trees.py:546``update_tree()`. Publish validation at line 587. Version increment at line 639. Step sync should be inserted after line 641 (after version increment, before tag replacement at line 643).
- **Step library model:** `backend/app/models/step_library.py``StepLibrary` table, 20 cols. No `source_tree_id` yet.
- **Step schemas:** `backend/app/schemas/step_library.py``StepContent` (lines 1519), `StepLibraryResponse` (lines 4564). No `is_flow_synced` yet.
- **Frontend Step type:** `frontend/src/types/step.ts:15``Step` interface. No `is_flow_synced` yet.
- **StepCard:** `frontend/src/components/step-library/StepCard.tsx` — ownership check via `isOwn = step.created_by === currentUserId`.
- **StepEditor More Options:** `frontend/src/components/procedural-editor/StepEditor.tsx:138``{showMore && <div>}` block inside the More Options section.
- **Fixtures:** `client`, `test_db`, `test_user`, `auth_headers`, `test_tree`, `test_admin`, `admin_auth_headers` from `backend/tests/conftest.py`.
- **Next migration:** base off `e65b9f8fd458_add_feedback_table.py`.
- **Test runner:** `cd backend && backend/venv/bin/python -m pytest tests/test_step_sync.py -v --override-ini="addopts="`
---
### Task 1: Database migration — add sync columns to `step_library`
**Files:**
- Create: `backend/alembic/versions/030_add_step_library_sync_fields.py`
**Step 1: Create the migration file manually (do NOT use autogenerate)**
```bash
cd /home/michaelchihlas/dev/patherly/backend
alembic revision -m "add_step_library_sync_fields"
```
This creates a new file in `backend/alembic/versions/`. Open it and replace the `upgrade()` and `downgrade()` bodies with:
```python
def upgrade() -> None:
op.add_column('step_library', sa.Column('source_tree_id', sa.UUID(), nullable=True))
op.add_column('step_library', sa.Column('source_node_id', sa.String(255), nullable=True))
op.add_column('step_library', sa.Column('is_flow_synced', sa.Boolean(), nullable=False, server_default='false'))
op.add_column('step_library', sa.Column('last_synced_at', sa.DateTime(timezone=True), nullable=True))
op.create_foreign_key(
'fk_step_library_source_tree',
'step_library', 'trees',
['source_tree_id'], ['id'],
ondelete='SET NULL'
)
op.create_unique_constraint(
'uq_step_library_source_node',
'step_library',
['source_tree_id', 'source_node_id']
)
op.create_index('ix_step_library_source_tree_id', 'step_library', ['source_tree_id'])
def downgrade() -> None:
op.drop_index('ix_step_library_source_tree_id', 'step_library')
op.drop_constraint('uq_step_library_source_node', 'step_library', type_='unique')
op.drop_constraint('fk_step_library_source_tree', 'step_library', type_='foreignkey')
op.drop_column('step_library', 'last_synced_at')
op.drop_column('step_library', 'is_flow_synced')
op.drop_column('step_library', 'source_node_id')
op.drop_column('step_library', 'source_tree_id')
```
**Step 2: Run the migration**
```bash
cd /home/michaelchihlas/dev/patherly/backend
alembic upgrade head
```
Expected: `Running upgrade ... -> <new_revision>, add_step_library_sync_fields`
**Step 3: Verify the columns exist**
```bash
docker exec -it patherly_postgres psql -U postgres -d patherly -c "\d step_library" | grep -E "source_|is_flow|last_sync"
```
Expected: 4 new columns listed.
**Step 4: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/alembic/versions/
git commit -m "feat: add sync tracking columns to step_library
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 2: Update `StepLibrary` model with new columns
**Files:**
- Modify: `backend/app/models/step_library.py`
**Step 1: Read the file to find the exact insertion point**
Read `backend/app/models/step_library.py` — find the `is_active` column (last column before relationships). Add the 4 new columns after `is_active`:
```python
# Sync tracking (flow-sourced steps)
source_tree_id: Mapped[Optional[uuid_pkg.UUID]] = mapped_column(
UUID(as_uuid=True),
ForeignKey('trees.id', ondelete='SET NULL'),
nullable=True,
index=True
)
source_node_id: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
is_flow_synced: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
last_synced_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
```
**Step 2: Add the `source_tree` relationship**
In the relationships section of `StepLibrary`, add after the existing `account` relationship:
```python
source_tree: Mapped[Optional["Tree"]] = relationship(
"Tree",
foreign_keys=[source_tree_id],
lazy="select"
)
```
**Step 3: Verify imports**
The file already imports `Optional`, `UUID`, `Boolean`, `String`, `DateTime`, `ForeignKey`, `relationship`. Confirm `mapped_column` and `Mapped` are imported. No new imports should be needed.
**Step 4: Run the backend to check for import errors**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -c "from app.models.step_library import StepLibrary; print('OK')"
```
Expected: `OK`
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/app/models/step_library.py
git commit -m "feat: add sync columns and source_tree relationship to StepLibrary model
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 3: Update schemas and add `group_label` to `StepContent`
**Files:**
- Modify: `backend/app/schemas/step_library.py`
**Step 1: Add `group_label` to `StepContent`**
Find `StepContent` (lines 1519) and add `group_label`:
```python
class StepContent(BaseModel):
instructions: str = Field(..., min_length=1)
help_text: Optional[str] = None
commands: Optional[list[StepCommand]] = None
group_label: Optional[str] = None # Section header this step belongs to (for flow-synced steps)
```
**Step 2: Add `is_flow_synced` and `source_tree_name` to `StepLibraryResponse`**
Find `StepLibraryResponse` (lines 4564) and add two fields at the end:
```python
is_flow_synced: bool = False
source_tree_name: Optional[str] = None
```
**Step 3: Verify import — `Optional` is already imported from typing**
Read the top of the file to confirm. No new imports needed.
**Step 4: Verify Python import**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -c "from app.schemas.step_library import StepLibraryResponse, StepContent; print('OK')"
```
Expected: `OK`
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/app/schemas/step_library.py
git commit -m "feat: add group_label to StepContent, is_flow_synced to StepLibraryResponse
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 4: Update step list endpoint to return `is_flow_synced` and `source_tree_name`
**Files:**
- Modify: `backend/app/api/endpoints/steps.py`
The list endpoint currently returns `StepLibraryResponse` objects built from ORM instances. We need to populate `is_flow_synced` and `source_tree_name`.
**Step 1: Read `backend/app/api/endpoints/steps.py` lines 58139**
Find where the list query is executed and where response objects are built. Look for the SELECT statement and how it maps to `StepLibraryResponse`.
**Step 2: Add `source_tree_name` to the list query via a join**
The list query will need to LEFT JOIN trees to get the name. Find the existing query and add:
```python
from app.models.tree import Tree as TreeModel
# In the list query, add a join:
query = (
select(
StepLibrary,
TreeModel.name.label('source_tree_name')
)
.outerjoin(TreeModel, StepLibrary.source_tree_id == TreeModel.id)
# ... existing filters ...
)
```
Then when building the response, pass `source_tree_name` from the joined result.
**Important:** Read the existing query structure carefully before modifying — match whatever pattern (scalar results, row mapping, etc.) is already used.
**Step 3: Also update the single-step GET endpoint** (lines 221265) to include `source_tree_name` in the same way.
**Step 4: Verify Python import**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -c "from app.api.endpoints.steps import router; print('OK')"
```
Expected: `OK`
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/app/api/endpoints/steps.py
git commit -m "feat: include is_flow_synced and source_tree_name in step list/detail responses
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 5: Create `backend/app/core/step_sync.py`
**Files:**
- Create: `backend/app/core/step_sync.py`
- Test: `backend/tests/test_step_sync.py`
This is the core of the feature. Write the test first.
**Step 1: Write the failing tests**
Create `backend/tests/test_step_sync.py`:
```python
"""Tests for flow-to-library step sync."""
import pytest
from uuid import uuid4
from app.core.step_sync import extract_steps_for_sync, resolve_step_visibility
class TestResolveStepVisibility:
"""Test visibility resolution logic."""
def test_public_flow_gives_public_steps(self):
result = resolve_step_visibility(is_public=True, account_id=None, node_override=None)
assert result == 'public'
def test_team_flow_gives_team_steps(self):
result = resolve_step_visibility(is_public=False, account_id=uuid4(), node_override=None)
assert result == 'team'
def test_private_flow_gives_team_steps(self):
result = resolve_step_visibility(is_public=False, account_id=None, node_override=None)
assert result == 'team'
def test_node_override_takes_precedence(self):
result = resolve_step_visibility(is_public=True, account_id=None, node_override='team')
assert result == 'team'
def test_public_override_on_team_flow(self):
result = resolve_step_visibility(is_public=False, account_id=uuid4(), node_override='public')
assert result == 'public'
class TestExtractStepsForSync:
"""Test step extraction from tree structures."""
def test_extracts_procedure_steps_from_procedural_flow(self):
tree_structure = {
"steps": [
{"id": "step_1", "type": "procedure_step", "title": "Verify prerequisites",
"description": "Check all prereqs", "content_type": "action"},
{"id": "end_1", "type": "procedure_end", "title": "Done"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert len(results) == 1
assert results[0]['source_node_id'] == 'step_1'
assert results[0]['title'] == 'Verify prerequisites'
assert results[0]['step_type'] == 'action'
assert results[0]['content']['instructions'] == 'Check all prereqs'
def test_skips_section_header_nodes(self):
tree_structure = {
"steps": [
{"id": "sec_1", "type": "section_header", "title": "Phase 1"},
{"id": "step_1", "type": "procedure_step", "title": "First step",
"description": "Do this"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert len(results) == 1
assert results[0]['source_node_id'] == 'step_1'
def test_captures_section_header_as_group_label(self):
tree_structure = {
"steps": [
{"id": "sec_1", "type": "section_header", "title": "Cable Checks"},
{"id": "step_1", "type": "procedure_step", "title": "Check cable",
"description": "Verify cable is seated"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert results[0]['content']['group_label'] == 'Cable Checks'
def test_normalizes_string_commands(self):
tree_structure = {
"steps": [
{"id": "step_1", "type": "procedure_step", "title": "Run command",
"description": "Execute this", "commands": "ping 8.8.8.8"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert results[0]['content']['commands'] == [{"label": "", "command": "ping 8.8.8.8", "command_type": None}]
def test_normalizes_commandblock_commands(self):
tree_structure = {
"steps": [
{"id": "step_1", "type": "procedure_step", "title": "Run PS",
"description": "Run powershell",
"commands": [{"code": "Get-Service", "language": "powershell", "label": "Check services"}]},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
cmds = results[0]['content']['commands']
assert len(cmds) == 1
assert cmds[0]['command'] == 'Get-Service'
assert cmds[0]['command_type'] == 'powershell'
assert cmds[0]['label'] == 'Check services'
def test_extracts_action_and_solution_from_troubleshooting(self):
tree_structure = {
"id": "root",
"type": "decision",
"question": "What is wrong?",
"options": [{"id": "o1", "label": "Thing A", "next_node_id": "act_1"}],
"children": [
{"id": "act_1", "type": "action", "title": "Fix thing A",
"description": "Do the fix", "next_node_id": "sol_1",
"children": [{"id": "sol_1", "type": "solution", "title": "All fixed",
"description": "Problem resolved"}]},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='troubleshooting'))
node_ids = {r['source_node_id'] for r in results}
assert 'act_1' in node_ids
assert 'sol_1' in node_ids
types = {r['source_node_id']: r['step_type'] for r in results}
assert types['act_1'] == 'action'
assert types['sol_1'] == 'solution'
def test_uses_title_as_instructions_fallback(self):
tree_structure = {
"steps": [
{"id": "step_1", "type": "procedure_step", "title": "Do the thing"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert results[0]['content']['instructions'] == 'Do the thing'
def test_empty_steps_list(self):
tree_structure = {"steps": []}
results = list(extract_steps_for_sync(tree_structure, tree_type='procedural'))
assert results == []
def test_maintenance_treated_same_as_procedural(self):
tree_structure = {
"steps": [
{"id": "step_1", "type": "procedure_step", "title": "Maintenance step",
"description": "Do maintenance"},
]
}
results = list(extract_steps_for_sync(tree_structure, tree_type='maintenance'))
assert len(results) == 1
```
**Step 2: Run tests to verify they fail**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -m pytest tests/test_step_sync.py -v --override-ini="addopts="
```
Expected: FAIL — `ImportError: cannot import name 'extract_steps_for_sync'`
**Step 3: Implement `backend/app/core/step_sync.py`**
```python
"""Sync steps from published flows into the step library."""
from __future__ import annotations
from typing import Any, Generator, Literal, Optional
from uuid import UUID, uuid4
from datetime import datetime, timezone
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
StepVisibility = Literal['private', 'team', 'public']
def resolve_step_visibility(
is_public: bool,
account_id: Optional[UUID],
node_override: Optional[str],
) -> StepVisibility:
"""Resolve the visibility for a synced step.
Priority: node-level library_visibility > flow visibility.
Flow visibility: public if is_public, otherwise 'team'.
"""
if node_override in ('team', 'public'):
return node_override # type: ignore[return-value]
return 'public' if is_public else 'team'
def _normalize_commands(raw: Any) -> list[dict]:
"""Normalize command field to list of StepCommand dicts."""
if not raw:
return []
if isinstance(raw, str):
return [{"label": "", "command": raw, "command_type": None}]
if isinstance(raw, list):
result = []
for item in raw:
if isinstance(item, str):
result.append({"label": "", "command": item, "command_type": None})
elif isinstance(item, dict):
result.append({
"label": item.get("label", ""),
"command": item.get("code", item.get("command", "")),
"command_type": item.get("language", item.get("command_type")),
})
return result
return []
def _walk_troubleshooting(node: dict) -> Generator[dict, None, None]:
"""Recursively yield action and solution nodes from a troubleshooting tree."""
node_type = node.get("type")
if node_type in ("action", "solution"):
yield node
for child in node.get("children", []):
yield from _walk_troubleshooting(child)
def extract_steps_for_sync(
tree_structure: dict,
tree_type: str,
) -> Generator[dict, None, None]:
"""Extract step dicts ready for upsert from a tree structure.
Yields dicts with keys:
source_node_id, title, step_type, content (dict), node_visibility_override
"""
if tree_type in ("procedural", "maintenance"):
steps = tree_structure.get("steps", [])
current_section: Optional[str] = None
for node in steps:
node_type = node.get("type")
if node_type == "section_header":
current_section = node.get("title") or node.get("section_header")
continue
if node_type != "procedure_step":
continue
description = node.get("description") or node.get("title", "")
content: dict = {
"instructions": description,
"help_text": node.get("expected_outcome"),
"commands": _normalize_commands(node.get("commands")) or None,
"group_label": current_section,
}
# Remove None values for cleanliness
content = {k: v for k, v in content.items() if v is not None}
# instructions is required — ensure it's present
content.setdefault("instructions", node.get("title", ""))
yield {
"source_node_id": node["id"],
"title": node.get("title", "Untitled step"),
"step_type": "action",
"content": content,
"node_visibility_override": node.get("library_visibility"),
}
elif tree_type == "troubleshooting":
for node in _walk_troubleshooting(tree_structure):
description = node.get("description") or node.get("title", "")
content = {
"instructions": description,
}
yield {
"source_node_id": node["id"],
"title": node.get("title", "Untitled step"),
"step_type": "action" if node["type"] == "action" else "solution",
"content": content,
"node_visibility_override": None,
}
async def sync_steps_from_tree(
db: AsyncSession,
tree_id: UUID,
tree_type: str,
tree_structure: dict,
author_id: UUID,
account_id: Optional[UUID],
is_public: bool,
) -> int:
"""Upsert step library entries from a published tree.
Returns the number of steps synced.
"""
from app.models.step_library import StepLibrary # avoid circular import
count = 0
now = datetime.now(timezone.utc)
for step_data in extract_steps_for_sync(tree_structure, tree_type):
visibility = resolve_step_visibility(
is_public=is_public,
account_id=account_id,
node_override=step_data["node_visibility_override"],
)
# Use raw SQL upsert keyed on (source_tree_id, source_node_id)
await db.execute(
text("""
INSERT INTO step_library (
id, title, step_type, content, created_by, account_id,
visibility, is_flow_synced, source_tree_id, source_node_id,
last_synced_at, tags, is_active,
usage_count, rating_average, rating_count,
helpful_yes, helpful_no, is_featured, is_verified,
created_at, updated_at
) VALUES (
gen_random_uuid(), :title, :step_type, :content::jsonb,
:created_by, :account_id, :visibility, true,
:source_tree_id, :source_node_id, :last_synced_at,
'{}', true,
0, 0, 0, 0, 0, false, false,
:now, :now
)
ON CONFLICT (source_tree_id, source_node_id)
DO UPDATE SET
title = EXCLUDED.title,
step_type = EXCLUDED.step_type,
content = EXCLUDED.content,
visibility = EXCLUDED.visibility,
last_synced_at = EXCLUDED.last_synced_at,
updated_at = EXCLUDED.updated_at,
is_active = true
"""),
{
"title": step_data["title"],
"step_type": step_data["step_type"],
"content": __import__('json').dumps(step_data["content"]),
"created_by": str(author_id),
"account_id": str(account_id) if account_id else None,
"visibility": visibility,
"source_tree_id": str(tree_id),
"source_node_id": step_data["source_node_id"],
"last_synced_at": now,
"now": now,
}
)
count += 1
# Soft-delete any previously synced steps for this tree that no longer exist
current_node_ids = [
s["source_node_id"]
for s in extract_steps_for_sync(tree_structure, tree_type)
]
if current_node_ids:
await db.execute(
text("""
UPDATE step_library
SET is_active = false, updated_at = :now
WHERE source_tree_id = :tree_id
AND is_flow_synced = true
AND source_node_id NOT IN :node_ids
"""),
{"tree_id": str(tree_id), "node_ids": tuple(current_node_ids), "now": now}
)
else:
# No steps extracted — deactivate all synced entries for this tree
await db.execute(
text("""
UPDATE step_library
SET is_active = false, updated_at = :now
WHERE source_tree_id = :tree_id AND is_flow_synced = true
"""),
{"tree_id": str(tree_id), "now": now}
)
return count
async def deactivate_synced_steps_for_tree(db: AsyncSession, tree_id: UUID) -> None:
"""Soft-delete all synced library entries for a tree (called on tree delete/deactivate)."""
await db.execute(
text("""
UPDATE step_library
SET is_active = false, updated_at = :now
WHERE source_tree_id = :tree_id AND is_flow_synced = true
"""),
{"tree_id": str(tree_id), "now": datetime.now(timezone.utc)}
)
```
**Step 4: Run tests**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -m pytest tests/test_step_sync.py -v --override-ini="addopts="
```
Expected: All tests PASS.
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/app/core/step_sync.py backend/tests/test_step_sync.py
git commit -m "feat: add step_sync module with extraction and upsert logic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 6: Wire sync into the publish endpoint
**Files:**
- Modify: `backend/app/api/endpoints/trees.py`
**Step 1: Read lines 583650 of the file to confirm exact positions**
Confirm line numbers for:
- The `if "status" in update_data and update_data["status"] == 'published':` block
- The version increment at `tree.version += 1`
- The tag replacement block that follows
**Step 2: Add the import at the top of the file**
Near the other core imports (grep for `from app.core`), add:
```python
from app.core.step_sync import sync_steps_from_tree, deactivate_synced_steps_for_tree
```
**Step 3: Insert sync call after version increment**
After `tree.version += 1` (line ~641) and before the tag replacement block, add:
```python
# Sync steps to library when publishing
if update_data.get("status") == 'published' or tree.status == 'published':
final_structure = update_data.get("tree_structure", tree.tree_structure)
final_type = update_data.get("tree_type", tree.tree_type)
await sync_steps_from_tree(
db=db,
tree_id=tree.id,
tree_type=final_type,
tree_structure=final_structure,
author_id=tree.author_id,
account_id=tree.account_id,
is_public=update_data.get("is_public", tree.is_public),
)
```
**Step 4: Add deactivation on tree delete**
Find the delete tree endpoint (search for `@router.delete`). After confirming the tree is being deleted, add before commit:
```python
await deactivate_synced_steps_for_tree(db, tree_id)
```
**Step 5: Verify Python import**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -c "from app.api.endpoints.trees import router; print('OK')"
```
Expected: `OK`
**Step 6: Write integration test**
Add to `backend/tests/test_step_sync.py` a new class:
```python
class TestSyncOnPublish:
"""Integration tests — sync triggered by publishing a tree."""
@pytest.mark.asyncio
async def test_publishing_procedural_tree_creates_library_steps(
self, client, auth_headers, test_db
):
# Create a procedural tree
tree_resp = await client.post("/trees", json={
"name": "Test Procedure",
"tree_type": "procedural",
"tree_structure": {
"steps": [
{"id": "step_1", "type": "procedure_step",
"title": "First step", "description": "Do this first"},
{"id": "step_2", "type": "procedure_step",
"title": "Second step", "description": "Do this second"},
{"id": "end_1", "type": "procedure_end", "title": "Done"},
]
}
}, headers=auth_headers)
assert tree_resp.status_code == 201
tree_id = tree_resp.json()["id"]
# Publish the tree
pub_resp = await client.put(f"/trees/{tree_id}", json={"status": "published"}, headers=auth_headers)
assert pub_resp.status_code == 200
# Check library has entries
lib_resp = await client.get("/steps", headers=auth_headers)
assert lib_resp.status_code == 200
steps = lib_resp.json()
synced = [s for s in steps if s.get("is_flow_synced")]
assert len(synced) == 2
titles = {s["title"] for s in synced}
assert "First step" in titles
assert "Second step" in titles
@pytest.mark.asyncio
async def test_republishing_updates_existing_library_steps(
self, client, auth_headers, test_db
):
# Create and publish
tree_resp = await client.post("/trees", json={
"name": "Update Test",
"tree_type": "procedural",
"tree_structure": {"steps": [
{"id": "step_1", "type": "procedure_step",
"title": "Original title", "description": "Original desc"},
]}
}, headers=auth_headers)
tree_id = tree_resp.json()["id"]
await client.put(f"/trees/{tree_id}", json={"status": "published"}, headers=auth_headers)
# Update step title and republish
await client.put(f"/trees/{tree_id}", json={
"tree_structure": {"steps": [
{"id": "step_1", "type": "procedure_step",
"title": "Updated title", "description": "Updated desc"},
]},
"status": "published"
}, headers=auth_headers)
# Check library entry was updated
lib_resp = await client.get("/steps", headers=auth_headers)
synced = [s for s in lib_resp.json() if s.get("is_flow_synced")]
assert len(synced) == 1
assert synced[0]["title"] == "Updated title"
```
**Step 7: Run integration tests**
```bash
cd /home/michaelchihlas/dev/patherly/backend
backend/venv/bin/python -m pytest tests/test_step_sync.py -v --override-ini="addopts="
```
Expected: All tests pass.
**Step 8: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add backend/app/api/endpoints/trees.py backend/tests/test_step_sync.py
git commit -m "feat: trigger step library sync on tree publish/delete
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 7: Frontend — update Step types
**Files:**
- Modify: `frontend/src/types/step.ts`
**Step 1: Read the file to find `Step` and `StepListItem` interfaces**
**Step 2: Add `is_flow_synced` and `source_tree_name` to `Step` (after `is_verified`)**
```typescript
is_flow_synced: boolean
source_tree_name: string | null
```
**Step 3: Add same fields to `StepListItem`**
`StepListItem` is the subset used in list views — add there too:
```typescript
is_flow_synced: boolean
source_tree_name: string | null
```
**Step 4: Verify build**
```bash
cd /home/michaelchihlas/dev/patherly/frontend && npm run build 2>&1 | tail -10
```
Expected: Clean build.
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add frontend/src/types/step.ts
git commit -m "feat: add is_flow_synced and source_tree_name to Step types
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 8: Frontend — read-only indicator in `StepCard`
**Files:**
- Modify: `frontend/src/components/step-library/StepCard.tsx`
**Step 1: Read the full `StepCard.tsx` to understand current structure**
Find where the Edit button is rendered (look for `onEdit` prop usage and the edit button JSX).
**Step 2: Add `Lock` to Lucide imports**
```tsx
import { ..., Lock } from 'lucide-react'
```
**Step 3: Replace the edit button condition**
Currently the edit button likely shows when `isOwn` is true. Change it so that for flow-synced steps, the edit button is replaced by a lock icon with a tooltip:
```tsx
{isOwn && (
step.is_flow_synced ? (
<span
title="Managed by source flow — fork to customize"
className="flex items-center justify-center rounded-md border border-border p-2 text-muted-foreground cursor-default opacity-50"
>
<Lock className="h-4 w-4" />
</span>
) : (
onEdit && (
<button
type="button"
onClick={() => onEdit(step)}
className="rounded-md border border-border p-2 text-muted-foreground hover:bg-accent hover:text-foreground"
title="Edit step"
>
<Pencil className="h-4 w-4" />
</button>
)
)
)}
```
**Step 4: Add "From Flow" badge**
In the badges/chips area of the card (near where `is_featured` or `is_verified` badges are shown), add:
```tsx
{step.is_flow_synced && (
<span className="rounded-full bg-blue-400/15 px-1.5 py-0.5 text-[9px] font-semibold uppercase tracking-wide text-blue-400">
From Flow
</span>
)}
```
**Step 5: Verify build**
```bash
cd /home/michaelchihlas/dev/patherly/frontend && npm run build 2>&1 | tail -10
```
Expected: Clean build.
**Step 6: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add frontend/src/components/step-library/StepCard.tsx
git commit -m "feat: show From Flow badge and lock icon on flow-synced StepCard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 9: Frontend — source flow link in `StepDetailModal`
**Files:**
- Modify: `frontend/src/components/step-library/StepDetailModal.tsx`
**Step 1: Read `StepDetailModal.tsx` to find where step metadata is displayed**
Look for where `author_name`, `usage_count`, `category_name` are rendered in the detail panel.
**Step 2: Add source flow attribution**
In the metadata section, add after the author line:
```tsx
{step.is_flow_synced && step.source_tree_name && (
<div className="flex items-center gap-1.5 text-xs text-muted-foreground">
<GitBranch className="h-3.5 w-3.5 shrink-0" />
<span>Sourced from <span className="font-medium text-foreground">{step.source_tree_name}</span></span>
</div>
)}
```
Add `GitBranch` to Lucide imports if not already present.
**Step 3: Verify build**
```bash
cd /home/michaelchihlas/dev/patherly/frontend && npm run build 2>&1 | tail -10
```
Expected: Clean build.
**Step 4: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add frontend/src/components/step-library/StepDetailModal.tsx
git commit -m "feat: show source flow name in StepDetailModal for synced steps
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
### Task 10: Frontend — Library Visibility select in `StepEditor`
**Files:**
- Modify: `frontend/src/components/procedural-editor/StepEditor.tsx`
- Modify: `frontend/src/types/tree.ts` (add `library_visibility` to `ProceduralStep`)
**Step 1: Add `library_visibility` to `ProceduralStep` in `frontend/src/types/tree.ts`**
Find `ProceduralStep` (line ~106) and add after `reference_url?`:
```typescript
library_visibility?: 'team' | 'public'
```
**Step 2: Read `StepEditor.tsx` lines 130200 to find the More Options block**
The `{showMore && <div>}` block starts around line 150. Find the last field inside it.
**Step 3: Add Library Visibility select inside the More Options block**
At the end of the `showMore` content block, before the closing `</div>`, add:
```tsx
{/* Library Visibility — only for procedure_step nodes */}
{step.type === 'procedure_step' && (
<div>
<label className="mb-1.5 block text-xs font-medium text-muted-foreground">
Library Visibility
</label>
<select
value={step.library_visibility ?? ''}
onChange={(e) => onUpdate({
library_visibility: e.target.value === '' ? undefined : e.target.value as 'team' | 'public'
})}
className="w-full rounded-lg border border-border bg-card px-3 py-2 text-sm text-foreground focus:border-primary focus:outline-none focus:ring-1 focus:ring-primary/20"
>
<option value="">Inherit from flow</option>
<option value="team">Team only</option>
<option value="public">Public</option>
</select>
<p className="mt-1 text-[10px] text-muted-foreground">
Controls visibility in the step library. Defaults to the flow's own visibility setting.
</p>
</div>
)}
```
**Step 4: Verify build**
```bash
cd /home/michaelchihlas/dev/patherly/frontend && npm run build 2>&1 | tail -10
```
Expected: Clean build.
**Step 5: Commit**
```bash
cd /home/michaelchihlas/dev/patherly
git add frontend/src/types/tree.ts frontend/src/components/procedural-editor/StepEditor.tsx
git commit -m "feat: add Library Visibility select to procedural StepEditor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>"
```
---
## Manual Verification Checklist
1. **Publish sync:** Create a procedural flow with 2+ steps and a section header → publish → open Step Library → synced steps appear with "From Flow" badge, correct section in group_label
2. **Republish update:** Edit a step title in a published flow → republish → Step Library shows updated title (not a duplicate)
3. **Visibility inherit:** Public flow → synced steps have `visibility: 'public'`. Team flow → `visibility: 'team'`
4. **Per-step override:** Set Library Visibility to "Team only" on a step in a public flow → publish → that step appears as `team` in library while others are `public`
5. **Read-only in library:** Synced step shows lock icon instead of edit button, tooltip reads "Managed by source flow"
6. **Source flow name:** Click a synced step → detail panel shows "Sourced from: [Flow Name]"
7. **Fork works:** Click "Save to Library" on a synced step → creates a personal copy with `is_flow_synced: false`, editable normally
8. **Troubleshooting sync:** Publish a troubleshooting flow → action and solution nodes appear in library (decision nodes do not)