From 208807cf94bbba5701505a3cbe9837c29d4d5516 Mon Sep 17 00:00:00 2001 From: chihlasm Date: Sat, 28 Feb 2026 20:43:34 -0500 Subject: [PATCH] docs: add cross-reference / loop-back implementation plan Co-Authored-By: Claude Opus 4.6 --- .../2026-02-28-cross-reference-loopback.md | 656 ++++++++++++++++++ 1 file changed, 656 insertions(+) create mode 100644 docs/plans/2026-02-28-cross-reference-loopback.md diff --git a/docs/plans/2026-02-28-cross-reference-loopback.md b/docs/plans/2026-02-28-cross-reference-loopback.md new file mode 100644 index 00000000..1b516289 --- /dev/null +++ b/docs/plans/2026-02-28-cross-reference-loopback.md @@ -0,0 +1,656 @@ +# Cross-Reference / Loop-Back Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Enable tree nodes to reference any other node in the tree (not just direct children), supporting loop-back patterns like "remediate → re-verify from earlier checkpoint." + +**Architecture:** Ghost references on existing tree structure. No schema change, no migration. A cross-reference is any `next_node_id` that points outside the current node's `children` array. The canvas renders these as dashed overlay arrows. Navigation already supports this pattern. + +**Tech Stack:** Python FastAPI (backend validation), React + @xyflow/react (canvas rendering), Zustand (store validation), TypeScript + +**Design Doc:** `docs/plans/2026-02-28-cross-reference-loopback-design.md` + +--- + +## Task 1: Backend — Relax Decision Option Validation + +Allow decision option `next_node_id` to reference ANY node in the tree, not just direct children. + +**Files:** +- Modify: `backend/app/core/ai_tree_validator.py:111-118` +- Test: `backend/tests/test_ai_tree_validator.py` + +**Step 1: Write the failing test — cross-reference option passes validation** + +Add a new test class `TestCrossReferenceSupport` at the bottom of `test_ai_tree_validator.py`: + +```python +class TestCrossReferenceSupport: + def test_option_referencing_non_child_node_in_tree_is_valid(self): + """A decision option can reference any node in the tree, not just direct children.""" + tree = _make_valid_tree() + # Make root option point to a grandchild (not a direct child) — cross-reference + tree["options"][0]["next_node_id"] = "fix-errors" # grandchild of root + errors = validate_generated_tree(tree) + # Should NOT have the "non-existent child" error for this reference + assert not any("non-existent child" in e for e in errors) + + def test_option_referencing_nonexistent_node_still_fails(self): + """Cross-references must still point to nodes that exist in the tree.""" + tree = _make_valid_tree() + tree["options"][0]["next_node_id"] = "totally-fake-id" + errors = validate_generated_tree(tree) + assert any("does not exist" in e for e in errors) + + def test_action_next_node_id_to_ancestor_is_valid(self): + """Action node can loop back to an ancestor node (the whole point of cross-refs).""" + tree = _make_valid_tree() + # Make the action node loop back to root + tree["children"][1]["next_node_id"] = "root" + errors = validate_generated_tree(tree) + assert not any("does not exist" in e for e in errors) +``` + +**Step 2: Run the tests to verify they fail** + +Run: `cd backend && python -m pytest tests/test_ai_tree_validator.py::TestCrossReferenceSupport -v` +Expected: `test_option_referencing_non_child_node_in_tree_is_valid` FAILS (currently raises "non-existent child" error). The other two should already pass. + +**Step 3: Update validator — check global existence, not just children** + +In `backend/app/core/ai_tree_validator.py`, replace lines 111-118 (the decision option next_node_id check): + +Old code (lines 111-118): +```python + next_id = opt.get("next_node_id") + if next_id: + all_referenced_ids.add(next_id) + if child_ids and next_id not in child_ids: + errors.append( + f"Option '{opt.get('label', '?')}' in node '{node_id}' " + f"references non-existent child '{next_id}'" + ) +``` + +New code: +```python + next_id = opt.get("next_node_id") + if next_id: + all_referenced_ids.add(next_id) +``` + +Then add a new global check after line 145 (after the action next_node_id existence check). This checks ALL option references exist anywhere in the tree: + +After the existing `for ref_id in action_next_ids:` block, add: +```python + # Check that all option next_node_ids exist in the tree (allows cross-references) + for ref_id in all_referenced_ids: + if ref_id not in all_ids: + errors.append( + f"Option next_node_id '{ref_id}' references a node that does not exist in the tree" + ) +``` + +**Step 4: Run all validator tests to verify they pass** + +Run: `cd backend && python -m pytest tests/test_ai_tree_validator.py -v` +Expected: ALL tests pass. The old `test_option_references_nonexistent_child` test in `TestReferenceIntegrity` will now fail because the error message changed from "non-existent child" to "does not exist in the tree". Update that test: + +In `TestReferenceIntegrity.test_option_references_nonexistent_child`, change: +```python + def test_option_references_nonexistent_child(self): + tree = _make_valid_tree() + tree["options"][0]["next_node_id"] = "nonexistent" + errors = validate_generated_tree(tree) + assert any("does not exist" in e for e in errors) +``` + +Run again: `cd backend && python -m pytest tests/test_ai_tree_validator.py -v` +Expected: ALL PASS + +**Step 5: Commit** + +```bash +git add backend/app/core/ai_tree_validator.py backend/tests/test_ai_tree_validator.py +git commit -m "feat: relax decision option validation — allow cross-references to any node in tree" +``` + +--- + +## Task 2: Frontend — Change Circular Reference Detection From Error to Warning + +Loop-backs are now intentional. The circular reference detector should warn instead of error. + +**Files:** +- Modify: `frontend/src/store/treeEditorStore.ts:791-824` + +**Step 1: Update severity from 'error' to 'warning' and improve messages** + +In `frontend/src/store/treeEditorStore.ts`, find the `detectCircularRefs` function (lines 792-824). Change both `severity: 'error'` to `severity: 'warning'` and update messages: + +Replace line 803-807: +```typescript + errors.push({ + nodeId: node.id, + message: `Circular reference detected: "${opt.label}" creates a loop`, + severity: 'error' + }) +``` +With: +```typescript + errors.push({ + nodeId: node.id, + message: `This path loops back to an earlier node via "${opt.label}"`, + severity: 'warning' + }) +``` + +Replace lines 815-819: +```typescript + errors.push({ + nodeId: node.id, + message: `Circular reference detected in node "${node.title || node.id}"`, + severity: 'error' + }) +``` +With: +```typescript + errors.push({ + nodeId: node.id, + message: `This node loops back to an earlier node ("${node.title || node.id}")`, + severity: 'warning' + }) +``` + +**Step 2: Build to verify** + +Run: `cd frontend && npm run build` +Expected: Build succeeds with no errors. + +**Step 3: Commit** + +```bash +git add frontend/src/store/treeEditorStore.ts +git commit -m "feat: change circular reference detection from error to warning — loops are intentional" +``` + +--- + +## Task 3: Canvas — Render Cross-Reference Edges as Dashed Arrows + +Add dashed purple overlay edges for any `next_node_id` pointing outside the current node's children. + +**Files:** +- Modify: `frontend/src/components/tree-editor/useTreeLayout.ts` + +**Step 1: Add cross-reference edge collection to the `walk` function** + +In `useTreeLayout.ts`, inside the `useMemo` callback (line 57), after the `walk(treeStructure, null)` call on line 129, add a second pass to collect cross-reference edges. + +Add this helper function before the `useTreeLayout` export (around line 40): + +```typescript +/** Collect all node IDs in the tree. */ +function collectAllIds(root: TreeStructure): Set { + const ids = new Set() + function walk(node: TreeStructure) { + ids.add(node.id) + node.children?.forEach(walk) + } + walk(root) + return ids +} + +/** Find all cross-reference edges (next_node_id pointing outside children). */ +function collectCrossRefEdges(root: TreeStructure): Array<{ source: string; target: string; label?: string }> { + const refs: Array<{ source: string; target: string; label?: string }> = [] + const allIds = collectAllIds(root) + + function walk(node: TreeStructure) { + const childIds = new Set(node.children?.map(c => c.id) ?? []) + + // Decision options pointing outside children + if (node.type === 'decision' && node.options) { + for (const opt of node.options) { + if (opt.next_node_id && !childIds.has(opt.next_node_id) && allIds.has(opt.next_node_id)) { + refs.push({ source: node.id, target: opt.next_node_id, label: opt.label }) + } + } + } + + // Action next_node_id pointing to non-child (always a cross-ref since actions use next_node_id not children) + if (node.type === 'action' && node.next_node_id && allIds.has(node.next_node_id) && !childIds.has(node.next_node_id)) { + refs.push({ source: node.id, target: node.next_node_id, label: 'loops back' }) + } + + node.children?.forEach(walk) + } + + walk(root) + return refs +} +``` + +**Step 2: Add cross-reference edges to the edges array** + +In the `useMemo` callback, after `walk(treeStructure, null)` (line 129) and before the return (line 131), add: + +```typescript + // Add cross-reference edges (dashed, purple) + if (treeStructure) { + const crossRefs = collectCrossRefEdges(treeStructure) + for (const ref of crossRefs) { + // Only add if both source and target nodes are visible (not collapsed away) + const sourceVisible = nodes.some(n => n.id === ref.source) + const targetVisible = nodes.some(n => n.id === ref.target) + if (sourceVisible && targetVisible) { + edges.push({ + id: `xref-${ref.source}->${ref.target}`, + source: ref.source, + target: ref.target, + type: 'smoothstep', + animated: true, + label: ref.label ? truncateLabel(ref.label) : undefined, + labelStyle: { fill: 'hsl(var(--primary))', fontSize: 10, fontWeight: 500 }, + labelBgStyle: { fill: 'hsl(var(--card))', fillOpacity: 0.95 }, + labelBgPadding: [4, 2] as [number, number], + style: { + stroke: 'hsl(var(--primary))', + strokeWidth: 2, + strokeDasharray: '6 3', + }, + markerEnd: { + type: 'arrowclosed' as const, + color: 'hsl(var(--primary))', + width: 16, + height: 16, + }, + }) + } + } + } +``` + +**Step 3: Build to verify** + +Run: `cd frontend && npm run build` +Expected: Build succeeds. Cross-reference edges render as animated, dashed purple arrows with arrowheads. + +**Step 4: Commit** + +```bash +git add frontend/src/components/tree-editor/useTreeLayout.ts +git commit -m "feat: render cross-reference edges as dashed purple arrows on canvas" +``` + +--- + +## Task 4: Editor UX — Node Picker Dropdown for Action Nodes + +Add a "Link to existing node" dropdown to `NodeFormAction.tsx`. + +**Files:** +- Modify: `frontend/src/components/tree-editor/NodeFormAction.tsx` +- Modify: `frontend/src/store/treeEditorStore.ts` (add helper to collect all nodes) + +**Step 1: Add `collectAllNodes` helper to the tree editor store** + +In `frontend/src/store/treeEditorStore.ts`, add a standalone exported helper function (near the top of the file, after imports, or as a utility): + +Find the `findNodeInTree` helper function. Near it, add: + +```typescript +/** Collect all nodes in the tree as a flat list with depth info. */ +export function collectAllNodesFlat( + root: TreeStructure | null +): Array<{ id: string; label: string; type: string; depth: number }> { + if (!root) return [] + const result: Array<{ id: string; label: string; type: string; depth: number }> = [] + + function walk(node: TreeStructure, depth: number) { + const label = node.type === 'decision' + ? (node.question || 'Untitled Decision') + : (node.title || `Untitled ${node.type}`) + result.push({ id: node.id, label, type: node.type, depth }) + node.children?.forEach(child => walk(child, depth + 1)) + } + + walk(root, 0) + return result +} +``` + +**Step 2: Update NodeFormAction to include the node picker** + +Replace the "Next step hint" section at the bottom of `NodeFormAction.tsx` (lines 161-170) with a full node picker: + +```tsx +import { Link2, X } from 'lucide-react' +import { collectAllNodesFlat } from '@/store/treeEditorStore' +``` + +(Add these to existing imports at top of file) + +Replace lines 161-170 (the `{hasNextNode ? ... : ...}` block): + +```tsx + {/* Link to existing node */} +
+ + {hasNextNode ? ( +
+ + Linked to: {(() => { + const treeStructure = useTreeEditorStore.getState().treeStructure + const allNodes = collectAllNodesFlat(treeStructure) + const target = allNodes.find(n => n.id === node.next_node_id) + return target ? target.label : node.next_node_id + })()} + + +
+ ) : ( + + )} +

+ {hasNextNode + ? 'This action will navigate to the linked node.' + : 'Select a node to navigate to after this action, or save to create a new placeholder.'} +

+
+``` + +**Step 3: Build to verify** + +Run: `cd frontend && npm run build` +Expected: Build succeeds. + +**Step 4: Commit** + +```bash +git add frontend/src/components/tree-editor/NodeFormAction.tsx frontend/src/store/treeEditorStore.ts +git commit -m "feat: add node picker dropdown to action node form for cross-references" +``` + +--- + +## Task 5: Editor UX — Node Picker for Decision Option Rows + +Add "Link to existing node" capability to each decision option row. + +**Files:** +- Modify: `frontend/src/components/tree-editor/NodeFormDecision.tsx` + +**Step 1: Add link icon and dropdown per option row** + +Add imports at top of `NodeFormDecision.tsx`: +```tsx +import { Link2 } from 'lucide-react' +import { collectAllNodesFlat } from '@/store/treeEditorStore' +``` + +In the option render callback (inside `renderItem` around line 161), after the label input and its error message, add a cross-reference link indicator per option. After the closing `` of the `flex-1` wrapper (around line 197), add: + +```tsx + {/* Cross-reference link indicator */} + {option.next_node_id && (() => { + const treeStructure = useTreeEditorStore.getState().treeStructure + const childIds = new Set(node.children?.map(c => c.id) ?? []) + // Only show if it's a cross-reference (points outside children) + if (childIds.has(option.next_node_id)) return null + const allNodes = collectAllNodesFlat(treeStructure) + const target = allNodes.find(n => n.id === option.next_node_id) + if (!target) return null + return ( +
+ +
+ ) + })()} +``` + +**Step 2: Add "Link to node" option below the options list** + +After the `DynamicArrayField` closing tag (line 201, before the root tip), add: + +```tsx + {/* Quick-link: assign an option to an existing node */} +
+ + + Link an option to an existing node (cross-reference) + +
+

+ Select an option, then pick a target node. This creates a loop-back or cross-reference. +

+
+ + +
+
+
+``` + +**Step 2: Build to verify** + +Run: `cd frontend && npm run build` +Expected: Build succeeds. + +**Step 3: Commit** + +```bash +git add frontend/src/components/tree-editor/NodeFormDecision.tsx +git commit -m "feat: add cross-reference node picker to decision option rows" +``` + +--- + +## Task 6: AI System Prompt — Update Structural Rules for Cross-References + +Update the AI chat system prompt to allow and encourage loop-back patterns. + +**Files:** +- Modify: `backend/app/core/ai_chat_service.py:68-75` + +**Step 1: Update STRUCTURAL RULES in SCHEMA_CONTEXT** + +Replace the STRUCTURAL RULES section (lines 68-75 of `ai_chat_service.py`): + +Old: +```python +STRUCTURAL RULES: +- Root node MUST be type "decision" +- Decision nodes contain their children in the "children" array +- Each decision option's next_node_id must reference a child node's id +- Action nodes use next_node_id to chain to the next step (NOT children) +- Solution nodes are terminal — no next_node_id or children +- All IDs must be unique strings (use descriptive slugs like "check-service-status") +``` + +New: +```python +STRUCTURAL RULES: +- Root node MUST be type "decision" +- Decision nodes contain their children in the "children" array +- Each decision option's next_node_id typically references a child node's id, BUT can also reference ANY other node in the tree for loop-back / re-verification patterns +- Action nodes use next_node_id to chain to the next step — this can point to any node in the tree, including ancestors, for loop-backs (e.g., "remediate → re-verify from earlier checkpoint") +- Solution nodes are terminal — no next_node_id or children +- All IDs must be unique strings (use descriptive slugs like "check-service-status") + +CROSS-REFERENCE / LOOP-BACK PATTERN: +When a troubleshooting path needs to loop back (e.g., after remediation, re-verify from an earlier checkpoint), set next_node_id to the target node's ID. Example: an action node "restart-ssh-service" can set next_node_id to "verify-ssh-connection" (an ancestor decision node) to create a re-verification loop. +``` + +**Step 2: Build backend to verify syntax** + +Run: `cd backend && python -c "from app.core.ai_chat_service import SCHEMA_CONTEXT; print('OK')"` +Expected: Prints "OK" with no import errors. + +**Step 3: Commit** + +```bash +git add backend/app/core/ai_chat_service.py +git commit -m "feat: update AI system prompt to allow cross-reference loop-back patterns" +``` + +--- + +## Task 7: Backend — Update Option Validation Error for `all_referenced_ids` + +The `all_referenced_ids` set currently holds only option `next_node_id` values. After Task 1's change, the global existence check also needs to handle the case where `action_next_ids` and `all_referenced_ids` may overlap. + +**Files:** +- Modify: `backend/app/core/ai_tree_validator.py` +- Test: `backend/tests/test_ai_tree_validator.py` + +**Step 1: Verify no double-counting between action and option refs** + +Check: `action_next_ids` are added to `all_referenced_ids` on line 128. After Task 1, we added a global check for `all_referenced_ids`. This means action refs get checked twice — once in the action-specific loop (lines 141-145) and once in the new option loop. We should only check option refs in the new loop. + +Update the global check added in Task 1 to exclude action refs: + +```python + # Check that all option next_node_ids exist in the tree (allows cross-references) + for ref_id in all_referenced_ids - action_next_ids: + if ref_id not in all_ids: + errors.append( + f"Option next_node_id '{ref_id}' references a node that does not exist in the tree" + ) +``` + +**Step 2: Run all validator tests** + +Run: `cd backend && python -m pytest tests/test_ai_tree_validator.py -v` +Expected: ALL PASS + +**Step 3: Commit** + +```bash +git add backend/app/core/ai_tree_validator.py +git commit -m "fix: prevent double-counting action refs in global option cross-reference check" +``` + +--- + +## Task 8: Full Integration Test + +Run the full backend test suite and frontend build to verify nothing is broken. + +**Files:** (none — testing only) + +**Step 1: Run backend tests** + +Run: `cd backend && python -m pytest --override-ini="addopts=" -v` +Expected: ALL PASS + +**Step 2: Run frontend build** + +Run: `cd frontend && npm run build` +Expected: Build succeeds with no errors. + +**Step 3: Manual smoke test** + +1. Start backend: `cd backend && uvicorn app.main:app --reload` +2. Start frontend: `cd frontend && npm run dev` +3. Open tree editor with an existing tree +4. Edit an action node → verify "Next Step" dropdown appears with all nodes listed +5. Select a node from a different branch → verify dashed purple arrow appears on canvas +6. Edit a decision node → expand "Link an option to an existing node" → create a cross-reference +7. Verify circular reference warning (not error) appears in validation panel +8. Navigate the tree → verify loop-back works (session follows `next_node_id`) + +--- + +## Summary + +| Task | What | Files | +|------|------|-------| +| 1 | Backend: relax option validation | `ai_tree_validator.py`, `test_ai_tree_validator.py` | +| 2 | Frontend: circular ref → warning | `treeEditorStore.ts` | +| 3 | Canvas: dashed purple cross-ref edges | `useTreeLayout.ts` | +| 4 | Editor: action node picker | `NodeFormAction.tsx`, `treeEditorStore.ts` | +| 5 | Editor: decision option picker | `NodeFormDecision.tsx` | +| 6 | AI prompt: loop-back awareness | `ai_chat_service.py` | +| 7 | Backend: fix ref overlap check | `ai_tree_validator.py` | +| 8 | Integration test | (testing only) |