fix: strengthen prompt to prevent next_node_id mismatches, keep strict validation

Rather than lowering the validation bar, improve the system prompt:
- Rule 6 now explicitly states next_node_id must match a direct child's id
- Added rule 10: build tree bottom-up to avoid forward-reference errors
- Corrective prompt now calls out the ID mismatch constraint specifically

Reverts the strict=False fallback — flows must be correct before saving.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
chihlasm
2026-02-21 13:48:50 -05:00
parent 21f542694e
commit c562a82f5d
2 changed files with 7 additions and 10 deletions

View File

@@ -77,10 +77,11 @@ Rules:
3. Every branch path MUST end in a solution node — no dead ends
4. Include realistic MSP commands (PowerShell preferred for Windows)
5. Use unique node IDs prefixed with the branch context (e.g., "dns-check-service")
6. Every option's next_node_id must match an existing child node's id
6. CRITICAL — next_node_id must exactly match the "id" of a direct child in the "children" array of that same node. Never reference an ID that does not appear as a child of the current node.
7. All option labels must be meaningful and specific
8. Decision nodes must have at least 2 options
9. Return a single root node with its children nested inside
10. Build the tree bottom-up in your head: create leaf nodes first, then reference their IDs in parent options
Few-shot example (abbreviated):
{"id": "dns-root", "type": "decision", "question": "Can the client resolve any DNS names?", "help_text": "Run: nslookup google.com", "options": [{"id": "dns-opt-none", "label": "No DNS resolution at all", "next_node_id": "dns-check-service"}, {"id": "dns-opt-partial", "label": "Some names resolve, others don't", "next_node_id": "dns-check-specific"}], "children": [{"id": "dns-check-service", "type": "action", "title": "Check DNS Service", "description": "Verify the DNS Client service is running", "commands": ["Get-Service -Name Dnscache"], "expected_outcome": "Service should be Running", "children": [{"id": "dns-resolved", "type": "solution", "title": "DNS Service Restored", "description": "DNS client service was stopped", "resolution_steps": ["Restart DNS Client service", "Flush DNS cache: ipconfig /flushdns", "Test resolution"]}]}, {"id": "dns-check-specific", "type": "solution", "title": "Selective DNS Failure", "description": "Specific records missing or stale", "resolution_steps": ["Check DNS server configuration", "Verify zone records", "Clear DNS cache"]}]}"""
@@ -91,6 +92,8 @@ CORRECTIVE_PROMPT_TEMPLATE = """Your previous JSON was invalid for ResolutionFlo
Validation errors:
{error_list}
IMPORTANT: If any error mentions a next_node_id referencing a non-existent child, you must ensure every option's next_node_id exactly matches the "id" field of one of the node's direct children. The child node must exist in the "children" array of the same parent node.
Return a corrected full JSON object only. No markdown, no prose, no code fences.
Fix ALL listed errors while maintaining the same troubleshooting/procedural logic."""
@@ -234,14 +237,8 @@ async def generate_branch_detail(
continue
raise ValueError(f"AI returned invalid JSON after retry: {e}")
# On the final attempt, use strict=False to accept cross-reference
# mismatches (next_node_id pointing to wrong child) — these are
# minor structural issues the user can fix in the editor.
is_final_attempt = attempt == 2
errors = validate_generated_tree(branch_tree, strict=not is_final_attempt)
errors = validate_generated_tree(branch_tree)
if not errors:
if is_final_attempt:
logger.warning("branch_detail accepted on final attempt (lenient validation): branch=%s", branch_name)
cost = _estimate_cost(total_input, total_output)
return branch_tree, total_input, total_output, cost

View File

@@ -24,7 +24,7 @@ class TreeValidationError(Exception):
super().__init__(f"Tree validation failed: {'; '.join(errors)}")
def validate_generated_tree(tree: dict[str, Any], strict: bool = True) -> list[str]:
def validate_generated_tree(tree: dict[str, Any]) -> list[str]:
"""Validate an AI-generated tree structure.
Returns a list of error strings. Empty list means valid.
@@ -106,7 +106,7 @@ def validate_generated_tree(tree: dict[str, Any], strict: bool = True) -> list[s
next_id = opt.get("next_node_id")
if next_id:
all_referenced_ids.add(next_id)
if strict and child_ids and next_id not in child_ids:
if child_ids and next_id not in child_ids:
errors.append(
f"Option '{opt.get('label', '?')}' in node '{node_id}' "
f"references non-existent child '{next_id}'"