fix: strengthen prompt to prevent next_node_id mismatches, keep strict validation
Rather than lowering the validation bar, improve the system prompt: - Rule 6 now explicitly states next_node_id must match a direct child's id - Added rule 10: build tree bottom-up to avoid forward-reference errors - Corrective prompt now calls out the ID mismatch constraint specifically Reverts the strict=False fallback — flows must be correct before saving. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -77,10 +77,11 @@ Rules:
|
|||||||
3. Every branch path MUST end in a solution node — no dead ends
|
3. Every branch path MUST end in a solution node — no dead ends
|
||||||
4. Include realistic MSP commands (PowerShell preferred for Windows)
|
4. Include realistic MSP commands (PowerShell preferred for Windows)
|
||||||
5. Use unique node IDs prefixed with the branch context (e.g., "dns-check-service")
|
5. Use unique node IDs prefixed with the branch context (e.g., "dns-check-service")
|
||||||
6. Every option's next_node_id must match an existing child node's id
|
6. CRITICAL — next_node_id must exactly match the "id" of a direct child in the "children" array of that same node. Never reference an ID that does not appear as a child of the current node.
|
||||||
7. All option labels must be meaningful and specific
|
7. All option labels must be meaningful and specific
|
||||||
8. Decision nodes must have at least 2 options
|
8. Decision nodes must have at least 2 options
|
||||||
9. Return a single root node with its children nested inside
|
9. Return a single root node with its children nested inside
|
||||||
|
10. Build the tree bottom-up in your head: create leaf nodes first, then reference their IDs in parent options
|
||||||
|
|
||||||
Few-shot example (abbreviated):
|
Few-shot example (abbreviated):
|
||||||
{"id": "dns-root", "type": "decision", "question": "Can the client resolve any DNS names?", "help_text": "Run: nslookup google.com", "options": [{"id": "dns-opt-none", "label": "No DNS resolution at all", "next_node_id": "dns-check-service"}, {"id": "dns-opt-partial", "label": "Some names resolve, others don't", "next_node_id": "dns-check-specific"}], "children": [{"id": "dns-check-service", "type": "action", "title": "Check DNS Service", "description": "Verify the DNS Client service is running", "commands": ["Get-Service -Name Dnscache"], "expected_outcome": "Service should be Running", "children": [{"id": "dns-resolved", "type": "solution", "title": "DNS Service Restored", "description": "DNS client service was stopped", "resolution_steps": ["Restart DNS Client service", "Flush DNS cache: ipconfig /flushdns", "Test resolution"]}]}, {"id": "dns-check-specific", "type": "solution", "title": "Selective DNS Failure", "description": "Specific records missing or stale", "resolution_steps": ["Check DNS server configuration", "Verify zone records", "Clear DNS cache"]}]}"""
|
{"id": "dns-root", "type": "decision", "question": "Can the client resolve any DNS names?", "help_text": "Run: nslookup google.com", "options": [{"id": "dns-opt-none", "label": "No DNS resolution at all", "next_node_id": "dns-check-service"}, {"id": "dns-opt-partial", "label": "Some names resolve, others don't", "next_node_id": "dns-check-specific"}], "children": [{"id": "dns-check-service", "type": "action", "title": "Check DNS Service", "description": "Verify the DNS Client service is running", "commands": ["Get-Service -Name Dnscache"], "expected_outcome": "Service should be Running", "children": [{"id": "dns-resolved", "type": "solution", "title": "DNS Service Restored", "description": "DNS client service was stopped", "resolution_steps": ["Restart DNS Client service", "Flush DNS cache: ipconfig /flushdns", "Test resolution"]}]}, {"id": "dns-check-specific", "type": "solution", "title": "Selective DNS Failure", "description": "Specific records missing or stale", "resolution_steps": ["Check DNS server configuration", "Verify zone records", "Clear DNS cache"]}]}"""
|
||||||
@@ -91,6 +92,8 @@ CORRECTIVE_PROMPT_TEMPLATE = """Your previous JSON was invalid for ResolutionFlo
|
|||||||
Validation errors:
|
Validation errors:
|
||||||
{error_list}
|
{error_list}
|
||||||
|
|
||||||
|
IMPORTANT: If any error mentions a next_node_id referencing a non-existent child, you must ensure every option's next_node_id exactly matches the "id" field of one of the node's direct children. The child node must exist in the "children" array of the same parent node.
|
||||||
|
|
||||||
Return a corrected full JSON object only. No markdown, no prose, no code fences.
|
Return a corrected full JSON object only. No markdown, no prose, no code fences.
|
||||||
Fix ALL listed errors while maintaining the same troubleshooting/procedural logic."""
|
Fix ALL listed errors while maintaining the same troubleshooting/procedural logic."""
|
||||||
|
|
||||||
@@ -234,14 +237,8 @@ async def generate_branch_detail(
|
|||||||
continue
|
continue
|
||||||
raise ValueError(f"AI returned invalid JSON after retry: {e}")
|
raise ValueError(f"AI returned invalid JSON after retry: {e}")
|
||||||
|
|
||||||
# On the final attempt, use strict=False to accept cross-reference
|
errors = validate_generated_tree(branch_tree)
|
||||||
# mismatches (next_node_id pointing to wrong child) — these are
|
|
||||||
# minor structural issues the user can fix in the editor.
|
|
||||||
is_final_attempt = attempt == 2
|
|
||||||
errors = validate_generated_tree(branch_tree, strict=not is_final_attempt)
|
|
||||||
if not errors:
|
if not errors:
|
||||||
if is_final_attempt:
|
|
||||||
logger.warning("branch_detail accepted on final attempt (lenient validation): branch=%s", branch_name)
|
|
||||||
cost = _estimate_cost(total_input, total_output)
|
cost = _estimate_cost(total_input, total_output)
|
||||||
return branch_tree, total_input, total_output, cost
|
return branch_tree, total_input, total_output, cost
|
||||||
|
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ class TreeValidationError(Exception):
|
|||||||
super().__init__(f"Tree validation failed: {'; '.join(errors)}")
|
super().__init__(f"Tree validation failed: {'; '.join(errors)}")
|
||||||
|
|
||||||
|
|
||||||
def validate_generated_tree(tree: dict[str, Any], strict: bool = True) -> list[str]:
|
def validate_generated_tree(tree: dict[str, Any]) -> list[str]:
|
||||||
"""Validate an AI-generated tree structure.
|
"""Validate an AI-generated tree structure.
|
||||||
|
|
||||||
Returns a list of error strings. Empty list means valid.
|
Returns a list of error strings. Empty list means valid.
|
||||||
@@ -106,7 +106,7 @@ def validate_generated_tree(tree: dict[str, Any], strict: bool = True) -> list[s
|
|||||||
next_id = opt.get("next_node_id")
|
next_id = opt.get("next_node_id")
|
||||||
if next_id:
|
if next_id:
|
||||||
all_referenced_ids.add(next_id)
|
all_referenced_ids.add(next_id)
|
||||||
if strict and child_ids and next_id not in child_ids:
|
if child_ids and next_id not in child_ids:
|
||||||
errors.append(
|
errors.append(
|
||||||
f"Option '{opt.get('label', '?')}' in node '{node_id}' "
|
f"Option '{opt.get('label', '?')}' in node '{node_id}' "
|
||||||
f"references non-existent child '{next_id}'"
|
f"references non-existent child '{next_id}'"
|
||||||
|
|||||||
Reference in New Issue
Block a user