# AI-Guided Flow Creation — Design Document > **Date:** 2026-02-12 > **Status:** Draft > **Phase:** 3 (AI Intelligence) > **Dependencies:** Global Categories, Flow Editor, Plan Limits, Session Tracking > **Estimated Effort:** 3-4 weeks (backend + frontend + prompt engineering) --- ## Overview AI-Guided Flow Creation is an interactive wizard that helps engineers build troubleshooting and procedural flows through a structured conversation. Rather than generating a complete flow from a single description (expensive, low-quality, black box), the wizard asks targeted questions at each stage, using the engineer's domain expertise to shape the output while AI fills in the details. **Core Principle:** The engineer knows the problem domain. The AI knows how to structure it. The wizard brings both together. **Key Differentiator:** This is NOT a "describe your problem and AI builds everything" feature. It's a collaborative creation tool where AI assists at specific, bounded points — keeping costs predictable, output quality high, and the engineer in control. --- ## Why This Approach | Approach | Cost/Flow | Quality | Engineer Control | Reusability | |---|---|---|---|---| | Full AI generation (one-shot) | ~$0.03 | Medium (needs heavy editing) | None until review | None | | **Guided wizard (this design)** | **~$0.01-0.02** | **High (shaped by engineer)** | **At every step** | **High (cached prompts)** | | Manual creation only | $0.00 | Varies | Full | None | The guided approach wins on every axis because: 1. **Smaller, targeted API calls** replace one large generation call 2. **Fixed question patterns** enable aggressive prompt caching (90% input savings) 3. **Engineer input at each stage** means less post-generation editing 4. **Structured data collection** before any AI call means better prompts and better output --- ## User Flow ### Stage 1: Foundation (No AI — Pure UI) The user provides structured metadata that will inform all subsequent AI calls. ``` ┌─────────────────────────────────────────────────────────────┐ │ CREATE A NEW FLOW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ Flow Type: │ │ ○ Troubleshooting (diagnostic branching — "what's wrong?")│ │ ○ Procedure (step-by-step — "how do I set this up?") │ │ │ │ Category: │ │ [▼ Networking ] │ │ │ │ Flow Name: │ │ [Printer Not Printing ] │ │ │ │ Brief Description: │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ User reports print jobs stuck in queue, printer │ │ │ │ shows as offline or errors when printing │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ Target Environment (optional): │ │ □ Windows □ macOS □ Linux │ │ □ Cloud/SaaS □ On-Premises □ Hybrid │ │ │ │ [Next →] │ └─────────────────────────────────────────────────────────────┘ ``` **Data collected:** flow_type, category_id, name, description, environment tags. **AI cost:** $0.00 --- ### Stage 2: Structure Scaffolding (Light AI) Based on Stage 1 input, AI suggests the top-level structure. The user confirms, removes, or adds items. **For Troubleshooting flows — AI suggests initial symptom branches:** ``` ┌─────────────────────────────────────────────────────────────┐ │ WHAT SYMPTOMS WOULD A USER REPORT? │ │ │ │ Based on "Printer Not Printing", here are common starting │ │ points. Check the ones that apply, add your own: │ │ │ │ AI Suggestions: │ │ ☑ Print jobs stuck in queue / never print │ │ ☑ Printer shows offline │ │ ☑ Prints but output is garbled or wrong │ │ ☐ Printer not found / can't add printer │ │ ☑ Specific application can't print (others can) │ │ │ │ + Add your own symptom: [____________________________] │ │ │ │ [← Back] [Next →] │ └─────────────────────────────────────────────────────────────┘ ``` **For Procedure flows — AI suggests major phases:** ``` ┌─────────────────────────────────────────────────────────────┐ │ WHAT ARE THE MAJOR PHASES? │ │ │ │ Based on "New Server Build", here are typical phases. │ │ Check the ones that apply, reorder, add your own: │ │ │ │ AI Suggestions: │ │ ☑ 1. Pre-requisites & Planning │ │ ☑ 2. OS Installation & Base Config │ │ ☑ 3. Network Configuration │ │ ☑ 4. Domain Join & Security │ │ ☑ 5. Role/Application Installation │ │ ☑ 6. Verification & Handoff │ │ │ │ + Add a phase: [____________________________] │ │ │ │ [← Back] [Next →] │ └─────────────────────────────────────────────────────────────┘ ``` **API call:** 1 call to Haiku 4.5. Input: system prompt (cached) + stage 1 metadata. Output: 5-8 suggestions as JSON array. **Estimated cost:** ~$0.003-0.005 --- ### Stage 3: Branch/Step Detail (Light AI, per branch) For each item the user selected in Stage 2, AI suggests the diagnostic steps or sub-steps. The user processes one branch/phase at a time. **For Troubleshooting — diagnostic steps per symptom:** ``` ┌─────────────────────────────────────────────────────────────┐ │ BRANCH: "Print jobs stuck in queue" │ │ │ │ What steps should an engineer follow for this symptom? │ │ │ │ AI Suggestions: │ │ ☑ Check Print Spooler service status │ │ Action: Run Get-Service Spooler — restart if stopped │ │ │ │ ☑ Clear the print queue │ │ Action: Stop spooler, delete files in │ │ C:\Windows\System32\spool\PRINTERS, restart spooler │ │ │ │ ☑ Verify printer port and driver │ │ Decision: Is the printer networked or USB? │ │ → Networked: ping printer IP, check port config │ │ → USB: check cable, try different port │ │ │ │ ☑ Test with different application │ │ Decision: Does it print from Notepad? │ │ → Yes: Application-specific issue │ │ → No: System-level print problem │ │ │ │ + Add your own step: [____________________________] │ │ │ │ [✎ Edit any step] [← Back] [Next →] │ └─────────────────────────────────────────────────────────────┘ ``` **For Procedure — detailed steps per phase:** ``` ┌─────────────────────────────────────────────────────────────┐ │ PHASE: "OS Installation & Base Config" │ │ │ │ What steps should be performed in this phase? │ │ │ │ AI Suggestions: │ │ ☑ Boot from installation media │ │ Action: Mount ISO / insert USB, boot to BIOS, set │ │ boot order │ │ │ │ ☑ Configure disk partitions │ │ Action: Create partitions per standard │ │ 📝 Record: "Partition layout used" │ │ │ │ ☑ Set hostname │ │ Action: Follow naming convention: SITE-ROLE-## │ │ 📝 Record: "Hostname assigned" │ │ │ │ ☑ Configure local admin account │ │ Action: Set password per policy, disable built-in │ │ Administrator │ │ │ │ ☑ Install Windows Updates │ │ Action: Run full update cycle, reboot, repeat until │ │ clean │ │ │ │ + Add your own step: [____________________________] │ │ │ │ [← Back] [Next →] │ └─────────────────────────────────────────────────────────────┘ ``` **Note the 📝 Record fields** — for procedure flows, AI can suggest which steps should capture data (hostname, IP, etc.). This feeds directly into the session documentation. **API call:** 1 call per branch/phase. Input: system prompt (cached) + stage 1 metadata + branch name + context from other branches. Output: 3-6 detailed steps as JSON. **Estimated cost:** ~$0.003-0.005 per branch. 5 branches = ~$0.015-0.025 --- ### Stage 4: Review & Refine (No AI) The complete flow is assembled and shown in the existing tree/flow editor. User can: - Rearrange nodes - Edit text on any step - Add/remove branches - Preview the flow as an end-user would see it - Save as draft or publish ``` ┌─────────────────────────────────────────────────────────────┐ │ REVIEW YOUR FLOW │ │ │ │ "Printer Not Printing" — Troubleshooting Flow │ │ Category: Networking | 5 branches | 23 steps │ │ │ │ [Visual tree/flow editor — existing component] │ │ │ │ ┌─ Root: What is the printing issue? │ │ ├── Jobs stuck in queue (5 steps) │ │ ├── Printer shows offline (4 steps) │ │ ├── Garbled output (3 steps) │ │ ├── App-specific failure (4 steps) │ │ └── Can't add printer (4 steps) │ │ │ │ [Save as Draft] [Publish] │ └─────────────────────────────────────────────────────────────┘ ``` **AI cost:** $0.00 --- ## Total Cost Per Flow | Stage | AI Calls | Estimated Cost | |---|---|---| | 1. Foundation | 0 | $0.000 | | 2. Structure Scaffolding | 1 | $0.003-0.005 | | 3. Branch/Step Detail | 1 per branch (avg 5) | $0.015-0.025 | | 4. Review & Refine | 0 | $0.000 | | **Total** | **~6 calls** | **$0.018-0.030** | **With prompt caching active** (system prompt cached across all users): - First user of the day: ~$0.025 (full system prompt cost) - Subsequent users: ~$0.012-0.018 (90% savings on system prompt input) --- ## Cost Projections By Plan ### Per-User Monthly Cost to ResolutionFlow | Plan | AI Flows/Month | Est. Cost/Month | Subscription Price | Margin | |---|---|---|---|---| | Free | 2 | $0.04-0.06 | $0 | -$0.06 (acquisition cost) | | Pro | 30 (1/day) | $0.54-0.90 | $29 | 97%+ | | Pro (heavy) | 300 (10/day) | $5.40-9.00 | $29 | 69-81% | | Team (per user) | 300 (10/day) | $5.40-9.00 | ~$20/seat | 55-73% | ### At Scale (Monthly Platform Cost) | Scenario | Users | Flows/Month | Monthly AI Cost | |---|---|---|---| | Beta (15 users) | 15 | ~150 | $2.70-4.50 | | Early growth | 100 | ~1,500 | $27-45 | | Scaling | 500 | ~7,500 | $135-225 | | Target ($10K MRR) | ~400 | ~6,000 | $108-180 | **Bottom line:** Even at 10/day per user, AI costs stay under 30% of subscription revenue. At typical usage (1-3/day), AI costs are negligible — under 5% of revenue. --- ## Plan Limits Integration ### New Fields for plan_limits Table ```sql ALTER TABLE plan_limits ADD COLUMN ai_flows_per_month INTEGER DEFAULT NULL, ADD COLUMN ai_calls_per_flow INTEGER DEFAULT NULL; ``` | Plan | ai_flows_per_month | ai_calls_per_flow | |---|---|---| | Free | 2 | 6 | | Pro | 50 | 10 | | Team | 200 (per account) | 10 | `ai_calls_per_flow` caps how many branches can get AI suggestions per flow. This prevents a user from creating a 50-branch monstrosity that costs $0.25 per generation. ### Tracking Table ```sql CREATE TABLE ai_usage_log ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), account_id UUID NOT NULL REFERENCES accounts(id), user_id UUID NOT NULL REFERENCES users(id), usage_type VARCHAR(50) NOT NULL, -- 'flow_scaffold', 'branch_detail', 'branch_suggest' model VARCHAR(100) NOT NULL, -- 'claude-haiku-4-5' input_tokens INTEGER NOT NULL, output_tokens INTEGER NOT NULL, estimated_cost_usd NUMERIC(10, 6) NOT NULL, flow_id UUID REFERENCES trees(id), metadata JSONB DEFAULT '{}', -- stage, branch name, etc. created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE INDEX idx_ai_usage_account_month ON ai_usage_log (account_id, created_at); ``` This gives you: - Per-account usage tracking for limit enforcement - Cost visibility in admin dashboard - Data to optimize prompts over time (which calls use the most tokens?) --- ## Backend Implementation ### New Files ``` app/ services/ ai_flow_generator.py # Core AI service — prompt construction, API calls, parsing api/ endpoints/ ai_generation.py # API endpoints for wizard stages core/ ai_config.py # Model selection, pricing constants, feature flag checks ``` ### API Endpoints ``` POST /api/v1/ai/flow/scaffold Body: { flow_type, category_id, name, description, environment_tags } Returns: { suggestions: ["symptom1", "symptom2", ...] } Auth: Required, plan limit checked AI: 1 Haiku call POST /api/v1/ai/flow/branch-detail Body: { flow_type, category_id, name, description, branch_name, existing_branches } Returns: { steps: [ { type, title, action/question, help_text, sub_branches?, record_fields? } ] } Auth: Required, plan limit checked, per-flow call limit checked AI: 1 Haiku call POST /api/v1/ai/flow/assemble Body: { flow_type, category_id, name, description, branches: [ { name, steps: [...] } ] } Returns: { tree_structure: { ... } } // Valid tree structure ready for create_tree Auth: Required AI: 0 calls — pure assembly logic, runs through normalize_node + validation ``` ### Prompt Architecture The system prompt is the same across all users and all calls within a stage. This is critical for prompt caching. **System Prompt (Stage 2 — Troubleshooting):** ``` You are a senior MSP engineer helping build troubleshooting decision trees. Given a problem description and category, suggest 4-7 initial symptom branches that a support engineer would encounter. Rules: - Each suggestion should be a distinct, common symptom (not overlapping) - Order from most common to least common - Use plain language that a Tier 1 engineer would understand - Focus on observable symptoms, not root causes - Respond in JSON format only: { "suggestions": ["symptom1", "symptom2"] } ``` **System Prompt (Stage 2 — Procedure):** ``` You are a senior MSP engineer helping build procedural checklists. Given a project description and category, suggest 4-8 major phases that the project should follow. Rules: - Phases should be in logical execution order - Each phase should be a distinct stage of work - Use standard MSP/IT terminology - Respond in JSON format only: { "suggestions": ["phase1", "phase2"] } ``` **System Prompt (Stage 3 — Branch Detail):** ``` You are a senior MSP engineer helping build troubleshooting steps. Given a symptom/branch name and context, suggest 3-6 diagnostic steps. For each step, provide: - type: "action" (do something), "decision" (yes/no check), or "solution" (resolution) - title: short step name - content: detailed instructions including commands where relevant - sub_branches: (for decisions only) what the yes/no paths look like - record_fields: (for procedures only) data the engineer should document Rules: - Include specific commands (PowerShell preferred for Windows environments) - Action steps should be concrete and actionable - Decision steps should have clear yes/no outcomes - End branches with solution nodes where possible - Respond in JSON format only ``` **Caching strategy:** System prompts are sent as cacheable prefixes. With a 5-minute TTL, any user hitting the wizard within 5 minutes of another user gets 90% savings on input tokens. During active hours, this cache will almost always be warm. --- ## Frontend Implementation ### New Components ``` frontend/src/ pages/ FlowWizardPage.tsx # Main wizard container with stage navigation components/ wizard/ WizardStageFoundation.tsx # Stage 1: type, category, name, description WizardStageScaffold.tsx # Stage 2: AI suggestions with checkboxes WizardStageBranchDetail.tsx # Stage 3: per-branch step suggestions WizardStageReview.tsx # Stage 4: assembled flow preview WizardProgress.tsx # Stage indicator bar SuggestionCheckbox.tsx # Reusable AI suggestion with check/uncheck CustomItemInput.tsx # "Add your own" input field api/ aiGeneration.ts # API client for AI endpoints hooks/ useAiWizard.ts # State management for wizard flow ``` ### Wizard State Management ```typescript interface WizardState { // Stage 1 flowType: 'troubleshooting' | 'procedure'; categoryId: string; name: string; description: string; environmentTags: string[]; // Stage 2 scaffoldSuggestions: string[]; // AI-generated selectedBranches: string[]; // User-confirmed customBranches: string[]; // User-added // Stage 3 branchDetails: Record; // Per-branch steps currentBranchIndex: number; // Stage 4 assembledStructure: TreeStructure | null; // Meta currentStage: 1 | 2 | 3 | 4; aiCallsUsed: number; isGenerating: boolean; } ``` ### UX Considerations 1. **Loading states:** AI calls take 2-5 seconds. Show a subtle loading animation with context ("Analyzing common symptoms..."), not a spinner. 2. **Graceful degradation:** If an AI call fails, show an empty state with "Add your own" prominent. The wizard should never be blocked by AI failure. 3. **Edit-in-place:** Users should be able to click any AI suggestion to edit the text before accepting it. 4. **Skip AI:** Every stage should have a "Skip suggestions, I'll add my own" option. Power users may want to build manually but still benefit from the wizard structure. 5. **Mobile consideration:** The wizard should work on tablet at minimum. MSP engineers may use it on-site with an iPad. --- ## Environment & Configuration ### Environment Variables ``` # .env / Railway ANTHROPIC_API_KEY=sk-ant-... AI_GENERATION_MODEL=claude-haiku-4-5-20251001 AI_GENERATION_ENABLED=true # Kill switch AI_MAX_RETRIES=2 AI_REQUEST_TIMEOUT=30 # seconds ``` ### Feature Flag Integration Use existing feature_flags system: ```json { "ai_flow_generation": { "enabled": true, "plans": ["pro", "team"], "beta_override": true // Allow specific free users during beta } } ``` --- ## Rollout Strategy ### Phase 3a: Internal Testing (Week 1) - Backend endpoints + prompt engineering - Test with admin account only - Iterate on prompt quality — this is where most time goes - Validate cost estimates against real API usage ### Phase 3b: Beta Testing (Week 2-3) - Frontend wizard UI - Enable for beta testers (feature flag) - Collect feedback on suggestion quality - Monitor ai_usage_log for actual costs ### Phase 3c: General Availability (Week 4) - Enable for all Pro/Team plans - Free tier gets limited access (2/month) - Admin dashboard shows AI cost metrics - Plan limit enforcement active --- ## Future Enhancements (Not in Initial Scope) These are explicitly deferred to keep initial scope manageable: 1. **AI-suggested branch improvements** — "Based on 15 unresolved sessions on this branch, AI suggests adding these steps." Requires session outcome tracking (separate feature). 2. **AI step refinement** — User selects an existing step and says "make this more detailed" or "add the PowerShell commands." Single-step AI call, very cheap. 3. **Template flows from wizard patterns** — If many users create similar flows (e.g., "printer troubleshooting"), cache the assembled structure as a template. Future users get instant results with zero AI cost. 4. **Multi-language generation** — Same wizard, but AI generates steps in the user's language. Prompt change only, no architecture change. 5. **AI from session data** — After enough sessions are tracked, use anonymized session paths to improve AI suggestions. "80% of engineers check the spooler service first" → AI learns to suggest that first. --- ## Key Risks & Mitigations | Risk | Impact | Mitigation | |---|---|---| | Anthropic API outage | Wizard Stage 2-3 blocked | Graceful fallback to manual entry; AI is enhancement, not requirement | | Poor suggestion quality | Users lose trust in feature | Extensive prompt testing before launch; user can always edit/override | | Cost overrun from heavy usage | Margin erosion | Per-account monthly limits in plan_limits; ai_usage_log for monitoring | | Prompt injection via user input | Unexpected AI output | Sanitize user input; structured JSON output parsing; never execute AI output as code | | Haiku model deprecated | Service interruption | Model name in env variable, not hardcoded; swap to successor model | | Users treat AI as authoritative | Bad troubleshooting steps followed blindly | "Draft" status by default; review stage required; clear "AI-suggested" labeling | --- ## Success Metrics | Metric | Target | How to Measure | |---|---|---| | Wizard completion rate | >70% of starts finish Stage 4 | Track stage progression in analytics | | AI suggestion acceptance rate | >50% of suggestions kept | Compare suggestions shown vs. selected | | Time to create flow (wizard vs manual) | 60%+ faster with wizard | Compare creation timestamps | | Post-wizard editing rate | <30% of nodes edited after assembly | Compare Stage 4 structure vs. published version | | Cost per flow | <$0.03 average | ai_usage_log aggregation | | User satisfaction | Positive feedback from beta | Direct feedback + usage retention |