26 KiB
AI-Guided Flow Creation — Design Document
Date: 2026-02-12 Status: Draft Phase: 3 (AI Intelligence) Dependencies: Global Categories, Flow Editor, Plan Limits, Session Tracking Estimated Effort: 3-4 weeks (backend + frontend + prompt engineering)
Overview
AI-Guided Flow Creation is an interactive wizard that helps engineers build troubleshooting and procedural flows through a structured conversation. Rather than generating a complete flow from a single description (expensive, low-quality, black box), the wizard asks targeted questions at each stage, using the engineer's domain expertise to shape the output while AI fills in the details.
Core Principle: The engineer knows the problem domain. The AI knows how to structure it. The wizard brings both together.
Key Differentiator: This is NOT a "describe your problem and AI builds everything" feature. It's a collaborative creation tool where AI assists at specific, bounded points — keeping costs predictable, output quality high, and the engineer in control.
Why This Approach
| Approach | Cost/Flow | Quality | Engineer Control | Reusability |
|---|---|---|---|---|
| Full AI generation (one-shot) | ~$0.03 | Medium (needs heavy editing) | None until review | None |
| Guided wizard (this design) | ~$0.01-0.02 | High (shaped by engineer) | At every step | High (cached prompts) |
| Manual creation only | $0.00 | Varies | Full | None |
The guided approach wins on every axis because:
- Smaller, targeted API calls replace one large generation call
- Fixed question patterns enable aggressive prompt caching (90% input savings)
- Engineer input at each stage means less post-generation editing
- Structured data collection before any AI call means better prompts and better output
User Flow
Stage 1: Foundation (No AI — Pure UI)
The user provides structured metadata that will inform all subsequent AI calls.
┌─────────────────────────────────────────────────────────────┐
│ CREATE A NEW FLOW │
├─────────────────────────────────────────────────────────────┤
│ │
│ Flow Type: │
│ ○ Troubleshooting (diagnostic branching — "what's wrong?")│
│ ○ Procedure (step-by-step — "how do I set this up?") │
│ │
│ Category: │
│ [▼ Networking ] │
│ │
│ Flow Name: │
│ [Printer Not Printing ] │
│ │
│ Brief Description: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ User reports print jobs stuck in queue, printer │ │
│ │ shows as offline or errors when printing │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Target Environment (optional): │
│ □ Windows □ macOS □ Linux │
│ □ Cloud/SaaS □ On-Premises □ Hybrid │
│ │
│ [Next →] │
└─────────────────────────────────────────────────────────────┘
Data collected: flow_type, category_id, name, description, environment tags. AI cost: $0.00
Stage 2: Structure Scaffolding (Light AI)
Based on Stage 1 input, AI suggests the top-level structure. The user confirms, removes, or adds items.
For Troubleshooting flows — AI suggests initial symptom branches:
┌─────────────────────────────────────────────────────────────┐
│ WHAT SYMPTOMS WOULD A USER REPORT? │
│ │
│ Based on "Printer Not Printing", here are common starting │
│ points. Check the ones that apply, add your own: │
│ │
│ AI Suggestions: │
│ ☑ Print jobs stuck in queue / never print │
│ ☑ Printer shows offline │
│ ☑ Prints but output is garbled or wrong │
│ ☐ Printer not found / can't add printer │
│ ☑ Specific application can't print (others can) │
│ │
│ + Add your own symptom: [____________________________] │
│ │
│ [← Back] [Next →] │
└─────────────────────────────────────────────────────────────┘
For Procedure flows — AI suggests major phases:
┌─────────────────────────────────────────────────────────────┐
│ WHAT ARE THE MAJOR PHASES? │
│ │
│ Based on "New Server Build", here are typical phases. │
│ Check the ones that apply, reorder, add your own: │
│ │
│ AI Suggestions: │
│ ☑ 1. Pre-requisites & Planning │
│ ☑ 2. OS Installation & Base Config │
│ ☑ 3. Network Configuration │
│ ☑ 4. Domain Join & Security │
│ ☑ 5. Role/Application Installation │
│ ☑ 6. Verification & Handoff │
│ │
│ + Add a phase: [____________________________] │
│ │
│ [← Back] [Next →] │
└─────────────────────────────────────────────────────────────┘
API call: 1 call to Haiku 4.5. Input: system prompt (cached) + stage 1 metadata. Output: 5-8 suggestions as JSON array. Estimated cost: ~$0.003-0.005
Stage 3: Branch/Step Detail (Light AI, per branch)
For each item the user selected in Stage 2, AI suggests the diagnostic steps or sub-steps. The user processes one branch/phase at a time.
For Troubleshooting — diagnostic steps per symptom:
┌─────────────────────────────────────────────────────────────┐
│ BRANCH: "Print jobs stuck in queue" │
│ │
│ What steps should an engineer follow for this symptom? │
│ │
│ AI Suggestions: │
│ ☑ Check Print Spooler service status │
│ Action: Run Get-Service Spooler — restart if stopped │
│ │
│ ☑ Clear the print queue │
│ Action: Stop spooler, delete files in │
│ C:\Windows\System32\spool\PRINTERS, restart spooler │
│ │
│ ☑ Verify printer port and driver │
│ Decision: Is the printer networked or USB? │
│ → Networked: ping printer IP, check port config │
│ → USB: check cable, try different port │
│ │
│ ☑ Test with different application │
│ Decision: Does it print from Notepad? │
│ → Yes: Application-specific issue │
│ → No: System-level print problem │
│ │
│ + Add your own step: [____________________________] │
│ │
│ [✎ Edit any step] [← Back] [Next →] │
└─────────────────────────────────────────────────────────────┘
For Procedure — detailed steps per phase:
┌─────────────────────────────────────────────────────────────┐
│ PHASE: "OS Installation & Base Config" │
│ │
│ What steps should be performed in this phase? │
│ │
│ AI Suggestions: │
│ ☑ Boot from installation media │
│ Action: Mount ISO / insert USB, boot to BIOS, set │
│ boot order │
│ │
│ ☑ Configure disk partitions │
│ Action: Create partitions per standard │
│ 📝 Record: "Partition layout used" │
│ │
│ ☑ Set hostname │
│ Action: Follow naming convention: SITE-ROLE-## │
│ 📝 Record: "Hostname assigned" │
│ │
│ ☑ Configure local admin account │
│ Action: Set password per policy, disable built-in │
│ Administrator │
│ │
│ ☑ Install Windows Updates │
│ Action: Run full update cycle, reboot, repeat until │
│ clean │
│ │
│ + Add your own step: [____________________________] │
│ │
│ [← Back] [Next →] │
└─────────────────────────────────────────────────────────────┘
Note the 📝 Record fields — for procedure flows, AI can suggest which steps should capture data (hostname, IP, etc.). This feeds directly into the session documentation.
API call: 1 call per branch/phase. Input: system prompt (cached) + stage 1 metadata + branch name + context from other branches. Output: 3-6 detailed steps as JSON. Estimated cost: ~$0.003-0.005 per branch. 5 branches = ~$0.015-0.025
Stage 4: Review & Refine (No AI)
The complete flow is assembled and shown in the existing tree/flow editor. User can:
- Rearrange nodes
- Edit text on any step
- Add/remove branches
- Preview the flow as an end-user would see it
- Save as draft or publish
┌─────────────────────────────────────────────────────────────┐
│ REVIEW YOUR FLOW │
│ │
│ "Printer Not Printing" — Troubleshooting Flow │
│ Category: Networking | 5 branches | 23 steps │
│ │
│ [Visual tree/flow editor — existing component] │
│ │
│ ┌─ Root: What is the printing issue? │
│ ├── Jobs stuck in queue (5 steps) │
│ ├── Printer shows offline (4 steps) │
│ ├── Garbled output (3 steps) │
│ ├── App-specific failure (4 steps) │
│ └── Can't add printer (4 steps) │
│ │
│ [Save as Draft] [Publish] │
└─────────────────────────────────────────────────────────────┘
AI cost: $0.00
Total Cost Per Flow
| Stage | AI Calls | Estimated Cost |
|---|---|---|
| 1. Foundation | 0 | $0.000 |
| 2. Structure Scaffolding | 1 | $0.003-0.005 |
| 3. Branch/Step Detail | 1 per branch (avg 5) | $0.015-0.025 |
| 4. Review & Refine | 0 | $0.000 |
| Total | ~6 calls | $0.018-0.030 |
With prompt caching active (system prompt cached across all users):
- First user of the day: ~$0.025 (full system prompt cost)
- Subsequent users: ~$0.012-0.018 (90% savings on system prompt input)
Cost Projections By Plan
Per-User Monthly Cost to ResolutionFlow
| Plan | AI Flows/Month | Est. Cost/Month | Subscription Price | Margin |
|---|---|---|---|---|
| Free | 2 | $0.04-0.06 | $0 | -$0.06 (acquisition cost) |
| Pro | 30 (1/day) | $0.54-0.90 | $29 | 97%+ |
| Pro (heavy) | 300 (10/day) | $5.40-9.00 | $29 | 69-81% |
| Team (per user) | 300 (10/day) | $5.40-9.00 | ~$20/seat | 55-73% |
At Scale (Monthly Platform Cost)
| Scenario | Users | Flows/Month | Monthly AI Cost |
|---|---|---|---|
| Beta (15 users) | 15 | ~150 | $2.70-4.50 |
| Early growth | 100 | ~1,500 | $27-45 |
| Scaling | 500 | ~7,500 | $135-225 |
| Target ($10K MRR) | ~400 | ~6,000 | $108-180 |
Bottom line: Even at 10/day per user, AI costs stay under 30% of subscription revenue. At typical usage (1-3/day), AI costs are negligible — under 5% of revenue.
Plan Limits Integration
New Fields for plan_limits Table
ALTER TABLE plan_limits
ADD COLUMN ai_flows_per_month INTEGER DEFAULT NULL,
ADD COLUMN ai_calls_per_flow INTEGER DEFAULT NULL;
| Plan | ai_flows_per_month | ai_calls_per_flow |
|---|---|---|
| Free | 2 | 6 |
| Pro | 50 | 10 |
| Team | 200 (per account) | 10 |
ai_calls_per_flow caps how many branches can get AI suggestions per flow. This prevents a user from creating a 50-branch monstrosity that costs $0.25 per generation.
Tracking Table
CREATE TABLE ai_usage_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
account_id UUID NOT NULL REFERENCES accounts(id),
user_id UUID NOT NULL REFERENCES users(id),
usage_type VARCHAR(50) NOT NULL, -- 'flow_scaffold', 'branch_detail', 'branch_suggest'
model VARCHAR(100) NOT NULL, -- 'claude-haiku-4-5'
input_tokens INTEGER NOT NULL,
output_tokens INTEGER NOT NULL,
estimated_cost_usd NUMERIC(10, 6) NOT NULL,
flow_id UUID REFERENCES trees(id),
metadata JSONB DEFAULT '{}', -- stage, branch name, etc.
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_ai_usage_account_month
ON ai_usage_log (account_id, created_at);
This gives you:
- Per-account usage tracking for limit enforcement
- Cost visibility in admin dashboard
- Data to optimize prompts over time (which calls use the most tokens?)
Backend Implementation
New Files
app/
services/
ai_flow_generator.py # Core AI service — prompt construction, API calls, parsing
api/
endpoints/
ai_generation.py # API endpoints for wizard stages
core/
ai_config.py # Model selection, pricing constants, feature flag checks
API Endpoints
POST /api/v1/ai/flow/scaffold
Body: { flow_type, category_id, name, description, environment_tags }
Returns: { suggestions: ["symptom1", "symptom2", ...] }
Auth: Required, plan limit checked
AI: 1 Haiku call
POST /api/v1/ai/flow/branch-detail
Body: { flow_type, category_id, name, description, branch_name, existing_branches }
Returns: { steps: [ { type, title, action/question, help_text, sub_branches?, record_fields? } ] }
Auth: Required, plan limit checked, per-flow call limit checked
AI: 1 Haiku call
POST /api/v1/ai/flow/assemble
Body: { flow_type, category_id, name, description, branches: [ { name, steps: [...] } ] }
Returns: { tree_structure: { ... } } // Valid tree structure ready for create_tree
Auth: Required
AI: 0 calls — pure assembly logic, runs through normalize_node + validation
Prompt Architecture
The system prompt is the same across all users and all calls within a stage. This is critical for prompt caching.
System Prompt (Stage 2 — Troubleshooting):
You are a senior MSP engineer helping build troubleshooting decision trees.
Given a problem description and category, suggest 4-7 initial symptom branches
that a support engineer would encounter.
Rules:
- Each suggestion should be a distinct, common symptom (not overlapping)
- Order from most common to least common
- Use plain language that a Tier 1 engineer would understand
- Focus on observable symptoms, not root causes
- Respond in JSON format only: { "suggestions": ["symptom1", "symptom2"] }
System Prompt (Stage 2 — Procedure):
You are a senior MSP engineer helping build procedural checklists.
Given a project description and category, suggest 4-8 major phases
that the project should follow.
Rules:
- Phases should be in logical execution order
- Each phase should be a distinct stage of work
- Use standard MSP/IT terminology
- Respond in JSON format only: { "suggestions": ["phase1", "phase2"] }
System Prompt (Stage 3 — Branch Detail):
You are a senior MSP engineer helping build troubleshooting steps.
Given a symptom/branch name and context, suggest 3-6 diagnostic steps.
For each step, provide:
- type: "action" (do something), "decision" (yes/no check), or "solution" (resolution)
- title: short step name
- content: detailed instructions including commands where relevant
- sub_branches: (for decisions only) what the yes/no paths look like
- record_fields: (for procedures only) data the engineer should document
Rules:
- Include specific commands (PowerShell preferred for Windows environments)
- Action steps should be concrete and actionable
- Decision steps should have clear yes/no outcomes
- End branches with solution nodes where possible
- Respond in JSON format only
Caching strategy: System prompts are sent as cacheable prefixes. With a 5-minute TTL, any user hitting the wizard within 5 minutes of another user gets 90% savings on input tokens. During active hours, this cache will almost always be warm.
Frontend Implementation
New Components
frontend/src/
pages/
FlowWizardPage.tsx # Main wizard container with stage navigation
components/
wizard/
WizardStageFoundation.tsx # Stage 1: type, category, name, description
WizardStageScaffold.tsx # Stage 2: AI suggestions with checkboxes
WizardStageBranchDetail.tsx # Stage 3: per-branch step suggestions
WizardStageReview.tsx # Stage 4: assembled flow preview
WizardProgress.tsx # Stage indicator bar
SuggestionCheckbox.tsx # Reusable AI suggestion with check/uncheck
CustomItemInput.tsx # "Add your own" input field
api/
aiGeneration.ts # API client for AI endpoints
hooks/
useAiWizard.ts # State management for wizard flow
Wizard State Management
interface WizardState {
// Stage 1
flowType: 'troubleshooting' | 'procedure';
categoryId: string;
name: string;
description: string;
environmentTags: string[];
// Stage 2
scaffoldSuggestions: string[]; // AI-generated
selectedBranches: string[]; // User-confirmed
customBranches: string[]; // User-added
// Stage 3
branchDetails: Record<string, BranchStep[]>; // Per-branch steps
currentBranchIndex: number;
// Stage 4
assembledStructure: TreeStructure | null;
// Meta
currentStage: 1 | 2 | 3 | 4;
aiCallsUsed: number;
isGenerating: boolean;
}
UX Considerations
-
Loading states: AI calls take 2-5 seconds. Show a subtle loading animation with context ("Analyzing common symptoms..."), not a spinner.
-
Graceful degradation: If an AI call fails, show an empty state with "Add your own" prominent. The wizard should never be blocked by AI failure.
-
Edit-in-place: Users should be able to click any AI suggestion to edit the text before accepting it.
-
Skip AI: Every stage should have a "Skip suggestions, I'll add my own" option. Power users may want to build manually but still benefit from the wizard structure.
-
Mobile consideration: The wizard should work on tablet at minimum. MSP engineers may use it on-site with an iPad.
Environment & Configuration
Environment Variables
# .env / Railway
ANTHROPIC_API_KEY=sk-ant-...
AI_GENERATION_MODEL=claude-haiku-4-5-20251001
AI_GENERATION_ENABLED=true # Kill switch
AI_MAX_RETRIES=2
AI_REQUEST_TIMEOUT=30 # seconds
Feature Flag Integration
Use existing feature_flags system:
{
"ai_flow_generation": {
"enabled": true,
"plans": ["pro", "team"],
"beta_override": true // Allow specific free users during beta
}
}
Rollout Strategy
Phase 3a: Internal Testing (Week 1)
- Backend endpoints + prompt engineering
- Test with admin account only
- Iterate on prompt quality — this is where most time goes
- Validate cost estimates against real API usage
Phase 3b: Beta Testing (Week 2-3)
- Frontend wizard UI
- Enable for beta testers (feature flag)
- Collect feedback on suggestion quality
- Monitor ai_usage_log for actual costs
Phase 3c: General Availability (Week 4)
- Enable for all Pro/Team plans
- Free tier gets limited access (2/month)
- Admin dashboard shows AI cost metrics
- Plan limit enforcement active
Future Enhancements (Not in Initial Scope)
These are explicitly deferred to keep initial scope manageable:
-
AI-suggested branch improvements — "Based on 15 unresolved sessions on this branch, AI suggests adding these steps." Requires session outcome tracking (separate feature).
-
AI step refinement — User selects an existing step and says "make this more detailed" or "add the PowerShell commands." Single-step AI call, very cheap.
-
Template flows from wizard patterns — If many users create similar flows (e.g., "printer troubleshooting"), cache the assembled structure as a template. Future users get instant results with zero AI cost.
-
Multi-language generation — Same wizard, but AI generates steps in the user's language. Prompt change only, no architecture change.
-
AI from session data — After enough sessions are tracked, use anonymized session paths to improve AI suggestions. "80% of engineers check the spooler service first" → AI learns to suggest that first.
Key Risks & Mitigations
| Risk | Impact | Mitigation |
|---|---|---|
| Anthropic API outage | Wizard Stage 2-3 blocked | Graceful fallback to manual entry; AI is enhancement, not requirement |
| Poor suggestion quality | Users lose trust in feature | Extensive prompt testing before launch; user can always edit/override |
| Cost overrun from heavy usage | Margin erosion | Per-account monthly limits in plan_limits; ai_usage_log for monitoring |
| Prompt injection via user input | Unexpected AI output | Sanitize user input; structured JSON output parsing; never execute AI output as code |
| Haiku model deprecated | Service interruption | Model name in env variable, not hardcoded; swap to successor model |
| Users treat AI as authoritative | Bad troubleshooting steps followed blindly | "Draft" status by default; review stage required; clear "AI-suggested" labeling |
Success Metrics
| Metric | Target | How to Measure |
|---|---|---|
| Wizard completion rate | >70% of starts finish Stage 4 | Track stage progression in analytics |
| AI suggestion acceptance rate | >50% of suggestions kept | Compare suggestions shown vs. selected |
| Time to create flow (wizard vs manual) | 60%+ faster with wizard | Compare creation timestamps |
| Post-wizard editing rate | <30% of nodes edited after assembly | Compare Stage 4 structure vs. published version |
| Cost per flow | <$0.03 average | ai_usage_log aggregation |
| User satisfaction | Positive feedback from beta | Direct feedback + usage retention |