resolutionflow/docs/plans/KB-Accelerator-Merged-Implementation-Plan.md

# KB Accelerator — Merged Implementation Plan

## Document Context

| Field | Value |
|-------|-------|
| **Document** | KB Accelerator — Merged Implementation Plan |
| **Version** | 1.0 |
| **Date** | March 2026 |
| **Status** | Approved for Implementation |
| **Source Plans** | Claude Code design review + Codex implementation plan |
| **Design Doc** | `docs/plans/KB-Accelerator-Design-Document.md` |

This plan merges the best elements of two independent implementation plans produced by Claude Code and Codex against the KB Accelerator design document. Where the plans conflicted, explicit decisions were made and are documented below.

---

## 1. Summary of Decisions

### Agreed by Both Plans (Carry Forward As-Is)

- Dedicated KB Accelerator frontend experience — own route (`/kb-accelerator`), own sidebar nav item, own screens
- `account_id` tenancy everywhere — all design doc references to "organization" map to existing `account_id`
- Text + paste + DOCX in Phase 1; PDF, HTML, Markdown in Phase 2
- Both flow types (troubleshooting + procedural) supported from Phase 1
- Single-phase AI conversion by default; optional detailed analysis for Pro/Team
- 3 lifetime conversions for free tier, enforced per account (not per user)
- Hard server-side tier enforcement via PlanLimits columns
- Store extracted text + metadata only — raw uploaded files are not persisted
- File validation + pluggable scan hook interface (no-op default, AV integration ready)
- Per-node review actions: approve, edit, delete, regenerate, insert, plus bulk approve
- Side-by-side two-panel review UI with confidence indicators (green/amber/red left accent borders)
- `import_metadata` JSONB on trees table for provenance — no new FK column on trees
- HTTP polling for progress tracking (no SSE, no WebSockets)
- Multipart `files[]` + shared options for batch upload request shape (Phase 3)
- Auto-advance pipeline: upload → extraction → AI conversion → land on review screen (no manual stage gates)
- Auto-commit as draft for batch imports (Phase 3)
- Feature-flagged analysis preview screen (Pro/Team only)
- Basic shared visibility for Team tier (view/read, not collaborative editing)
- Sidebar nav item + "Import KB Article" CTA in flow library header

### Conflict Resolutions

| Decision | Chosen Approach | Rationale |
|---|---|---|
| **AI Infrastructure** | **Codex: Dedicated KB module** consuming shared AI service layer (model routing, token tracking, quota). NOT coupled to `AIChatSession`. | A KB import is a document conversion, not a chat session. Coupling to `AIChatSession` muddies analytics, session history, and data model semantics. Using shared AI *services* without coupling to the AI *data model* is the right separation. |
| **Per-node staging** | **Codex: Dedicated `kb_import_nodes` table** with proper columns for confidence, source excerpt, approval status. | Queryable (e.g., "all nodes below 0.7 confidence across imports"), normalized, clean PATCH semantics. Avoids the `_kb_meta` JSONB prefix hack which is fragile and risks junk data in production trees if stripping is missed. |
| **Batch import** | **Claude Code: Defer to Phase 3.** | Core single-article conversion must be validated first. Batch adds queue management, partial failure handling, and batch status UI — significant complexity for a feature nobody has requested yet. |
| **Conversational refinement** | **Claude Code's idea, Codex's architecture. Defer to Phase 2.** Built as a scoped chat panel in the review screen, NOT coupled to `AIChatSession`. | High-value feature, but Phase 1 must nail the core loop (upload → convert → review → commit). Refinement panel in Phase 2 uses a dedicated KB chat endpoint scoped to the import context. |
| **Step Library matching** | **Defer to Phase 2.** | Same reasoning — nail the core loop first, then layer on matching. |
| **Status values** | **Claude Code: Simplified to 4** — `processing`, `ready`, `committed`, `failed`. | With single-phase AI and auto-advance, granular statuses (uploaded, extracting, analyzing, generating, reviewed) add complexity without user value. |

---

## 2. Architecture Overview

### Backend: Dedicated KB Module + Shared AI Services

KB Accelerator is a self-contained backend module with its own tables, endpoints, services, and business logic. It does NOT create or depend on `AIChatSession` records.

When AI processing is needed, the KB module calls the existing shared AI service layer:

- **Model routing** via `get_model_for_action()` — add `kb_convert` and `kb_analyze` to `ACTION_MODEL_MAP`
- **Token tracking** via existing token counting utilities
- **Quota enforcement** via `ai_quota_service` (`check_ai_quota`, `record_ai_usage`)
- **Cost tracking** via existing cost recording patterns
- **Anthropic API calls** via existing `AsyncAnthropic` client patterns

The KB module owns its own prompt engineering, extraction logic, pipeline orchestration, and data persistence.

### Frontend: Dedicated KB Accelerator Experience

The frontend is a standalone multi-step wizard UI under `/kb-accelerator`. Users never see "AI Chat" branding or feel like they've left KB Accelerator. The conversational refinement panel (Phase 2) is visually integrated into the KB review screen — it reuses `EditorAIPanel` component internals but is branded and scoped to the KB context.

### Processing Pipeline

```
User uploads file/paste
        │
        ▼
┌─────────────────┐
│   1. UPLOAD      │  Validate format, size, tier permissions
│   & EXTRACT      │  Extract text + structural metadata
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   2. CONVERT     │  Single AI call → tree structure + confidence scores
│   (AI)           │  OR two-phase (Pro/Team optional): analyze → generate
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   3. REVIEW      │  Side-by-side UI, per-node actions, edit/approve/delete
│   (User)         │  + Conversational refinement panel (Phase 2)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   4. COMMIT      │  Create Tree record, set import_metadata, strip staging data
│                  │  Step Library match suggestions (Phase 2)
└─────────────────┘
```

---

## 3. Data Model

### New Table: `kb_imports` (Migration 054)

| Column | Type | Nullable | Description |
|---|---|---|---|
| `id` | UUID PK | No | Primary key (`gen_random_uuid()`) |
| `account_id` | UUID FK → accounts | No | Tenancy scoping |
| `created_by` | UUID FK → users | No | Who initiated the import |
| `source_filename` | VARCHAR(500) | Yes | Original filename (null for paste) |
| `source_format` | VARCHAR(20) | No | Enum: `txt`, `paste`, `docx` (Phase 1); `pdf`, `html`, `md` (Phase 2) |
| `source_text` | TEXT | No | Extracted plain text content |
| `source_metadata` | JSONB | Yes | Structural metadata from extraction (headings, lists, emphasis) |
| `target_type` | VARCHAR(20) | No | Enum: `troubleshooting`, `procedural` |
| `status` | VARCHAR(20) | No | Enum: `processing`, `ready`, `committed`, `failed` |
| `confidence_avg` | FLOAT | Yes | Average confidence across all generated nodes |
| `error_message` | TEXT | Yes | Error details if status = `failed` |
| `processing_time_ms` | INTEGER | Yes | Total processing time in milliseconds |
| `ai_tokens_input` | INTEGER | Yes | Total input tokens used for AI processing |
| `ai_tokens_output` | INTEGER | Yes | Total output tokens used for AI processing |
| `tree_id` | UUID FK → trees | Yes | Set after user commits (null until then) |
| `batch_id` | UUID | Yes | Groups batch imports together (Phase 3) |
| `created_at` | TIMESTAMPTZ | No | Auto-set on creation |
| `updated_at` | TIMESTAMPTZ | No | Auto-updated on modification |

**Indexes:** `account_id`, `status`, `batch_id`, `created_by`, `created_at DESC`.

### New Table: `kb_import_nodes` (Migration 054)

Stores individual generated nodes/steps during the review phase. Each row represents one node in the AI-generated flow before the user commits it to an actual tree.

| Column | Type | Nullable | Description |
|---|---|---|---|
| `id` | UUID PK | No | Primary key |
| `kb_import_id` | UUID FK → kb_imports | No | Parent import |
| `node_order` | INTEGER | No | Position in the generated flow (0-indexed) |
| `node_type` | VARCHAR(20) | No | Enum: `question`, `resolution`, `step`, `section_header`, `warning` |
| `content` | JSONB | No | Node content (question text, step text, options array, etc.) |
| `parent_node_id` | UUID FK → kb_import_nodes | Yes | Parent node (for tree structure) |
| `source_excerpt` | TEXT | Yes | Exact text from source document this node was derived from |
| `confidence_score` | FLOAT | No | AI confidence in this node's accuracy (0.0–1.0) |
| `user_edited` | BOOLEAN | No | Default `false`. Set `true` when user modifies content |
| `user_approved` | BOOLEAN | No | Default `false`. Set `true` when user explicitly approves |
| `created_at` | TIMESTAMPTZ | No | Auto-set on creation |
| `updated_at` | TIMESTAMPTZ | No | Auto-updated on modification |

**Indexes:** `kb_import_id`, `confidence_score`.

### Tree `import_metadata` JSONB Schema (Set on Commit)

When a user commits a KB Accelerator flow, the resulting tree's `import_metadata` column is populated:

```json
{
  "source": "kb_accelerator",
  "kb_import_id": "uuid-here",
  "source_filename": "Exchange-Troubleshooting.docx",
  "source_format": "docx",
  "confidence_avg": 0.85,
  "node_count": 12,
  "converted_at": "2026-03-10T14:30:00Z"
}
```

### PlanLimits Extensions

Add the following columns to the existing `plan_limits` table (and corresponding `account_limit_overrides`, admin schemas, subscription schemas, and frontend types):

| Column | Type | Description |
|---|---|---|
| `kb_accelerator_enabled` | BOOLEAN | Whether KB Accelerator is available on this plan |
| `kb_max_lifetime_conversions` | INTEGER, nullable | Lifetime cap (null = unlimited). Free = 3. |
| `kb_batch_max_size` | INTEGER, nullable | Max files per batch upload (null = disabled). Phase 3. |
| `kb_allowed_formats` | JSONB | Array of allowed format strings. Free = `["txt", "paste"]`. Pro/Team = all. |
| `kb_detailed_analysis` | BOOLEAN | Whether optional two-phase analysis is available |
| `kb_conversational_refinement` | BOOLEAN | Whether AI refinement panel is available (Phase 2) |
| `kb_step_library_matching` | BOOLEAN | Whether Step Library matching is available (Phase 2) |
| `kb_history_limit` | INTEGER, nullable | Max visible import history entries (null = unlimited). Free = 3. |

**Seed defaults:**

| Plan | enabled | lifetime_cap | batch_max | formats | detailed_analysis | refinement | step_matching | history_limit |
|---|---|---|---|---|---|---|---|---|
| **Free** | true | 3 | null | `["txt", "paste"]` | false | false | false | 3 |
| **Pro** | true | null | 5 | `["txt", "paste", "docx", "pdf", "html", "md"]` | true | true | true | null |
| **Team** | true | null | 10 | `["txt", "paste", "docx", "pdf", "html", "md"]` | true | true | true | null |

---

## 4. API Design

All endpoints under `/api/v1/kb-accelerator`. All require authentication. All records scoped to `account_id`. Role enforcement: `require_engineer_or_admin`.

### Endpoints

| Method | Endpoint | Description | Phase |
|---|---|---|---|
| `POST` | `/upload` | Upload file or paste text. Creates `kb_import`, starts extraction, triggers auto-convert. Returns `kb_import_id`. | 1 |
| `GET` | `/{id}` | Get import status, source text preview, generated nodes, confidence stats. | 1 |
| `GET` | `/` | List imports for current account. Pagination + status filter. Respects `kb_history_limit`. | 1 |
| `POST` | `/{id}/convert` | Manually trigger or re-trigger AI conversion. For retry/regeneration scenarios. | 1 |
| `PATCH` | `/{id}/nodes/{node_id}` | Edit a specific node. Operations: `approve`, `reject`, `edit`, `delete`, `regenerate`, `insert_after`. | 1 |
| `POST` | `/{id}/commit` | Finalize: create Tree record from reviewed nodes, populate `import_metadata`, update status to `committed`. | 1 |
| `DELETE` | `/{id}` | Cancel and clean up an in-progress or abandoned import. | 1 |
| `GET` | `/quota` | Return current plan KB entitlements, usage counts, and UI flags (detailed_analysis, refinement, etc.). | 1 |
| `POST` | `/{id}/analyze` | (Pro/Team) Trigger detailed two-phase analysis before generation. | 2 |
| `POST` | `/{id}/refine` | Send a refinement message scoped to this import's context. Returns updated nodes. | 2 |
| `POST` | `/batch` | Submit multiple files. Returns `batch_id` + array of `kb_import_id`s. | 3 |
| `GET` | `/batch/{batch_id}` | Get grouped batch status and per-import outcomes. | 3 |
| `GET` | `/metrics` | KPI dashboard data: conversion rate, avg confidence, format usage, etc. | 3 |

### Upload Endpoint Detail

**`POST /api/v1/kb-accelerator/upload`**

Accepts `multipart/form-data` (file upload) or `application/json` (text paste).

**Request — File Upload:**
- `file`: UploadFile (required) — the KB article file
- `target_type`: string (optional) — `"troubleshooting"` or `"procedural"`. If omitted, AI decides.

**Request — Text Paste:**
- `content`: string (required) — raw text content
- `title`: string (optional) — suggested title
- `target_type`: string (optional)

**Validation:**
- Max file size: 10MB
- Format whitelist: `.txt`, `.docx` (Phase 1); `.pdf`, `.html`, `.md` (Phase 2)
- MIME type verification (content matches extension)
- Tier format check against `kb_allowed_formats`
- Lifetime conversion count check against `kb_max_lifetime_conversions`

**Response (201 Created):**
```json
{
  "id": "uuid",
  "status": "processing",
  "source_format": "docx"
}
```

**Pipeline behavior:** After successful upload and extraction, the auto-convert pipeline triggers immediately. Frontend polls `GET /{id}` until status changes from `processing` to `ready` (or `failed`).

### Node Edit Endpoint Detail

**`PATCH /api/v1/kb-accelerator/{id}/nodes/{node_id}`**

Supports a union of operations:

- **`approve`**: Sets `user_approved = true`
- **`reject`**: Sets `user_approved = false`
- **`edit`**: Updates `content` JSONB, sets `user_edited = true`
- **`delete`**: Removes the node, reorders remaining nodes
- **`regenerate`**: Re-runs AI generation for this single node with optional user guidance text. Uses shared AI service.
- **`insert_after`**: Creates a new node after this one, shifts `node_order` for subsequent nodes

### Commit Endpoint Detail

**`POST /api/v1/kb-accelerator/{id}/commit`**

1. Validate all nodes are reviewed (or allow commit with unreviewed nodes — user's choice)
2. Build `tree_structure` JSONB from `kb_import_nodes` rows
3. Create Tree record with appropriate `tree_type` (`troubleshooting` or `procedural`)
4. For procedural flows: include generated `intake_form` schema from detected variables
5. Set `import_metadata` JSONB with provenance data
6. Update `kb_import.status` to `committed`, set `kb_import.tree_id`
7. Run best-effort RAG indexing on the new tree
8. Record audit event

**Batch behavior (Phase 3):** Successful batch items auto-commit as draft trees. Failed items retain `failed` status with error details.

---

## 5. AI Pipeline

### Single-Phase Conversion (Default)

One AI call that takes extracted text and returns a complete tree structure.

**System Prompt establishes:**
- AI role as MSP documentation specialist
- Target flow type (troubleshooting or procedural)
- ResolutionFlow tree schema with examples (reuse patterns from `ai_chat_service.py`)
- Confidence scoring instructions (0.0–1.0 per node with criteria)
- Source excerpt attribution requirement (every node must cite its source text)
- Variable detection instructions for procedural flows (`[VAR:name]` tokens)

**User message contains:**
- Extracted text with structural metadata (headings, lists, emphasis markers)
- Source filename and format for context

**Expected response:** Strict JSON matching the structure needed to populate `kb_import_nodes` rows, including `node_type`, `content`, `confidence_score`, `source_excerpt`, and parent-child relationships.

**Model routing:** Add `kb_convert` to `ACTION_MODEL_MAP` → maps to Sonnet (standard tier).

**Token tracking:** Record `ai_tokens_input` and `ai_tokens_output` on the `kb_import` record. Also call `record_ai_usage` for quota/cost tracking through the shared service.

### Two-Phase Analysis + Generation (Optional, Pro/Team)

**Phase 1 — Analysis:** AI returns structured JSON of detected elements (document type, problem statement, prerequisites, sequential steps, decision points, variables, warnings, resolutions, verification steps). Stored in `kb_import.source_metadata` or a dedicated analysis column.

**Phase 2 — Generation:** Takes Phase 1 analysis + original text → generates tree structure (same output as single-phase).

**Model routing:** Add `kb_analyze` to `ACTION_MODEL_MAP`.

### Confidence Scoring

| Score Range | Label | UI Indicator |
|---|---|---|
| 0.9 – 1.0 | High Confidence | Green left accent border |
| 0.7 – 0.89 | Medium Confidence | Amber left accent border |
| 0.5 – 0.69 | Low Confidence | Red left accent border |
| < 0.5 | Needs Review | Red left accent border + flag icon |

### Procedural Flow: Variable Detection

For procedural target type, the AI identifies instance-specific values and maps them to `[VAR:name]` tokens:

| Pattern | Variable Name | Form Field Type |
|---|---|---|
| IP addresses | `[VAR:ip_address]` | ip_address |
| Server/computer names | `[VAR:server_name]` | text |
| Domain names | `[VAR:domain_name]` | text |
| Usernames/email | `[VAR:username]` | text |
| License types | `[VAR:license_type]` | select |
| OU paths | `[VAR:ou_path]` | text |
| Port numbers | `[VAR:port]` | number |
| Subnet masks | `[VAR:subnet_mask]` | ip_address |

An `intake_form` JSONB schema is auto-generated from detected variables and stored on the committed tree.

---

## 6. Frontend Design

### Route: `/kb-accelerator`

Multi-step wizard with 3-4 screens, all within the existing app shell (sidebar + topbar). Uses the current design system: dark theme, cyan brand color, glass morphism, IBM Plex Sans / Bricolage Grotesque / JetBrains Mono fonts.

### Screen 1: Upload

- Drag-and-drop zone for files with format badges (DOCX, TXT in Phase 1)
- Tab switch to "Paste Text" with full-width textarea + title field
- Target type selector: two visual cards (Troubleshooting Flow / Procedure Flow) + "Let AI decide" option
- Primary action: "Convert" button (`bg-gradient-brand`)
- Pro/Team users see additional "Detailed Analysis" button alongside "Convert"
- Container: `.glass-card-static`
- Tier gating: free users see format restrictions and remaining conversion count

### Screen 2: Analysis Preview (Phase 2, Pro/Team Only, Feature-Flagged)

- Shows detected elements as color-coded cards: steps (blue), decision points (amber), warnings (red), variables (green), resolutions (emerald)
- Source text excerpts linked to each detection
- "Proceed to Generation" and "Re-analyze" action buttons
- Only accessible when user clicks "Detailed Analysis" on the upload screen

### Screen 3: Review (Core Experience)

**Two-Panel Side-by-Side Layout:**

- **Left panel:** Original document text with detected elements highlighted inline (color-matched to generated nodes)
- **Right panel:** Generated flow preview — tree visualization for troubleshooting, step list for procedures
- Clicking a node in the right panel highlights its source excerpt in the left panel, and vice versa
- Each node shows confidence score via left accent border pattern (green/amber/red)

**Per-Node Actions:**

- **Approve** (checkmark): Sets `user_approved = true`
- **Edit** (pencil): Opens inline editing for content, question text, options
- **Regenerate** (refresh): Re-runs AI for just this node with optional guidance
- **Delete** (trash): Removes node from generated flow
- **Add Node** (plus): Insert a manual node after this one

**Bulk Actions:**

- "Approve All High Confidence" — one-click approval for all nodes scoring ≥ 0.9
- "Commit to Library" — finalizes the flow

**AI Refinement Panel (Phase 2):** Slide-in panel on the review screen for conversational refinement. User types natural language instructions ("Add a warning about DNS propagation after step 4", "Split this decision point"). Scoped to the KB import context — NOT the general FlowPilot chat. Reuses `EditorAIPanel` component internals with KB-specific branding.

**Step Library Suggestions (Phase 2):** For procedural flows, matched steps show a "Link to Library" badge. Clicking shows the library step content and lets the user swap the generated step for the library step.

### Screen 4: Success

- Confirmation with link to the new flow in the editor
- "Convert Another" button
- Stats: average confidence score, node count, processing time

### Navigation

- **Sidebar:** "KB Accelerator" nav item with sparkle/lightning icon
- **Flow library header:** "Import KB Article" button next to "Create New Flow"

---

## 7. Tier Gating

| Capability | Free | Pro ($19/mo) | Team ($15/user/mo) |
|---|---|---|---|
| **Conversions** | 3 lifetime (account-wide) | Unlimited | Unlimited |
| **Formats** | TXT + paste only | All formats | All formats |
| **Target type selection** | AI decides only | Manual + AI | Manual + AI |
| **Detailed analysis** | No | Yes | Yes |
| **Conversational refinement** | No | Yes (Phase 2) | Yes (Phase 2) |
| **Step Library matching** | No | Yes (Phase 2) | Yes (Phase 2) |
| **Review actions** | Approve / Edit / Delete | Full (+ regenerate, insert, bulk approve) | Full (+ regenerate, insert, bulk approve) |
| **Import history** | Last 3 only | Full history | Full history + audit log |
| **Batch import** | No | Up to 5 articles (Phase 3) | Up to 10 articles (Phase 3) |
| **Team visibility** | N/A | N/A | Shared read access to imports |

**Enforcement:** Hard server-side checks on every endpoint. Check `subscription.plan` → `PlanLimits` columns. Free tier lifetime count = `COUNT(*) FROM kb_imports WHERE account_id = ? AND status = 'committed'`.

---

## 8. Build Phases

### Phase 1: Core Pipeline (Target: 2–3 Weeks)

The goal is a complete, working single-article conversion loop for text, paste, and DOCX inputs producing both troubleshooting and procedural flows.

**Backend:**

- Migration 054: `kb_imports` and `kb_import_nodes` tables
- Migration 055: `PlanLimits` KB Accelerator columns + seed defaults
- Upload endpoint — text, paste, DOCX extraction (python-docx)
- Single-phase AI conversion — prompt engineering, structured JSON parsing, node creation
- Node edit endpoint — approve, reject, edit, delete, regenerate, insert_after
- Commit endpoint — create Tree, set `import_metadata`, strip staging data, RAG indexing
- List/get import endpoints with pagination and status filter
- Quota endpoint — return plan entitlements and usage counts
- Delete/cancel endpoint
- Hard tier gating — format checks, lifetime conversion count, review action restrictions
- Add `kb_convert` to `ACTION_MODEL_MAP`
- Extraction service module (TXT, paste, DOCX) with pluggable architecture for Phase 2 formats
- Upload validation service — extension, MIME, size, pluggable scan hook (no-op default)

**Frontend:**

- Upload screen — drag-drop zone, paste tab, target type cards, "Let AI decide"
- Review screen — two-panel layout, confidence indicators, per-node actions, source highlighting
- Success screen — confirmation, stats, "Convert Another"
- Sidebar nav item + flow library CTA button
- KB Accelerator API client module (`kbAccelerator.ts`)
- TypeScript types (`kbAccelerator.ts`)
- HTTP polling for processing status
- Tier gating UI — format restrictions shown, remaining conversions shown, upgrade prompts for locked features

**Both flow types** (troubleshooting + procedural) supported from Phase 1 start.

### Phase 2: Rich Formats & Refinement (Target: 2–3 Weeks)

Layer on additional formats, the power-user analysis preview, conversational refinement, and Step Library matching.

**Backend:**

- PDF extraction via PyMuPDF with extraction preview/correction endpoint (user verifies extracted text before AI processing)
- HTML extraction via BeautifulSoup
- Markdown extraction via markdown-it-py
- Detailed analysis endpoint — two-phase AI (analyze → generate), Pro/Team gated
- Conversational refinement endpoint — scoped chat for the KB import context, uses shared AI service, NOT `AIChatSession`
- Step Library matching service — compare generated procedural steps against user's Step Library (text similarity or pgvector embeddings)
- Add `kb_analyze` and `kb_refine` to `ACTION_MODEL_MAP`

**Frontend:**

- PDF extraction preview screen — shows extracted text, highlights potential issues, user can edit before AI processing
- Analysis preview screen — feature-flagged for Pro/Team, shows detected elements as color-coded cards
- AI refinement slide-in panel on review screen — reuses `EditorAIPanel` internals with KB branding
- Step Library match suggestions — "Link to Library" badges on matched procedural steps
- "Approve All High Confidence" bulk action button

### Phase 3: Scale & Polish (Future)

Batch import, history dashboard, and analytics.

**Backend:**

- Batch upload endpoint — multipart `files[]` + shared options, returns `batch_id` + import IDs
- Batch status endpoint
- FastAPI background jobs for batch processing (DB-based job queue)
- Auto-commit as draft for successful batch items
- Import history dashboard endpoint
- Metrics/analytics endpoint — conversion rate, avg confidence, format usage, time trends

**Frontend:**

- Batch upload UI — multi-file drag-drop with per-file status indicators
- Batch results view — shows auto-committed drafts and failed items
- Import history dashboard with filters and search
- Analytics visualizations (conversion trends, confidence distributions)

---

## 9. Files to Create and Modify

### New Files

| File | Purpose |
|---|---|
| `backend/alembic/versions/054_add_kb_imports.py` | Migration: `kb_imports` + `kb_import_nodes` tables |
| `backend/alembic/versions/055_add_kb_plan_limits.py` | Migration: PlanLimits KB columns + seed defaults |
| `backend/app/models/kb_import.py` | SQLAlchemy models: `KBImport`, `KBImportNode` |
| `backend/app/schemas/kb_accelerator.py` | Pydantic schemas: request/response DTOs |
| `backend/app/api/endpoints/kb_accelerator.py` | API endpoints |
| `backend/app/core/kb_extraction_service.py` | Text extraction (TXT, paste, DOCX; extensible for Phase 2 formats) |
| `backend/app/core/kb_conversion_service.py` | AI prompt orchestration, JSON parsing, node creation |
| `backend/tests/test_kb_accelerator.py` | Integration tests |
| `frontend/src/api/kbAccelerator.ts` | API client module |
| `frontend/src/types/kbAccelerator.ts` | TypeScript types |
| `frontend/src/pages/KBAcceleratorPage.tsx` | Main page (multi-step wizard) |
| `frontend/src/components/kb-accelerator/UploadScreen.tsx` | Upload UI component |
| `frontend/src/components/kb-accelerator/ReviewScreen.tsx` | Two-panel review UI component |
| `frontend/src/components/kb-accelerator/SuccessScreen.tsx` | Post-commit confirmation component |
| `frontend/src/components/kb-accelerator/NodeCard.tsx` | Individual node display with confidence + actions |
| `frontend/src/components/kb-accelerator/SourcePanel.tsx` | Left panel: source text with highlights |

### Modified Files

| File | Change |
|---|---|
| `backend/app/models/__init__.py` | Import `KBImport`, `KBImportNode` |
| `backend/alembic/env.py` | Import KB models for migration detection |
| `backend/app/api/router.py` | Register `kb_accelerator` router |
| `backend/app/core/config.py` | Add `kb_convert` (Phase 1), `kb_analyze`, `kb_refine` (Phase 2) to `ACTION_MODEL_MAP` |
| `backend/app/models/plan_limits.py` | Add KB Accelerator limit columns |
| `frontend/src/router.tsx` | Add `/kb-accelerator` route |
| `frontend/src/components/layout/AppLayout.tsx` or `Sidebar.tsx` | Add KB Accelerator sidebar nav item |
| `frontend/src/types/index.ts` | Export KB Accelerator types |
| `frontend/src/api/index.ts` | Export KB Accelerator API client |

### Existing Files Reused (Not Modified)

| File | What's Reused |
|---|---|
| `backend/app/core/ai_chat_service.py` | Prompt patterns, structured output parsing examples |
| `backend/app/core/ai_quota_service.py` | `check_ai_quota()`, `record_ai_usage()` |
| `backend/app/core/ai_provider_service.py` | `get_model_for_action()`, Anthropic client patterns |
| `frontend/src/components/tree-editor/EditorAIPanel.tsx` | Component internals reused for refinement panel (Phase 2) |

---

## 10. Test Plan

### Backend Integration Tests

**Upload & Extraction:**
- Upload text/paste → verify `kb_import` created with status `processing`
- Upload DOCX → verify extraction produces `source_text` and `source_metadata`
- Upload unsupported format → verify 400 rejection
- Upload exceeding 10MB → verify 413 rejection
- Upload DOCX on free tier → verify 403 (format not in plan)
- Upload when lifetime limit reached → verify 403 with upgrade message

**AI Conversion:**
- Convert troubleshooting article → verify `kb_import_nodes` created with correct types, confidence scores, source excerpts
- Convert procedural article → verify step nodes created with `[VAR:name]` tokens and `intake_form` data
- Convert with AI failure → verify status set to `failed` with error message
- Verify token counts recorded on `kb_import`
- Verify `record_ai_usage` called through shared service

**Node Review Actions:**
- Approve node → verify `user_approved = true`
- Edit node → verify `content` updated, `user_edited = true`
- Delete node → verify removed, `node_order` resequenced
- Regenerate node → verify AI called, node content replaced, new confidence score
- Insert after → verify new node created with correct `node_order`, subsequent nodes shifted

**Commit:**
- Commit troubleshooting import → verify Tree created with correct `tree_type`, `tree_structure`, `import_metadata`
- Commit procedural import → verify Tree created with `intake_form` populated
- Verify `kb_import.status` = `committed`, `tree_id` set
- Verify committed tree appears in flow library
- Verify RAG indexing triggered (best-effort)

**Tier Enforcement:**
- Free tier: 4th conversion rejected (account-scoped lifetime count)
- Free tier: DOCX upload rejected, paste accepted
- Pro tier: unlimited conversions, all formats accepted
- Team tier: other account members can view import (shared visibility)

### Frontend Tests

- Upload flow: file drag-drop, paste, target type selection, validation messages
- Polling: status transitions from `processing` to `ready`
- Review screen: node display, confidence colors, source highlighting, click-to-highlight linking
- Node actions: inline edit, approve, delete — optimistic UI updates
- Commit flow: success screen, link to editor works
- Tier gating: free tier sees upgrade prompts, format restrictions shown, conversion count displayed

### E2E Smoke Test

1. Paste a sample KB article text
2. Select "Troubleshooting Flow"
3. Click "Convert"
4. Wait for processing → land on review screen
5. Verify nodes displayed with confidence indicators
6. Edit one low-confidence node
7. Approve all high-confidence nodes
8. Click "Commit to Library"
9. Verify flow appears in library
10. Open flow in tree editor — verify structure is correct

---

## 11. Success Metrics (Post-Launch)

| Metric | Target | Why It Matters |
|---|---|---|
| **Conversion completion rate** | > 70% | Imports reaching `committed` status. Below 70% = review too burdensome or quality too low. |
| **Average confidence score** | > 0.75 | Across all generated nodes. Indicates AI pipeline accuracy. |
| **Time from upload to commit** | < 10 minutes | Full cycle should feel fast. 30+ minutes = AI not saving enough time. |
| **Free-to-Pro conversion rate** | > 15% | Users who exhaust 3 free conversions then upgrade. Validates feature drives revenue. |
| **Repeat usage (Pro/Team)** | > 3 imports/month | Sustained usage indicates feature is part of workflow, not a one-time novelty. |
| **Node edit rate** | < 30% | Percentage of nodes edited before commit. Lower = AI output more usable as-is. |
| **Imported flow usage rate** | > 50% | Committed flows used in sessions within 30 days. Low = conversion producing shelfware. |

---

*End of Plan*

*ResolutionFlow LLC — March 2026*