diff --git a/docs/plans/2026-03-19-phase5-analytics-enhancement-design.md b/docs/plans/2026-03-19-phase5-analytics-enhancement-design.md new file mode 100644 index 00000000..f148a0ab --- /dev/null +++ b/docs/plans/2026-03-19-phase5-analytics-enhancement-design.md @@ -0,0 +1,166 @@ +# Phase 5: Analytics Enhancement — Design Document + +> **Date:** 2026-03-19 +> **Status:** Approved +> **Audience:** Team leads (primary), individual engineers (secondary) +> **Future note:** A dedicated `/insights` dashboard is planned for a later phase — keep analytics queries modular for reuse. + +--- + +## Overview + +Extend the existing FlowPilot Analytics page with tabbed sections for coverage analysis, flow quality scoring, and PSA metrics. This completes the pivot architecture's Phase 5 ("Analytics + Polish") and provides the data team leads need to justify ROI and identify knowledge gaps. + +## Page Structure + +Add 4 tab sections to `FlowPilotAnalyticsPage`. Each tab fetches data lazily (only when selected). + +| Tab | Content | Data Source | +|-----|---------|-------------| +| **Overview** | Existing metrics (sessions, resolution rate, MTTR, rating, confidence tiers, domain breakdown) | Existing `GET /analytics/flowpilot` | +| **Coverage** | Domain grid heatmap, domain-to-flow mapping, knowledge gap summary | New `GET /analytics/flowpilot/coverage` | +| **Flow Quality** | Flow scoring table, top/bottom performers, needs-attention badges | New `GET /analytics/flowpilot/flow-quality` | +| **PSA** | Time entries, hours tracked, push success funnel, trend chart | Extended existing endpoint + new tracking | + +--- + +## Backend + +### New Endpoint: Coverage Data + +`GET /analytics/flowpilot/coverage?period=7d|30d|90d` + +Response: +```json +{ + "domains": [ + { + "domain": "Active Directory", + "flow_count": 12, + "session_count": 87, + "resolution_rate": 0.92, + "escalation_rate": 0.05, + "guided_rate": 0.78, + "avg_resolution_minutes": 12.5 + } + ], + "unmapped_session_count": 15, + "total_domains": 8 +} +``` + +Domain-to-flow mapping: match flows to domains via their category (categories already exist on trees). Sessions have `problem_domain`. Cross-reference to compute coverage per domain. + +### New Endpoint: Flow Quality Data + +`GET /analytics/flowpilot/flow-quality?period=7d|30d|90d&sort=quality|usage|success_rate` + +Response: +```json +{ + "flows": [ + { + "flow_id": "uuid", + "name": "AD Password Reset", + "usage_count": 45, + "success_rate": 0.91, + "last_matched_at": "2026-03-18T...", + "avg_confidence": 0.82, + "quality_score": 0.87 + } + ], + "top_performers": [...], + "needs_attention": [...] +} +``` + +Quality score formula: `(success_rate * 0.5) + (guided_rate * 0.3) + (recency_score * 0.2)` where recency_score decays from 1.0 (used today) to 0.0 (not used in 90+ days). + +### Flow Model Additions + +Add to `trees` table: +- `usage_count` (Integer, default 0) — incremented when a session matches this flow +- `success_rate` (Float, nullable) — recalculated periodically by analytics query +- `last_matched_at` (DateTime(timezone=True), nullable) — updated on flow match + +### PSA Activity Tracking + +Add `psa_activity_log` table: +- `id` (UUID PK) +- `account_id` (UUID FK) +- `session_id` (UUID FK, nullable) +- `activity_type` (String) — "time_entry_posted", "note_posted", "status_updated" +- `hours_logged` (Float, nullable) — for time entries +- `psa_ticket_id` (String) +- `created_at` (DateTime) + +Wire into the ConnectWise provider: when a time entry or note is successfully pushed, log to this table. + +Extended PSA analytics response: +```json +{ + "total_time_entries": 142, + "total_hours_logged": 356.5, + "avg_hours_per_session": 2.51, + "push_funnel": { + "total_sessions": 500, + "linked_to_ticket": 380, + "doc_pushed": 350, + "time_entry_logged": 142 + }, + "daily_trend": [ + {"date": "2026-03-18", "entries": 8, "hours": 19.5} + ] +} +``` + +--- + +## Frontend + +### Coverage Tab — Domain Grid Heatmap + +A table with colored cells: + +| Domain | Flows | Sessions | Resolution % | Escalation % | Guided % | +|--------|-------|----------|-------------|-------------|---------| +| Active Directory | 12 | 87 | 92% (green) | 5% (green) | 78% (green) | +| Microsoft 365 | 3 | 45 | 58% (red) | 32% (red) | 28% (red) | + +Cell coloring: +- Resolution rate: `bg-emerald-400/10` (>75%), `bg-amber-400/10` (50-75%), `bg-rose-500/10` (<50%) +- Escalation rate: green (<10%), amber (10-25%), red (>25%) +- Guided rate: green (>60%), amber (30-60%), red (<30%) +- Flow count: green (5+), amber (1-4), red (0) + +Domains with red cells show "Create Flow" suggestion. + +### Flow Quality Tab — Sortable Table + +`.glass-card-static` table: +- Columns: Flow Name, Usage, Success Rate, Last Used, Avg Confidence, Quality Score +- Top 5 highlighted in emerald, bottom 5 in rose +- "Needs attention" badge: success_rate < 50% or unused 30+ days +- Click flow name → navigate to editor + +### PSA Tab + +- Metric cards: total entries, total hours, avg hours/session +- Push success funnel: horizontal funnel visualization with conversion rates +- Trend chart (Recharts area chart): daily time entries and hours + +--- + +## Testing + +Backend: +- Coverage endpoint returns correct domain mapping +- Flow quality scoring formula produces expected results +- PSA activity logging works on push +- All endpoints team_admin gated + +Frontend: +- Tab switching loads correct data +- Heatmap cells colored correctly +- Flow quality table sorts correctly +- PSA funnel displays conversion rates