resolutionflow/docs/plans/2026-02-15-analytics-feedback-design.md

Analytics & User Feedback — Design Document

Date: February 15, 2026 Status: Approved Audience: Team admins + individual engineers (both equally)

---

Goal

Add analytics dashboards and user feedback systems to ResolutionFlow so MSP managers can measure team productivity and flow effectiveness, engineers can track their own performance, and flow authors get actionable feedback to improve their troubleshooting trees.

Architecture

Live queries against existing PostgreSQL tables (sessions, trees, step_library) with one new table (session_ratings). No materialized views or external analytics services — direct aggregation queries with proper indexes. Time-series data returned as daily-bucketed arrays for Recharts visualization on the frontend.

Tech Stack Additions

• Backend: New endpoint modules (analytics.py, ratings.py), one Alembic migration
• Frontend: Recharts (charting), two new pages, one new modal, inline step feedback components
• No new infrastructure — runs on existing PostgreSQL + Railway deployment

---

1. Feedback System Design

Step-Level Feedback: Thumbs Up / Thumbs Down

When: Inline during session navigation, always visible on each step.

UX:

• Small thumb-up and thumb-down icons displayed on each step in TreeNavigationPage and ProceduralNavigationPage
• Non-intrusive: muted icons that highlight on selection
• First-time tooltip: "Rate this step to help improve flows" (dismissible, shown once)
• After rating: selected thumb highlights (green for up, red for down), other thumb dims
• Tapping the same thumb again un-rates (toggle behavior)

Data model:

• Uses existing step_ratings table with was_helpful boolean
• One rating per user per step per session (unique constraint on step_id + user_id + session_id)
• Updates step_library.helpful_yes / helpful_no aggregate counts

Endpoint:

• POST /steps/{step_id}/feedback — body: { session_id, was_helpful: true|false }
• DELETE /steps/{step_id}/feedback/{session_id} — un-rate

Flow-Level Feedback: CSAT 1-5 + Optional Comment

When: After session completion, prompted after the SessionOutcomeModal.

UX:

• Modal with 1-5 star selector (or numbered buttons)
• Optional comment textarea (500 char max)
• "Skip" button to dismiss without rating
• Shown once per completed session

Data model:

• New session_ratings table:
  • id UUID PK
  • session_id FK unique (one rating per session)
  • user_id FK
  • tree_id FK (denormalized for aggregation)
  • account_id FK (denormalized for team scoping)
  • rating Integer 1-5 (CHECK constraint)
  • comment String(500), nullable
  • created_at DateTime(timezone=True)

Endpoint:

• POST /sessions/{session_id}/rate — body: { rating: 1-5, comment?: string }
• GET /trees/{tree_id}/ratings — paginated list of ratings/comments for flow authors

---

2. Analytics Endpoints

All analytics endpoints accept ?period=7d|30d|90d (default 30d) and return data scoped to the user's account.

Team Analytics — GET /analytics/team

Access: team_admin or super_admin only.

Response:

{
  "summary": {
    "total_sessions": 247,
    "completed_sessions": 198,
    "completion_rate": 0.801,
    "avg_duration_minutes": 12.4,
    "active_engineers": 8,
    "outcome_breakdown": {
      "resolved": 142,
      "escalated": 31,
      "workaround": 18,
      "unresolved": 7
    }
  },
  "time_series": [
    { "date": "2026-02-01", "sessions": 12, "resolved": 8, "escalated": 2, "workaround": 1, "unresolved": 1 },
    ...
  ],
  "top_flows": [
    { "tree_id": "...", "name": "DNS Resolution", "sessions": 42, "completion_rate": 0.88, "avg_duration_minutes": 8.2, "avg_csat": 4.1 },
    ...
  ],
  "top_engineers": [
    { "user_id": "...", "name": "Jane Smith", "sessions": 34, "completion_rate": 0.91, "avg_duration_minutes": 10.1 },
    ...
  ]
}

Optional filter: ?engineer_id=<uuid> to scope to one engineer.

Personal Analytics — GET /analytics/me

Access: Any authenticated user.

Response: Same shape as team analytics but scoped to the requesting user only. No engineer leaderboard — replaced with "my top flows" list.

Flow Analytics — GET /analytics/flows/{tree_id}

Access: Anyone who can view the flow.

Response:

{
  "summary": {
    "total_sessions": 42,
    "completion_rate": 0.88,
    "avg_duration_minutes": 8.2,
    "avg_csat": 4.1,
    "total_ratings": 28,
    "outcome_breakdown": { ... }
  },
  "time_series": [
    { "date": "2026-02-01", "sessions": 3, "avg_duration_minutes": 7.5 },
    ...
  ],
  "step_feedback": [
    { "node_id": "abc", "node_title": "Check DNS Settings", "helpful_yes": 18, "helpful_no": 2, "helpful_rate": 0.9 },
    ...
  ],
  "recent_comments": [
    { "rating": 5, "comment": "Very helpful flow", "user_name": "John", "created_at": "..." },
    ...
  ]
}

---

3. Frontend Pages

Team Analytics Page — /analytics

Access: team_admin+ (hidden from engineers/viewers in nav).

Layout:

• Page header: "Team Analytics" + period dropdown (7d / 30d / 90d)
• Row 1: Stat cards — Total Sessions, Completion Rate, Avg Duration, Active Engineers
• Row 2: Time-series chart (sessions per day, stacked by outcome) — Recharts AreaChart
• Row 3: Two columns:
  • Left: Flow Leaderboard table (top flows by usage, with completion rate + avg duration + CSAT)
  • Right: Engineer Leaderboard table (top engineers by session count, with success rate + avg duration)

My Analytics Page — /analytics/me

Access: Any authenticated user.

Layout:

• Page header: "My Analytics" + period dropdown
• Row 1: Stat cards — My Sessions, My Completion Rate, My Avg Duration, My Outcome Split
• Row 2: Sessions-per-day line chart
• Row 3: Two columns:
  • Left: My Top Flows table (most-used flows with personal stats)
  • Right: Outcome distribution donut chart (Recharts PieChart)

Flow Analytics Panel

Location: New tab or expandable section on tree detail/editor views.

Layout:

• Summary stat cards: Usage, Completion Rate, Avg Duration, CSAT
• Session trend mini-chart (sparkline or small area chart)
• Step feedback table: each step with helpful rate bar + thumbs count
• Recent CSAT comments list (latest 5-10)

Navigation

• Sidebar: Add "Analytics" nav item with BarChart3 icon between "Sessions" and "Exports"
• Team admins see /analytics (team view) as default
• Engineers see /analytics/me as default
• Both can navigate between views if they have permission

---

4. Database Migration

New table: session_ratings

CREATE TABLE session_ratings (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    session_id UUID NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
    user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    tree_id UUID NOT NULL REFERENCES trees(id) ON DELETE CASCADE,
    account_id UUID NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
    rating INTEGER NOT NULL CHECK (rating >= 1 AND rating <= 5),
    comment VARCHAR(500),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE (session_id)
);

New indexes for analytics queries:

CREATE INDEX ix_session_ratings_tree_created ON session_ratings(tree_id, created_at);
CREATE INDEX ix_session_ratings_account_created ON session_ratings(account_id, created_at);
CREATE INDEX ix_sessions_account_completed ON sessions(account_id, completed_at);
CREATE INDEX ix_sessions_account_tree_completed ON sessions(account_id, tree_id, completed_at);
CREATE INDEX ix_step_ratings_step_helpful ON step_ratings(step_id, was_helpful);

Modify step_ratings usage:

• Existing rating (1-5) and review_text columns remain in DB but are no longer populated
• Only was_helpful boolean is used going forward
• Add session_id to unique constraint if not already present: UNIQUE(step_id, user_id, session_id)

---

5. Charting Library

Recharts (recharts npm package)

• React-native, composable components
• Supports: AreaChart (time-series), BarChart (comparisons), PieChart (outcome donut), LineChart (trends)
• Lightweight (~45KB gzipped)
• Dark theme compatible via custom colors matching our design tokens

Chart color palette (matching design system):

• Primary: hsl(243, 75%, 59%) (purple — matches --primary)
• Resolved: #34d399 (emerald-400)
• Escalated: #f87171 (red-400)
• Workaround: #fbbf24 (yellow-400)
• Unresolved: #94a3b8 (slate-400)

---

6. Step Feedback UX Detail

Inline thumbs placement:

• Positioned at the bottom of each step card, right-aligned
• Two icons: ThumbsUp and ThumbsDown from Lucide
• Default state: text-muted-foreground (subtle, not distracting)
• Hover: icon scales slightly, tooltip appears
• Selected up: text-emerald-400 with subtle fill
• Selected down: text-red-400 with subtle fill
• Toggle: clicking selected thumb un-selects it

First-time hint:

• On the first session where thumbs are available, show a subtle inline note below the first step: "New: Rate steps with thumbs to help improve flows"
• Store dismissal in localStorage
• Auto-dismisses after first thumb interaction

CSAT Modal (post-completion):

• Appears after SessionOutcomeModal closes
• Five numbered buttons (1-5) or star icons in a row
• Label: "How would you rate this flow?"
• Sublabel: "Your feedback helps flow authors improve"
• Optional textarea: "Any comments? (optional)"
• Buttons: "Submit" (primary) + "Skip" (text link)
• localStorage tracks which sessions have been rated to prevent re-prompting

---

7. Scope & Non-Goals

In scope:

• Team analytics dashboard with time-series charts
• Personal analytics dashboard
• Flow-level analytics panel
• Step thumbs up/down inline feedback
• Flow CSAT 1-5 + comment at session end
• Period filtering (7d/30d/90d)

Not in scope (future):

• Real-time streaming analytics
• Export analytics to CSV/PDF
• Custom date range picker (just preset periods for v1)
• Comparison mode (this period vs. last period)
• Automation adoption metrics
• Client/ticket correlation analytics
• Email digest/reports