resolutionflow/docs/plans/archive/2026-03-19-phase5-analytics-impl.md

Phase 5: Analytics Enhancement — Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Extend the FlowPilot Analytics page with tabbed sections for coverage heatmap, flow quality scoring, and PSA time tracking metrics.

Architecture: Add two new backend endpoints (/coverage and /flow-quality) alongside the existing /analytics/flowpilot endpoint. Add a psa_activity_log table for time entry tracking. Add flow usage columns to trees. Refactor the frontend analytics page into a tabbed layout with lazy-loaded sections.

Tech Stack: FastAPI, SQLAlchemy 2.0 (async), React 19, TypeScript, Tailwind CSS v4, Recharts

Design doc: docs/plans/2026-03-19-phase5-analytics-enhancement-design.md

---

Task 1: Database — add flow tracking columns + PSA activity log table

Files:

• Modify: backend/app/models/tree.py
• Create: backend/app/models/psa_activity_log.py
• Create: migration file

Step 1: Add flow tracking columns to Tree model

In backend/app/models/tree.py, add after gallery_sort_order:

# Flow quality tracking (Phase 5)
usage_count: Mapped[int] = mapped_column(Integer, default=0)
success_rate: Mapped[Optional[float]] = mapped_column(Float, nullable=True)
last_matched_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)

Import Float from sqlalchemy if not already imported.

Step 2: Create PSA activity log model

Create backend/app/models/psa_activity_log.py:

"""PSA activity log — tracks time entries, note posts, and status updates pushed to PSA."""
import uuid
from datetime import datetime, timezone
from typing import Optional

from sqlalchemy import String, DateTime, ForeignKey, Float
from sqlalchemy.orm import Mapped, mapped_column
from sqlalchemy.dialects.postgresql import UUID

from app.core.database import Base


class PsaActivityLog(Base):
    __tablename__ = "psa_activity_logs"

    id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
    account_id: Mapped[uuid.UUID] = mapped_column(
        UUID(as_uuid=True), ForeignKey("accounts.id", ondelete="CASCADE"), nullable=False, index=True
    )
    session_id: Mapped[Optional[uuid.UUID]] = mapped_column(
        UUID(as_uuid=True), ForeignKey("ai_sessions.id", ondelete="SET NULL"), nullable=True
    )
    activity_type: Mapped[str] = mapped_column(String(50), nullable=False)  # "time_entry_posted", "note_posted", "status_updated"
    hours_logged: Mapped[Optional[float]] = mapped_column(Float, nullable=True)
    psa_ticket_id: Mapped[Optional[str]] = mapped_column(String(100), nullable=True)
    created_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
    )

Step 3: Register model in imports

Ensure the model is importable. Check how other models are registered (e.g., in backend/app/models/__init__.py if it exists, or via alembic's env.py).

Step 4: Generate and run migration

cd /projects/patherly/backend && alembic revision --autogenerate -m "add flow tracking columns and psa_activity_logs table"
alembic upgrade head

Step 5: Commit

git commit -m "feat(analytics): add flow tracking columns and psa_activity_logs table"

---

Task 2: Backend — coverage endpoint

Files:

• Modify: backend/app/api/endpoints/flowpilot_analytics.py
• Modify: backend/app/schemas/flowpilot_analytics.py
• Create: backend/tests/test_analytics_coverage.py

Step 1: Add schemas

In backend/app/schemas/flowpilot_analytics.py, add:

class CoverageDomainRow(BaseModel):
    domain: str
    flow_count: int
    session_count: int
    resolution_rate: float
    escalation_rate: float
    guided_rate: float
    avg_resolution_minutes: float | None = None


class CoverageResponse(BaseModel):
    domains: list[CoverageDomainRow]
    unmapped_session_count: int
    total_domains: int

Step 2: Add endpoint

In backend/app/api/endpoints/flowpilot_analytics.py, add:

@router.get("/coverage", response_model=CoverageResponse)
@limiter.limit("15/minute")
async def get_coverage(
    request: Request,
    current_user: Annotated[User, Depends(get_current_active_user)],
    db: Annotated[AsyncSession, Depends(get_db)],
    _: None = Depends(require_team_admin),
    period: str = Query("30d", pattern="^(7d|30d|90d)$"),
):
    """Coverage heatmap data — per-domain metrics for flow coverage analysis."""
    if not current_user.account_id:
        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="No account")

    account_id = current_user.account_id
    period_start = _get_period_start(period)

    # Get all domains from sessions in period
    domain_stats = await db.execute(
        select(
            AISession.problem_domain,
            func.count(AISession.id).label("session_count"),
            func.sum(case((AISession.status == "resolved", 1), else_=0)).label("resolved"),
            func.sum(case((AISession.status == "escalated", 1), else_=0)).label("escalated"),
            func.sum(case((AISession.confidence_tier == "guided", 1), else_=0)).label("guided"),
        )
        .where(
            AISession.account_id == account_id,
            AISession.created_at >= period_start,
            AISession.problem_domain.isnot(None),
        )
        .group_by(AISession.problem_domain)
    )
    domain_rows = domain_rows_result = domain_stats.all()

    # Count flows per domain via category name matching
    # Trees have category_id → Category.name, sessions have problem_domain
    # Match by comparing Category.name to problem_domain
    from app.models.category import Category
    flow_counts_result = await db.execute(
        select(Category.name, func.count(Tree.id))
        .join(Tree, Tree.category_id == Category.id)
        .where(Tree.account_id == account_id, Tree.is_active == True)
        .group_by(Category.name)
    )
    flow_counts = {row[0]: row[1] for row in flow_counts_result.all()}

    # Count unmapped sessions (no problem_domain)
    unmapped_result = await db.execute(
        select(func.count(AISession.id))
        .where(
            AISession.account_id == account_id,
            AISession.created_at >= period_start,
            AISession.problem_domain.is_(None),
        )
    )
    unmapped_count = unmapped_result.scalar() or 0

    # Build response
    domains = []
    for row in domain_rows:
        total = int(row.session_count)
        resolved = int(row.resolved)
        escalated = int(row.escalated)
        guided = int(row.guided)
        domain_name = row.problem_domain

        domains.append(CoverageDomainRow(
            domain=domain_name,
            flow_count=flow_counts.get(domain_name, 0),
            session_count=total,
            resolution_rate=round(resolved / total, 3) if total else 0,
            escalation_rate=round(escalated / total, 3) if total else 0,
            guided_rate=round(guided / total, 3) if total else 0,
        ))

    # Sort by session count descending
    domains.sort(key=lambda d: d.session_count, reverse=True)

    return CoverageResponse(
        domains=domains,
        unmapped_session_count=unmapped_count,
        total_domains=len(domains),
    )

Step 3: Write tests

Create backend/tests/test_analytics_coverage.py testing:

• Endpoint requires team_admin auth
• Returns domain breakdown with correct counts
• Unmapped sessions counted
• Empty state returns empty list

Step 4: Run tests and verify

cd /projects/patherly/backend && python -m pytest tests/test_analytics_coverage.py -v --override-ini="addopts="

Step 5: Commit

git commit -m "feat(analytics): add coverage heatmap endpoint"

---

Task 3: Backend — flow quality endpoint

Files:

• Modify: backend/app/api/endpoints/flowpilot_analytics.py
• Modify: backend/app/schemas/flowpilot_analytics.py
• Create: backend/tests/test_analytics_flow_quality.py

Step 1: Add schemas

class FlowQualityRow(BaseModel):
    flow_id: str
    name: str
    tree_type: str
    usage_count: int
    success_rate: float | None = None
    last_matched_at: datetime | None = None
    avg_confidence: float | None = None
    quality_score: float


class FlowQualityResponse(BaseModel):
    flows: list[FlowQualityRow]
    top_performers: list[FlowQualityRow]
    needs_attention: list[FlowQualityRow]

Step 2: Add endpoint

@router.get("/flow-quality", response_model=FlowQualityResponse)
@limiter.limit("15/minute")
async def get_flow_quality(
    request: Request,
    current_user: Annotated[User, Depends(get_current_active_user)],
    db: Annotated[AsyncSession, Depends(get_db)],
    _: None = Depends(require_team_admin),
    period: str = Query("30d", pattern="^(7d|30d|90d)$"),
    sort: str = Query("quality", pattern="^(quality|usage|success_rate)$"),
):

Query logic:

• Get all active flows for the account
• For each flow, count sessions where matched_flow_id == flow.id in period
• Calculate success_rate = resolved / total matched
• Calculate quality_score = (success_rate * 0.5) + (guided_rate * 0.3) + (recency_score * 0.2)
• Recency score: 1.0 if used today, decays linearly to 0.0 at 90 days
• Top performers: top 5 by quality_score
• Needs attention: flows with success_rate < 0.5 or not used in 30+ days

Step 3: Write tests, run, commit

git commit -m "feat(analytics): add flow quality scoring endpoint"

---

Task 4: Backend — PSA activity logging + enhanced PSA metrics

Files:

• Modify: backend/app/services/psa/connectwise/provider.py (or wherever note/time entry posting happens)
• Modify: backend/app/api/endpoints/flowpilot_analytics.py
• Modify: backend/app/schemas/flowpilot_analytics.py

Step 1: Add PSA activity logging

Find where the ConnectWise provider posts notes and time entries. After a successful push, log to psa_activity_logs:

from app.models.psa_activity_log import PsaActivityLog

activity = PsaActivityLog(
    account_id=account_id,
    session_id=session_id,
    activity_type="note_posted",  # or "time_entry_posted"
    hours_logged=hours,  # for time entries
    psa_ticket_id=ticket_id,
)
db.add(activity)
await db.commit()

Step 2: Add enhanced PSA schemas

class PsaFunnel(BaseModel):
    total_sessions: int
    linked_to_ticket: int
    doc_pushed: int
    time_entry_logged: int


class PsaDailyTrend(BaseModel):
    date: str
    entries: int
    hours: float


class EnhancedPsaMetrics(BaseModel):
    total_time_entries: int
    total_hours_logged: float
    avg_hours_per_session: float
    push_funnel: PsaFunnel
    daily_trend: list[PsaDailyTrend]

Step 3: Add PSA metrics endpoint

@router.get("/psa-metrics", response_model=EnhancedPsaMetrics)

Query psa_activity_logs and ai_sessions to build the funnel and trend data.

Step 4: Write tests, run, commit

git commit -m "feat(analytics): add PSA activity logging and enhanced PSA metrics endpoint"

---

Task 5: Backend — wire flow matching stats

Files:

• Modify: backend/app/services/flowpilot_engine.py (or wherever flow matching happens)

Step 1: Update flow stats on match

Find where matched_flow_id is set on an AISession. At that point, also update the matched flow:

# When a flow is matched to a session:
flow.usage_count = (flow.usage_count or 0) + 1
flow.last_matched_at = datetime.now(timezone.utc)

Step 2: Update success_rate on resolution

When a session resolves and has a matched_flow_id, recalculate that flow's success_rate:

# After session resolves:
if session.matched_flow_id:
    total = await db.execute(
        select(func.count(AISession.id))
        .where(AISession.matched_flow_id == session.matched_flow_id)
    )
    resolved = await db.execute(
        select(func.count(AISession.id))
        .where(AISession.matched_flow_id == session.matched_flow_id, AISession.status == "resolved")
    )
    flow.success_rate = round(resolved.scalar() / total.scalar(), 3) if total.scalar() else None

Step 3: Commit

git commit -m "feat(analytics): wire flow usage tracking into session matching and resolution"

---

Task 6: Frontend — types and API client updates

Files:

• Modify: frontend/src/types/flowpilot-analytics.ts
• Modify: frontend/src/api/flowpilotAnalytics.ts

Step 1: Add types

// Coverage
export interface CoverageDomainRow {
  domain: string
  flow_count: number
  session_count: number
  resolution_rate: number
  escalation_rate: number
  guided_rate: number
  avg_resolution_minutes: number | null
}

export interface CoverageResponse {
  domains: CoverageDomainRow[]
  unmapped_session_count: number
  total_domains: number
}

// Flow Quality
export interface FlowQualityRow {
  flow_id: string
  name: string
  tree_type: string
  usage_count: number
  success_rate: number | null
  last_matched_at: string | null
  avg_confidence: number | null
  quality_score: number
}

export interface FlowQualityResponse {
  flows: FlowQualityRow[]
  top_performers: FlowQualityRow[]
  needs_attention: FlowQualityRow[]
}

// Enhanced PSA
export interface PsaFunnel {
  total_sessions: number
  linked_to_ticket: number
  doc_pushed: number
  time_entry_logged: number
}

export interface PsaDailyTrend {
  date: string
  entries: number
  hours: number
}

export interface EnhancedPsaMetrics {
  total_time_entries: number
  total_hours_logged: number
  avg_hours_per_session: number
  push_funnel: PsaFunnel
  daily_trend: PsaDailyTrend[]
}

Step 2: Add API methods

async getCoverage(period: string = '30d'): Promise<CoverageResponse> {
  const response = await apiClient.get<CoverageResponse>('/analytics/flowpilot/coverage', { params: { period } })
  return response.data
},

async getFlowQuality(period: string = '30d', sort: string = 'quality'): Promise<FlowQualityResponse> {
  const response = await apiClient.get<FlowQualityResponse>('/analytics/flowpilot/flow-quality', { params: { period, sort } })
  return response.data
},

async getPsaMetrics(period: string = '30d'): Promise<EnhancedPsaMetrics> {
  const response = await apiClient.get<EnhancedPsaMetrics>('/analytics/flowpilot/psa-metrics', { params: { period } })
  return response.data
},

Step 3: Run build, commit

cd /projects/patherly/frontend && npm run build
git commit -m "feat(analytics): add coverage, flow quality, and PSA metrics types and API client"

---

Task 7: Frontend — tabbed layout + Coverage heatmap tab

Files:

• Modify: frontend/src/pages/FlowPilotAnalyticsPage.tsx
• Create: frontend/src/components/analytics/CoverageHeatmap.tsx

Step 1: Refactor page into tabs

Add tab state and tab bar to FlowPilotAnalyticsPage.tsx:

• Tabs: "Overview", "Coverage", "Flow Quality", "PSA"
• Active tab: bg-primary/10 text-foreground border-b-2 border-primary
• Inactive: text-muted-foreground hover:text-foreground
• Move existing dashboard content into the Overview tab
• Each non-overview tab fetches its data lazily on first selection

Step 2: Build CoverageHeatmap component

frontend/src/components/analytics/CoverageHeatmap.tsx:

• .glass-card-static table container
• Table headers: Domain, Flows, Sessions, Resolution %, Escalation %, Guided %
• Cell coloring functions:
  • Resolution: bg-emerald-400/10 text-emerald-400 (>75%), bg-amber-400/10 text-amber-400 (50-75%), bg-rose-500/10 text-rose-500 (<50%)
  • Escalation: green (<10%), amber (10-25%), red (>25%)
  • Guided: green (>60%), amber (30-60%), red (<30%)
  • Flows: green (5+), amber (1-4), red (0)
• Domains with 0 flows show "Create Flow" link
• Responsive: horizontal scroll on mobile (overflow-x-auto)

Step 3: Run build, commit

git commit -m "feat(analytics): add tabbed layout and coverage heatmap"

---

Task 8: Frontend — Flow Quality tab

Files:

• Create: frontend/src/components/analytics/FlowQualityTable.tsx
• Modify: frontend/src/pages/FlowPilotAnalyticsPage.tsx

Step 1: Build FlowQualityTable component

• .glass-card-static sortable table
• Columns: Flow Name (link to editor), Usage, Success Rate, Last Used, Avg Confidence, Quality Score
• Column headers clickable to sort
• Top 5 rows: left border border-l-2 border-emerald-400
• Bottom 5 rows: left border border-l-2 border-rose-500
• "Needs attention" badge (bg-amber-400/10 text-amber-400 font-label text-[0.625rem]) on flows with success_rate < 50% or unused 30+ days
• Quality score displayed as a colored bar (0-100% width, emerald/amber/rose)
• Click flow name → navigate to /trees/{id}/edit

Step 2: Wire into tab, run build, commit

git commit -m "feat(analytics): add flow quality scoring table"

---

Task 9: Frontend — PSA metrics tab

Files:

• Create: frontend/src/components/analytics/PsaMetricsPanel.tsx
• Modify: frontend/src/pages/FlowPilotAnalyticsPage.tsx

Step 1: Build PsaMetricsPanel component

• Metric cards row (3 cards): Total Time Entries, Total Hours Logged, Avg Hours/Session
• Push success funnel: horizontal bar visualization showing conversion at each step (sessions → linked → pushed → time entry). Show counts + percentage between steps.
• Trend chart: Recharts AreaChart with dual Y-axes — entries (bars) and hours (area) over the period

Step 2: Wire into tab, run build, commit

git commit -m "feat(analytics): add PSA metrics panel with funnel and trend chart"

---

Task 10: Final verification and docs

Step 1: Run full backend tests

cd /projects/patherly/backend && python -m pytest --override-ini="addopts="

Step 2: Run frontend build

cd /projects/patherly/frontend && npm run build

Step 3: Update CURRENT-STATE.md

Mark Phase 5 as complete. Update What's Next.

Step 4: Commit

git commit -m "docs: update CURRENT-STATE.md — Phase 5 Analytics Enhancement complete"