resolutionflow/docs/plans/archive/plan-improvements-for-analytics.md

# Post-Implementation Improvements — Analytics & Feedback

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Context:** The analytics & feedback feature has just been implemented from `docs/plans/2026-02-15-analytics-feedback-implementation.md`. These are five targeted improvements identified from a comparative review. Apply them to the existing code on the current branch (`feat/analytics-feedback`).

**Reference files you'll need to modify:**
- `backend/app/api/endpoints/analytics.py` — analytics query logic
- `backend/app/schemas/analytics.py` — response schemas
- `backend/app/api/endpoints/ratings.py` — rating endpoints
- `backend/app/api/endpoints/steps.py` — existing step rating routes
- `frontend/src/types/analytics.ts` — TypeScript types
- `frontend/src/pages/TeamAnalyticsPage.tsx` — team dashboard

---

## Task 1: Switch from Average to Median Duration

**Why:** One 3-hour debugging session shouldn't skew stats for a flow that normally takes 8 minutes. Median is the standard for time-based metrics in analytics.

**Backend changes:**

In `backend/app/schemas/analytics.py`:
- Rename `avg_duration_minutes` → `median_duration_minutes` in `AnalyticsSummary`, `TopFlow`, and `TopEngineer`

In `backend/app/api/endpoints/analytics.py`:
- Replace `func.avg(...)` duration calculations with a median approach
- PostgreSQL supports `percentile_cont(0.5) WITHIN GROUP (ORDER BY ...)` for median
- Example replacement for the `_build_summary` helper:

```python
from sqlalchemy import text

# Replace the avg duration query with:
duration_q = await db.execute(
    select(
        func.percentile_cont(0.5).within_group(
            func.extract('epoch', Session.completed_at - Session.started_at) / 60
        )
    ).where(*base_filter, Session.completed_at.isnot(None))
)
median_duration = round(float(duration_q.scalar() or 0), 1)
```

- Apply the same change to the top_flows and top_engineers subqueries (replace `func.avg` with `percentile_cont(0.5)` for duration)
- Update all references from `avg_duration` to `median_duration` in the response construction

**Frontend changes:**

In `frontend/src/types/analytics.ts`:
- Rename `avg_duration_minutes` → `median_duration_minutes` in `TopFlow`, `TopEngineer`, and `AnalyticsSummary`

In `frontend/src/pages/TeamAnalyticsPage.tsx` and `frontend/src/pages/MyAnalyticsPage.tsx`:
- Update stat card label from "Avg Duration" → "Median Duration"
- Update any references to `avg_duration_minutes` → `median_duration_minutes`

In `frontend/src/components/analytics/FlowAnalyticsPanel.tsx`:
- Same rename for the flow summary stat card

**Verify:** `cd backend && python -m pytest --override-ini="addopts=" -v` and `cd frontend && npm run build`

**Commit:** `git commit -am "refactor: use median instead of average for duration metrics"`

---

## Task 2: Add Step Dropoff Metrics to Flow Analytics

**Why:** Dropoff data is passive (requires no user action) and tells flow authors exactly where engineers get stuck and abandon sessions. This is the most actionable metric for flow improvement.

**Backend changes:**

In `backend/app/schemas/analytics.py`:
- Add fields to `StepFeedbackSummary`:

```python
class StepFeedbackSummary(BaseModel):
    node_id: str
    node_title: str
    helpful_yes: int
    helpful_no: int
    helpful_rate: float
    visit_count: int = 0        # NEW
    dropoff_count: int = 0      # NEW
    dropoff_rate: float = 0.0   # NEW
```

In `backend/app/api/endpoints/analytics.py`, in the `get_flow_analytics` function:
- After the existing step_feedback section, add dropoff calculation logic:

```python
# Step dropoff analysis — build from session decisions JSONB
# For each session in the period for this tree:
#   - Extract all node_ids from the decisions array
#   - The LAST node_id in an incomplete session = dropoff point
#   - Count visits per node and dropoffs per node

from sqlalchemy.dialects.postgresql import JSONB

# Get all sessions for this tree in period
sessions_q = await db.execute(
    select(Session.id, Session.decisions, Session.completed_at)
    .where(Session.tree_id == tree_id, Session.started_at >= period_start)
)
sessions_data = sessions_q.all()

# Build node visit and dropoff maps
node_visits: dict[str, int] = {}
node_dropoffs: dict[str, int] = {}

for sess in sessions_data:
    decisions = sess.decisions or []
    for decision in decisions:
        node_id = decision.get("node_id") or decision.get("nodeId", "")
        if node_id:
            node_visits[node_id] = node_visits.get(node_id, 0) + 1
    
    # If session not completed, last decision node is a dropoff
    if not sess.completed_at and decisions:
        last_decision = decisions[-1]
        last_node = last_decision.get("node_id") or last_decision.get("nodeId", "")
        if last_node:
            node_dropoffs[last_node] = node_dropoffs.get(last_node, 0) + 1

# Merge dropoff data into step_feedback list
# Get node titles from the tree's content JSONB
tree_result = await db.execute(select(Tree.content).where(Tree.id == tree_id))
tree_content = tree_result.scalar_one_or_none() or {}
nodes = tree_content.get("nodes", [])
node_title_map = {n.get("id", ""): n.get("title", n.get("label", "Unnamed")) for n in nodes}

# Build combined step feedback with visits + dropoffs + thumbs
all_node_ids = set(list(node_visits.keys()) + [sf.node_id for sf in step_feedback])
combined_feedback = []
for nid in all_node_ids:
    visits = node_visits.get(nid, 0)
    dropoffs = node_dropoffs.get(nid, 0)
    # Find existing thumbs data if any
    existing = next((sf for sf in step_feedback if sf.node_id == nid), None)
    combined_feedback.append(StepFeedbackSummary(
        node_id=nid,
        node_title=node_title_map.get(nid, "Unknown Step"),
        helpful_yes=existing.helpful_yes if existing else 0,
        helpful_no=existing.helpful_no if existing else 0,
        helpful_rate=existing.helpful_rate if existing else 0.0,
        visit_count=visits,
        dropoff_count=dropoffs,
        dropoff_rate=round(dropoffs / visits, 3) if visits > 0 else 0.0,
    ))

# Sort by dropoff_rate descending so worst steps are first
combined_feedback.sort(key=lambda x: x.dropoff_rate, reverse=True)
step_feedback = combined_feedback
```

**Frontend changes:**

In `frontend/src/types/analytics.ts`:
- Add to `StepFeedbackSummary`:

```typescript
export interface StepFeedbackSummary {
  node_id: string
  node_title: string
  helpful_yes: number
  helpful_no: number
  helpful_rate: number
  visit_count: number      // NEW
  dropoff_count: number    // NEW
  dropoff_rate: number     // NEW
}
```

In `frontend/src/components/analytics/FlowAnalyticsPanel.tsx`:
- Add dropoff columns to the step feedback table: "Visits", "Dropoffs", "Dropoff Rate"
- Highlight rows where `dropoff_rate > 0.2` with a subtle red/warning background

**Verify:** `cd backend && python -m pytest --override-ini="addopts=" -v` and `cd frontend && npm run build`

**Commit:** `git commit -am "feat: add step dropoff metrics to flow analytics"`

---

## Task 3: Add Backward-Compatible /ratings Alias Routes

**Why:** The existing backend uses `/{step_id}/rate` but the frontend uses `/{step_id}/ratings`. This is a real inconsistency that needs fixing.

**Backend changes:**

In `backend/app/api/endpoints/steps.py`:
- Find the existing routes: `POST /{step_id}/rate`, `PUT /{step_id}/rate`, `DELETE /{step_id}/rate`
- Add alias routes that point to the same handler functions:

```python
# After the existing rate_step function:
@router.post("/{step_id}/ratings", response_model=StepRatingResponse, status_code=status.HTTP_201_CREATED,
             include_in_schema=False)
async def rate_step_alias(
    step_id: UUID,
    rating_data: StepRatingCreate,
    db: AsyncSession = Depends(get_db),
    current_user: User = Depends(get_current_user)
):
    """Alias for POST /{step_id}/rate — backward compatibility."""
    return await rate_step(step_id, rating_data, db, current_user)


@router.put("/{step_id}/ratings", response_model=StepRatingResponse,
            include_in_schema=False)
async def update_rating_alias(
    step_id: UUID,
    rating_data: StepRatingUpdate,
    db: AsyncSession = Depends(get_db),
    current_user: User = Depends(get_current_user)
):
    """Alias for PUT /{step_id}/rate — backward compatibility."""
    return await update_rating(step_id, rating_data, db, current_user)


@router.delete("/{step_id}/ratings", status_code=204,
               include_in_schema=False)
async def delete_rating_alias(
    step_id: UUID,
    db: AsyncSession = Depends(get_db),
    current_user: User = Depends(get_current_user)
):
    """Alias for DELETE /{step_id}/rate — backward compatibility."""
    return await delete_rating(step_id, db, current_user)
```

Note: Use `include_in_schema=False` to keep the OpenAPI docs clean — only the canonical `/rate` routes appear in docs.

**Verify:** `cd backend && python -m pytest --override-ini="addopts=" -v`

**Commit:** `git commit -am "fix: add /ratings alias routes for backward compatibility"`

---

## Task 4: Anonymize Feedback Comments in Analytics

**Why:** In a team tool where managers see feedback, engineers will give more honest feedback if their identity isn't attached. Anonymous feedback = better signal.

**Backend changes:**

In `backend/app/schemas/analytics.py`:
- Remove `user_name` from `FlowRatingItem`:

```python
class FlowRatingItem(BaseModel):
    rating: int
    comment: Optional[str]
    created_at: datetime
    # user_name removed — feedback is anonymous in analytics views
```

In `backend/app/api/endpoints/analytics.py`, in `get_flow_analytics`:
- Remove the `User.name` join from the recent_comments query:

```python
# Recent comments — anonymous
comments_q = await db.execute(
    select(SessionRating.rating, SessionRating.comment, SessionRating.created_at)
    .where(
        SessionRating.tree_id == tree_id,
        SessionRating.comment.isnot(None),
        SessionRating.comment != "",
    )
    .order_by(SessionRating.created_at.desc())
    .limit(10)
)
recent_comments = [
    FlowRatingItem(
        rating=row.rating,
        comment=row.comment,
        created_at=row.created_at,
    )
    for row in comments_q.all()
]
```

**Frontend changes:**

In `frontend/src/types/analytics.ts`:
- Remove `user_name` from `FlowRatingItem`:

```typescript
export interface FlowRatingItem {
  rating: number
  comment?: string
  created_at: string
  // user_name removed — anonymous feedback
}
```

In `frontend/src/components/analytics/FlowAnalyticsPanel.tsx`:
- Remove any rendering of `user_name` in the recent comments list
- Comments should show: star rating + comment text + relative timestamp only

**Verify:** `cd backend && python -m pytest --override-ini="addopts=" -v` and `cd frontend && npm run build`

**Commit:** `git commit -am "privacy: anonymize feedback comments in analytics views"`

---

## Task 5: Add Explicit active_engineers Metric

**Why:** "How many of my techs actually used this tool this month" is a key adoption metric for MSP managers.

**Backend changes:**

In `backend/app/schemas/analytics.py`:
- Verify `AnalyticsSummary` already has an `active_engineers` field. If not, add:

```python
class AnalyticsSummary(BaseModel):
    total_sessions: int
    completed_sessions: int
    completion_rate: float
    median_duration_minutes: float
    active_engineers: int = 0  # ADD if missing
    outcome_breakdown: OutcomeBreakdown
```

In `backend/app/api/endpoints/analytics.py`:
- In the `_build_summary` helper (or the team analytics endpoint), add the active engineers count:

```python
# Active engineers: distinct users with >= 1 session in window
active_q = await db.execute(
    select(func.count(func.distinct(Session.user_id)))
    .where(*base_filter)
)
active_engineers = active_q.scalar() or 0
```

- Include `active_engineers=active_engineers` in the `AnalyticsSummary(...)` construction
- For personal analytics (`/me`), set `active_engineers=1` (it's always just the requesting user)

**Frontend changes:**

In `frontend/src/types/analytics.ts`:
- Verify `AnalyticsSummary` has `active_engineers: number`. Add if missing.

In `frontend/src/pages/TeamAnalyticsPage.tsx`:
- Verify there's a stat card for "Active Engineers". If missing, add one using the `summary.active_engineers` value.

**Verify:** `cd backend && python -m pytest --override-ini="addopts=" -v` and `cd frontend && npm run build`

**Commit:** `git commit -am "feat: add explicit active_engineers metric to team analytics"`

---

## Final: Full Test Suite + Build Verification

```bash
cd backend && python -m pytest --override-ini="addopts=" -v
cd frontend && npm run build
```

Fix any failures, then:

```bash
git add -A
git commit -m "fix: post-improvement test and build fixes"
```