* docs: add analytics & user feedback design document Covers team analytics, personal analytics, flow analytics, step-level thumbs up/down feedback, and flow CSAT ratings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add analytics & feedback implementation plan 12-task TDD plan covering session ratings, step feedback, team/personal/flow analytics endpoints, and frontend pages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add session_ratings table and analytics indexes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add SessionRating model and analytics schemas Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add session CSAT rating endpoint with tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add step thumbs feedback and /ratings alias routes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add team, personal, and flow analytics endpoints Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add recharts, analytics types, and API client Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add inline step thumbs up/down feedback during sessions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add CSAT rating modal after session completion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add Team Analytics page with charts and leaderboards Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add Flow Analytics panel with step dropoff and CSAT data Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add My Analytics page with personal stats and charts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
50 KiB
Analytics & User Feedback — Implementation Plan
For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
Goal: Add team analytics, personal analytics, flow analytics dashboards and a two-tier feedback system (step thumbs up/down + flow CSAT 1-5) to ResolutionFlow.
Architecture: Live aggregation queries against existing PostgreSQL tables plus one new session_ratings table. Four new API endpoint groups (/analytics/team, /analytics/me, /analytics/flows/{id}, /sessions/{id}/rate). Recharts for time-series visualization. Step feedback reuses existing step_ratings table with simplified thumbs-only input.
Tech Stack: Python FastAPI, SQLAlchemy 2.0 async, Alembic, React 19, TypeScript, Recharts, Tailwind CSS v3
Reference: Design Document
See docs/plans/2026-02-15-analytics-feedback-design.md for full design rationale and endpoint response shapes.
Reference: Existing Code
- Step rating endpoints already exist:
backend/app/api/endpoints/steps.py:427-618(POST/PUT/DELETE/steps/{id}/rate+ aggregation helper) - Session model:
backend/app/models/session.py— hasoutcome,completed_at,started_at,tree_id,user_idcolumns - Tree model:
backend/app/models/tree.py— hasusage_count,account_id - Session does NOT have
account_id— must join throughtrees.account_idorusers.account_id - Latest migration:
038_remove_workspace_system.py - Route registration:
backend/app/api/router.py— import module +api_router.include_router() - Frontend routes:
frontend/src/router.tsx— lazy imports + Suspense wrappers - Sidebar nav:
frontend/src/components/layout/Sidebar.tsx:119-153 - Permissions:
frontend/src/hooks/usePermissions.ts—isSuperAdmin,isAccountOwner,effectiveRole
Task 1: Database Migration
Files:
- Create:
backend/alembic/versions/039_add_session_ratings_and_analytics_indexes.py
Step 1: Generate migration
cd backend
Create migration file manually (autogenerate won't pick up indexes properly):
"""Add session_ratings table and analytics indexes
Revision ID: 039
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import UUID
revision = '039_add_session_ratings_and_analytics_indexes'
down_revision = '038_remove_workspace_system'
def upgrade():
# New session_ratings table
op.create_table(
'session_ratings',
sa.Column('id', UUID(as_uuid=True), server_default=sa.text('gen_random_uuid()'), primary_key=True),
sa.Column('session_id', UUID(as_uuid=True), sa.ForeignKey('sessions.id', ondelete='CASCADE'), nullable=False),
sa.Column('user_id', UUID(as_uuid=True), sa.ForeignKey('users.id', ondelete='CASCADE'), nullable=False),
sa.Column('tree_id', UUID(as_uuid=True), sa.ForeignKey('trees.id', ondelete='CASCADE'), nullable=False),
sa.Column('account_id', UUID(as_uuid=True), sa.ForeignKey('accounts.id', ondelete='CASCADE'), nullable=False),
sa.Column('rating', sa.Integer, nullable=False),
sa.Column('comment', sa.String(500), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('NOW()'), nullable=False),
sa.CheckConstraint('rating >= 1 AND rating <= 5', name='ck_session_ratings_rating_range'),
sa.UniqueConstraint('session_id', name='uq_session_ratings_session_id'),
)
# Analytics indexes
op.create_index('ix_session_ratings_tree_created', 'session_ratings', ['tree_id', 'created_at'])
op.create_index('ix_session_ratings_account_created', 'session_ratings', ['account_id', 'created_at'])
op.create_index('ix_sessions_completed', 'sessions', ['completed_at'])
op.create_index('ix_step_ratings_step_helpful', 'step_ratings', ['step_id', 'was_helpful'])
# Make step_ratings.rating column nullable (thumbs-only mode)
op.alter_column('step_ratings', 'rating', nullable=True)
# Drop old unique constraint on step_ratings (step_id, user_id) and replace with (step_id, user_id, session_id)
op.drop_constraint('uq_step_rating_per_user', 'step_ratings', type_='unique')
op.create_unique_constraint('uq_step_rating_per_user_session', 'step_ratings', ['step_id', 'user_id', 'session_id'])
def downgrade():
op.drop_constraint('uq_step_rating_per_user_session', 'step_ratings', type_='unique')
op.create_unique_constraint('uq_step_rating_per_user', 'step_ratings', ['step_id', 'user_id'])
op.alter_column('step_ratings', 'rating', nullable=False)
op.drop_index('ix_step_ratings_step_helpful', 'step_ratings')
op.drop_index('ix_sessions_completed', 'sessions')
op.drop_index('ix_session_ratings_account_created', 'session_ratings')
op.drop_index('ix_session_ratings_tree_created', 'session_ratings')
op.drop_table('session_ratings')
Step 2: Run the migration
cd backend && alembic upgrade head
Expected: Migration applies successfully with no errors.
Step 3: Verify table exists
docker exec -it patherly_postgres psql -U postgres -d patherly -c "\d session_ratings"
Expected: Table with columns id, session_id, user_id, tree_id, account_id, rating, comment, created_at.
Step 4: Commit
git add backend/alembic/versions/039_*
git commit -m "feat: add session_ratings table and analytics indexes"
Task 2: SessionRating Model + Schemas
Files:
- Create:
backend/app/models/session_rating.py - Modify:
backend/app/models/__init__.py - Create:
backend/app/schemas/analytics.py - Modify:
backend/app/schemas/__init__.py(if it exists as a barrel export)
Step 1: Create the SessionRating model
Create backend/app/models/session_rating.py:
import uuid
from datetime import datetime, timezone
from sqlalchemy import Column, DateTime, ForeignKey, Integer, String, CheckConstraint, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import relationship
from app.core.database import Base
class SessionRating(Base):
__tablename__ = "session_ratings"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
session_id = Column(UUID(as_uuid=True), ForeignKey("sessions.id", ondelete="CASCADE"), nullable=False)
user_id = Column(UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False)
tree_id = Column(UUID(as_uuid=True), ForeignKey("trees.id", ondelete="CASCADE"), nullable=False)
account_id = Column(UUID(as_uuid=True), ForeignKey("accounts.id", ondelete="CASCADE"), nullable=False)
rating = Column(Integer, nullable=False)
comment = Column(String(500), nullable=True)
created_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), nullable=False)
__table_args__ = (
CheckConstraint("rating >= 1 AND rating <= 5", name="ck_session_ratings_rating_range"),
UniqueConstraint("session_id", name="uq_session_ratings_session_id"),
)
# Relationships
session = relationship("Session", foreign_keys=[session_id])
user = relationship("User", foreign_keys=[user_id])
tree = relationship("Tree", foreign_keys=[tree_id])
Step 2: Register in models/init.py
Add to backend/app/models/__init__.py:
from .session_rating import SessionRating
And add "SessionRating" to the __all__ list.
Step 3: Create analytics schemas
Create backend/app/schemas/analytics.py:
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field
# --- Session Rating Schemas ---
class SessionRatingCreate(BaseModel):
rating: int = Field(..., ge=1, le=5)
comment: Optional[str] = Field(None, max_length=500)
class SessionRatingResponse(BaseModel):
id: str
session_id: str
rating: int
comment: Optional[str]
created_at: datetime
model_config = {"from_attributes": True}
class FlowRatingItem(BaseModel):
rating: int
comment: Optional[str]
user_name: Optional[str]
created_at: datetime
# --- Step Feedback Schema ---
class StepFeedbackCreate(BaseModel):
session_id: str
was_helpful: bool
# --- Analytics Response Schemas ---
class OutcomeBreakdown(BaseModel):
resolved: int = 0
escalated: int = 0
workaround: int = 0
unresolved: int = 0
class AnalyticsSummary(BaseModel):
total_sessions: int
completed_sessions: int
completion_rate: float
avg_duration_minutes: float
outcome_breakdown: OutcomeBreakdown
class TimeSeriesPoint(BaseModel):
date: str
sessions: int = 0
resolved: int = 0
escalated: int = 0
workaround: int = 0
unresolved: int = 0
class TopFlow(BaseModel):
tree_id: str
name: str
sessions: int
completion_rate: float
avg_duration_minutes: float
avg_csat: Optional[float] = None
class TopEngineer(BaseModel):
user_id: str
name: str
sessions: int
completion_rate: float
avg_duration_minutes: float
class TeamAnalyticsResponse(BaseModel):
summary: AnalyticsSummary
time_series: list[TimeSeriesPoint]
top_flows: list[TopFlow]
top_engineers: list[TopEngineer]
class PersonalAnalyticsResponse(BaseModel):
summary: AnalyticsSummary
time_series: list[TimeSeriesPoint]
top_flows: list[TopFlow]
class StepFeedbackSummary(BaseModel):
node_id: str
node_title: str
helpful_yes: int
helpful_no: int
helpful_rate: float
class FlowAnalyticsResponse(BaseModel):
summary: AnalyticsSummary
avg_csat: Optional[float]
total_ratings: int
time_series: list[TimeSeriesPoint]
step_feedback: list[StepFeedbackSummary]
recent_comments: list[FlowRatingItem]
Step 4: Verify import works
cd backend && python -c "from app.models.session_rating import SessionRating; print('OK')"
Step 5: Commit
git add backend/app/models/session_rating.py backend/app/models/__init__.py backend/app/schemas/analytics.py
git commit -m "feat: add SessionRating model and analytics schemas"
Task 3: Session Rating Endpoint
Files:
- Create:
backend/app/api/endpoints/ratings.py - Modify:
backend/app/api/router.py - Create:
backend/tests/test_ratings.py
Step 1: Write tests
Create backend/tests/test_ratings.py:
import pytest
from httpx import AsyncClient
pytestmark = pytest.mark.asyncio
async def test_rate_session_success(client: AsyncClient, auth_headers: dict, completed_session_id: str):
"""Rate a completed session with CSAT score."""
response = await client.post(
f"/api/v1/sessions/{completed_session_id}/rate",
json={"rating": 4, "comment": "Very helpful flow"},
headers=auth_headers,
)
assert response.status_code == 201
data = response.json()
assert data["rating"] == 4
assert data["comment"] == "Very helpful flow"
async def test_rate_session_duplicate(client: AsyncClient, auth_headers: dict, completed_session_id: str):
"""Cannot rate same session twice."""
await client.post(
f"/api/v1/sessions/{completed_session_id}/rate",
json={"rating": 4},
headers=auth_headers,
)
response = await client.post(
f"/api/v1/sessions/{completed_session_id}/rate",
json={"rating": 5},
headers=auth_headers,
)
assert response.status_code == 409
async def test_rate_session_invalid_rating(client: AsyncClient, auth_headers: dict, completed_session_id: str):
"""Rating must be 1-5."""
response = await client.post(
f"/api/v1/sessions/{completed_session_id}/rate",
json={"rating": 6},
headers=auth_headers,
)
assert response.status_code == 422
async def test_rate_incomplete_session(client: AsyncClient, auth_headers: dict, active_session_id: str):
"""Cannot rate a session that hasn't been completed."""
response = await client.post(
f"/api/v1/sessions/{active_session_id}/rate",
json={"rating": 4},
headers=auth_headers,
)
assert response.status_code == 400
Step 2: Run tests to verify they fail
cd backend && python -m pytest tests/test_ratings.py -v --override-ini="addopts="
Expected: FAIL (endpoint doesn't exist yet).
Step 3: Implement the endpoint
Create backend/app/api/endpoints/ratings.py:
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.database import get_db
from app.api.deps import get_current_active_user
from app.models import User, Session, SessionRating
from app.schemas.analytics import SessionRatingCreate, SessionRatingResponse
router = APIRouter(tags=["ratings"])
@router.post("/sessions/{session_id}/rate", response_model=SessionRatingResponse, status_code=status.HTTP_201_CREATED)
async def rate_session(
session_id: UUID,
data: SessionRatingCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_active_user),
):
"""Submit a CSAT rating (1-5) for a completed session."""
# Verify session exists and belongs to user
result = await db.execute(select(Session).where(Session.id == session_id))
session = result.scalar_one_or_none()
if not session:
raise HTTPException(status_code=404, detail="Session not found")
if session.user_id != current_user.id:
raise HTTPException(status_code=403, detail="Not your session")
if not session.completed_at:
raise HTTPException(status_code=400, detail="Session not completed yet")
# Check for duplicate
existing = await db.execute(
select(SessionRating).where(SessionRating.session_id == session_id)
)
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Session already rated")
rating = SessionRating(
session_id=session_id,
user_id=current_user.id,
tree_id=session.tree_id,
account_id=current_user.account_id,
rating=data.rating,
comment=data.comment,
)
db.add(rating)
await db.commit()
await db.refresh(rating)
return SessionRatingResponse(
id=str(rating.id),
session_id=str(rating.session_id),
rating=rating.rating,
comment=rating.comment,
created_at=rating.created_at,
)
Step 4: Register the route
In backend/app/api/router.py, add:
from app.api.endpoints import ratings
And:
api_router.include_router(ratings.router)
Step 5: Run tests to verify they pass
cd backend && python -m pytest tests/test_ratings.py -v --override-ini="addopts="
Note: Tests may need fixture adjustments (completed_session_id, active_session_id). Create appropriate fixtures in conftest.py if they don't exist — a completed session is one where completed_at is set and outcome is populated.
Step 6: Commit
git add backend/app/api/endpoints/ratings.py backend/app/api/router.py backend/tests/test_ratings.py
git commit -m "feat: add session CSAT rating endpoint"
Task 4: Step Feedback Endpoint
Files:
- Modify:
backend/app/api/endpoints/ratings.py - Add tests to:
backend/tests/test_ratings.py
Step 1: Add step feedback tests
Append to backend/tests/test_ratings.py:
async def test_step_feedback_thumbs_up(client: AsyncClient, auth_headers: dict, step_id: str, session_id: str):
"""Submit thumbs-up feedback for a step."""
response = await client.post(
f"/api/v1/steps/{step_id}/feedback",
json={"session_id": session_id, "was_helpful": True},
headers=auth_headers,
)
assert response.status_code == 201
assert response.json()["was_helpful"] is True
async def test_step_feedback_toggle(client: AsyncClient, auth_headers: dict, step_id: str, session_id: str):
"""Submitting again for same step+session updates the rating."""
await client.post(
f"/api/v1/steps/{step_id}/feedback",
json={"session_id": session_id, "was_helpful": True},
headers=auth_headers,
)
response = await client.post(
f"/api/v1/steps/{step_id}/feedback",
json={"session_id": session_id, "was_helpful": False},
headers=auth_headers,
)
assert response.status_code == 200
assert response.json()["was_helpful"] is False
Step 2: Implement the endpoint
Add to backend/app/api/endpoints/ratings.py:
from app.models import StepLibrary, StepRating
from app.schemas.analytics import StepFeedbackCreate
@router.post("/steps/{step_id}/feedback", status_code=status.HTTP_201_CREATED)
async def submit_step_feedback(
step_id: UUID,
data: StepFeedbackCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_active_user),
):
"""Submit thumbs up/down feedback for a step used in a session."""
# Verify step exists
result = await db.execute(select(StepLibrary).where(StepLibrary.id == step_id))
step = result.scalar_one_or_none()
if not step:
raise HTTPException(status_code=404, detail="Step not found")
session_uuid = UUID(data.session_id)
# Check for existing feedback for this step+user+session
existing_result = await db.execute(
select(StepRating).where(
StepRating.step_id == step_id,
StepRating.user_id == current_user.id,
StepRating.session_id == session_uuid,
)
)
existing = existing_result.scalar_one_or_none()
if existing:
existing.was_helpful = data.was_helpful
status_code = 200
else:
rating = StepRating(
step_id=step_id,
user_id=current_user.id,
session_id=session_uuid,
was_helpful=data.was_helpful,
)
db.add(rating)
status_code = 201
# Update aggregates on step_library
await _update_step_helpful_counts(db, step_id)
await db.commit()
return {"step_id": str(step_id), "was_helpful": data.was_helpful, "status": "created" if status_code == 201 else "updated"}
async def _update_step_helpful_counts(db: AsyncSession, step_id: UUID):
"""Recalculate helpful_yes and helpful_no on step_library."""
from sqlalchemy import func
yes_count = await db.execute(
select(func.count()).where(StepRating.step_id == step_id, StepRating.was_helpful == True)
)
no_count = await db.execute(
select(func.count()).where(StepRating.step_id == step_id, StepRating.was_helpful == False)
)
await db.execute(
StepLibrary.__table__.update()
.where(StepLibrary.id == step_id)
.values(helpful_yes=yes_count.scalar(), helpful_no=no_count.scalar())
)
Step 3: Run tests
cd backend && python -m pytest tests/test_ratings.py -v --override-ini="addopts="
Step 4: Commit
git add backend/app/api/endpoints/ratings.py backend/tests/test_ratings.py
git commit -m "feat: add step thumbs up/down feedback endpoint"
Task 5: Team Analytics Endpoint
Files:
- Create:
backend/app/api/endpoints/analytics.py - Modify:
backend/app/api/router.py - Create:
backend/tests/test_analytics.py
Step 1: Write tests
Create backend/tests/test_analytics.py:
import pytest
from httpx import AsyncClient
pytestmark = pytest.mark.asyncio
async def test_team_analytics_success(client: AsyncClient, admin_auth_headers: dict):
"""Team admin can access team analytics."""
response = await client.get(
"/api/v1/analytics/team?period=30d",
headers=admin_auth_headers,
)
assert response.status_code == 200
data = response.json()
assert "summary" in data
assert "time_series" in data
assert "top_flows" in data
assert "top_engineers" in data
assert "total_sessions" in data["summary"]
assert "completion_rate" in data["summary"]
async def test_team_analytics_forbidden_for_engineer(client: AsyncClient, auth_headers: dict):
"""Regular engineers cannot access team analytics."""
response = await client.get(
"/api/v1/analytics/team",
headers=auth_headers,
)
assert response.status_code == 403
async def test_personal_analytics_success(client: AsyncClient, auth_headers: dict):
"""Any user can access personal analytics."""
response = await client.get(
"/api/v1/analytics/me?period=30d",
headers=auth_headers,
)
assert response.status_code == 200
data = response.json()
assert "summary" in data
assert "time_series" in data
assert "top_flows" in data
async def test_flow_analytics_success(client: AsyncClient, auth_headers: dict, tree_id: str):
"""Can access analytics for a visible flow."""
response = await client.get(
f"/api/v1/analytics/flows/{tree_id}",
headers=auth_headers,
)
assert response.status_code == 200
data = response.json()
assert "summary" in data
assert "step_feedback" in data
assert "recent_comments" in data
Step 2: Implement analytics endpoint
Create backend/app/api/endpoints/analytics.py:
from datetime import datetime, timezone, timedelta
from uuid import UUID
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import select, func, case, and_, cast, Date
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.database import get_db
from app.api.deps import get_current_active_user
from app.models import User, Session, Tree, SessionRating, StepRating
from app.schemas.analytics import (
TeamAnalyticsResponse, PersonalAnalyticsResponse, FlowAnalyticsResponse,
AnalyticsSummary, OutcomeBreakdown, TimeSeriesPoint,
TopFlow, TopEngineer, StepFeedbackSummary, FlowRatingItem,
)
router = APIRouter(prefix="/analytics", tags=["analytics"])
def _get_period_start(period: str) -> datetime:
days = {"7d": 7, "30d": 30, "90d": 90}.get(period, 30)
return datetime.now(timezone.utc) - timedelta(days=days)
async def _build_summary(db: AsyncSession, base_filter) -> AnalyticsSummary:
"""Build analytics summary from a filtered session query."""
# Total and completed counts
total_q = await db.execute(select(func.count()).select_from(Session).where(*base_filter))
total = total_q.scalar() or 0
completed_q = await db.execute(
select(func.count()).select_from(Session).where(*base_filter, Session.completed_at.isnot(None))
)
completed = completed_q.scalar() or 0
# Avg duration (minutes)
duration_q = await db.execute(
select(
func.avg(func.extract('epoch', Session.completed_at - Session.started_at) / 60)
).where(*base_filter, Session.completed_at.isnot(None))
)
avg_duration = round(float(duration_q.scalar() or 0), 1)
# Outcome breakdown
outcome_q = await db.execute(
select(Session.outcome, func.count()).where(
*base_filter, Session.completed_at.isnot(None), Session.outcome.isnot(None)
).group_by(Session.outcome)
)
outcomes = dict(outcome_q.all())
return AnalyticsSummary(
total_sessions=total,
completed_sessions=completed,
completion_rate=round(completed / total, 3) if total > 0 else 0,
avg_duration_minutes=avg_duration,
outcome_breakdown=OutcomeBreakdown(
resolved=outcomes.get("resolved", 0),
escalated=outcomes.get("escalated", 0),
workaround=outcomes.get("workaround", 0),
unresolved=outcomes.get("unresolved", 0),
),
)
async def _build_time_series(db: AsyncSession, base_filter, period_start: datetime) -> list[TimeSeriesPoint]:
"""Build daily time-series of session counts by outcome."""
rows = await db.execute(
select(
cast(Session.started_at, Date).label("date"),
func.count().label("sessions"),
func.count().filter(Session.outcome == "resolved").label("resolved"),
func.count().filter(Session.outcome == "escalated").label("escalated"),
func.count().filter(Session.outcome == "workaround").label("workaround"),
func.count().filter(Session.outcome == "unresolved").label("unresolved"),
).where(*base_filter, Session.started_at >= period_start)
.group_by(cast(Session.started_at, Date))
.order_by(cast(Session.started_at, Date))
)
return [
TimeSeriesPoint(
date=str(row.date), sessions=row.sessions,
resolved=row.resolved, escalated=row.escalated,
workaround=row.workaround, unresolved=row.unresolved,
)
for row in rows.all()
]
@router.get("/team", response_model=TeamAnalyticsResponse)
async def get_team_analytics(
period: str = Query("30d", pattern="^(7d|30d|90d)$"),
engineer_id: Optional[UUID] = None,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_active_user),
):
"""Team analytics — team_admin or super_admin only."""
if not (current_user.is_team_admin or current_user.role == "super_admin"):
raise HTTPException(status_code=403, detail="Team admin access required")
period_start = _get_period_start(period)
base_filter = [
Session.started_at >= period_start,
Tree.account_id == current_user.account_id,
]
if engineer_id:
base_filter.append(Session.user_id == engineer_id)
# Need to join Session to Tree for account_id scoping
# Adjust queries to use join
summary = await _build_summary_with_join(db, base_filter, period_start)
time_series = await _build_time_series_with_join(db, base_filter, period_start)
# Top flows
top_flows_q = await db.execute(
select(
Tree.id, Tree.name,
func.count(Session.id).label("sessions"),
(func.count().filter(Session.completed_at.isnot(None)) * 1.0 / func.count()).label("completion_rate"),
func.avg(func.extract('epoch', Session.completed_at - Session.started_at) / 60).label("avg_duration"),
)
.join(Tree, Session.tree_id == Tree.id)
.where(Session.started_at >= period_start, Tree.account_id == current_user.account_id)
.group_by(Tree.id, Tree.name)
.order_by(func.count(Session.id).desc())
.limit(10)
)
top_flows = [
TopFlow(
tree_id=str(row.id), name=row.name, sessions=row.sessions,
completion_rate=round(float(row.completion_rate or 0), 3),
avg_duration_minutes=round(float(row.avg_duration or 0), 1),
)
for row in top_flows_q.all()
]
# Top engineers
top_engineers_q = await db.execute(
select(
User.id, User.name,
func.count(Session.id).label("sessions"),
(func.count().filter(Session.completed_at.isnot(None)) * 1.0 / func.count()).label("completion_rate"),
func.avg(func.extract('epoch', Session.completed_at - Session.started_at) / 60).label("avg_duration"),
)
.join(User, Session.user_id == User.id)
.join(Tree, Session.tree_id == Tree.id)
.where(Session.started_at >= period_start, Tree.account_id == current_user.account_id)
.group_by(User.id, User.name)
.order_by(func.count(Session.id).desc())
.limit(10)
)
top_engineers = [
TopEngineer(
user_id=str(row.id), name=row.name or "Unknown", sessions=row.sessions,
completion_rate=round(float(row.completion_rate or 0), 3),
avg_duration_minutes=round(float(row.avg_duration or 0), 1),
)
for row in top_engineers_q.all()
]
return TeamAnalyticsResponse(
summary=summary, time_series=time_series,
top_flows=top_flows, top_engineers=top_engineers,
)
@router.get("/me", response_model=PersonalAnalyticsResponse)
async def get_personal_analytics(
period: str = Query("30d", pattern="^(7d|30d|90d)$"),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_active_user),
):
"""Personal analytics — any authenticated user."""
period_start = _get_period_start(period)
base_filter = [Session.started_at >= period_start, Session.user_id == current_user.id]
summary = await _build_summary(db, base_filter)
time_series = await _build_time_series(db, base_filter, period_start)
# My top flows
top_flows_q = await db.execute(
select(
Tree.id, Tree.name,
func.count(Session.id).label("sessions"),
(func.count().filter(Session.completed_at.isnot(None)) * 1.0 / func.count()).label("completion_rate"),
func.avg(func.extract('epoch', Session.completed_at - Session.started_at) / 60).label("avg_duration"),
)
.join(Tree, Session.tree_id == Tree.id)
.where(Session.started_at >= period_start, Session.user_id == current_user.id)
.group_by(Tree.id, Tree.name)
.order_by(func.count(Session.id).desc())
.limit(10)
)
top_flows = [
TopFlow(
tree_id=str(row.id), name=row.name, sessions=row.sessions,
completion_rate=round(float(row.completion_rate or 0), 3),
avg_duration_minutes=round(float(row.avg_duration or 0), 1),
)
for row in top_flows_q.all()
]
return PersonalAnalyticsResponse(
summary=summary, time_series=time_series, top_flows=top_flows,
)
@router.get("/flows/{tree_id}", response_model=FlowAnalyticsResponse)
async def get_flow_analytics(
tree_id: UUID,
period: str = Query("30d", pattern="^(7d|30d|90d)$"),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_active_user),
):
"""Analytics for a specific flow."""
# Verify tree exists and user can view it
result = await db.execute(select(Tree).where(Tree.id == tree_id))
tree = result.scalar_one_or_none()
if not tree:
raise HTTPException(status_code=404, detail="Flow not found")
period_start = _get_period_start(period)
base_filter = [Session.started_at >= period_start, Session.tree_id == tree_id]
summary = await _build_summary(db, base_filter)
time_series = await _build_time_series(db, base_filter, period_start)
# CSAT stats
csat_q = await db.execute(
select(func.avg(SessionRating.rating), func.count())
.where(SessionRating.tree_id == tree_id, SessionRating.created_at >= period_start)
)
csat_row = csat_q.one()
avg_csat = round(float(csat_row[0]), 1) if csat_row[0] else None
total_ratings = csat_row[1]
# Step feedback (thumbs) — aggregate by node_id from decisions JSONB
# This requires joining step_ratings with session decisions
# For v1, return step_library-level feedback if steps are from library
step_feedback: list[StepFeedbackSummary] = []
# Recent comments
comments_q = await db.execute(
select(SessionRating.rating, SessionRating.comment, User.name, SessionRating.created_at)
.join(User, SessionRating.user_id == User.id)
.where(
SessionRating.tree_id == tree_id,
SessionRating.comment.isnot(None),
SessionRating.comment != "",
)
.order_by(SessionRating.created_at.desc())
.limit(10)
)
recent_comments = [
FlowRatingItem(
rating=row.rating, comment=row.comment,
user_name=row.name, created_at=row.created_at,
)
for row in comments_q.all()
]
return FlowAnalyticsResponse(
summary=summary, avg_csat=avg_csat, total_ratings=total_ratings,
time_series=time_series, step_feedback=step_feedback,
recent_comments=recent_comments,
)
Note: The _build_summary_with_join and _build_time_series_with_join helper functions need to be implemented with proper Session-Tree joins for account scoping. The implementing engineer should refactor _build_summary and _build_time_series to optionally accept a join condition.
Step 3: Register route
In backend/app/api/router.py:
from app.api.endpoints import analytics
api_router.include_router(analytics.router)
Step 4: Run tests
cd backend && python -m pytest tests/test_analytics.py -v --override-ini="addopts="
Step 5: Commit
git add backend/app/api/endpoints/analytics.py backend/app/api/router.py backend/tests/test_analytics.py
git commit -m "feat: add team, personal, and flow analytics endpoints"
Task 6: Frontend Setup — Recharts, Types, API Client
Files:
- Modify:
frontend/package.json(install recharts) - Create:
frontend/src/types/analytics.ts - Modify:
frontend/src/types/index.ts - Create:
frontend/src/api/analytics.ts - Modify:
frontend/src/api/index.ts - Modify:
frontend/src/api/sessions.ts
Step 1: Install Recharts
cd frontend && npm install recharts
Step 2: Create analytics types
Create frontend/src/types/analytics.ts:
export interface OutcomeBreakdown {
resolved: number
escalated: number
workaround: number
unresolved: number
}
export interface AnalyticsSummary {
total_sessions: number
completed_sessions: number
completion_rate: number
avg_duration_minutes: number
outcome_breakdown: OutcomeBreakdown
}
export interface TimeSeriesPoint {
date: string
sessions: number
resolved: number
escalated: number
workaround: number
unresolved: number
}
export interface TopFlow {
tree_id: string
name: string
sessions: number
completion_rate: number
avg_duration_minutes: number
avg_csat?: number
}
export interface TopEngineer {
user_id: string
name: string
sessions: number
completion_rate: number
avg_duration_minutes: number
}
export interface TeamAnalyticsResponse {
summary: AnalyticsSummary
time_series: TimeSeriesPoint[]
top_flows: TopFlow[]
top_engineers: TopEngineer[]
}
export interface PersonalAnalyticsResponse {
summary: AnalyticsSummary
time_series: TimeSeriesPoint[]
top_flows: TopFlow[]
}
export interface StepFeedbackSummary {
node_id: string
node_title: string
helpful_yes: number
helpful_no: number
helpful_rate: number
}
export interface FlowRatingItem {
rating: number
comment?: string
user_name?: string
created_at: string
}
export interface FlowAnalyticsResponse {
summary: AnalyticsSummary
avg_csat?: number
total_ratings: number
time_series: TimeSeriesPoint[]
step_feedback: StepFeedbackSummary[]
recent_comments: FlowRatingItem[]
}
export type AnalyticsPeriod = '7d' | '30d' | '90d'
Step 3: Export from types/index.ts
Add to frontend/src/types/index.ts:
export type * from './analytics'
Step 4: Create analytics API client
Create frontend/src/api/analytics.ts:
import { apiClient } from './client'
import type {
TeamAnalyticsResponse,
PersonalAnalyticsResponse,
FlowAnalyticsResponse,
AnalyticsPeriod,
} from '@/types'
export const analyticsApi = {
async getTeamAnalytics(period: AnalyticsPeriod = '30d', engineerId?: string): Promise<TeamAnalyticsResponse> {
const params: Record<string, string> = { period }
if (engineerId) params.engineer_id = engineerId
const response = await apiClient.get<TeamAnalyticsResponse>('/analytics/team', { params })
return response.data
},
async getPersonalAnalytics(period: AnalyticsPeriod = '30d'): Promise<PersonalAnalyticsResponse> {
const response = await apiClient.get<PersonalAnalyticsResponse>('/analytics/me', { params: { period } })
return response.data
},
async getFlowAnalytics(treeId: string, period: AnalyticsPeriod = '30d'): Promise<FlowAnalyticsResponse> {
const response = await apiClient.get<FlowAnalyticsResponse>(`/analytics/flows/${treeId}`, { params: { period } })
return response.data
},
async rateSession(sessionId: string, rating: number, comment?: string): Promise<void> {
await apiClient.post(`/sessions/${sessionId}/rate`, { rating, comment })
},
async submitStepFeedback(stepId: string, sessionId: string, wasHelpful: boolean): Promise<void> {
await apiClient.post(`/steps/${stepId}/feedback`, { session_id: sessionId, was_helpful: wasHelpful })
},
}
Step 5: Export from api/index.ts
Add to frontend/src/api/index.ts:
export { analyticsApi } from './analytics'
Step 6: Verify build
cd frontend && npm run build
Step 7: Commit
git add frontend/package.json frontend/package-lock.json frontend/src/types/analytics.ts frontend/src/types/index.ts frontend/src/api/analytics.ts frontend/src/api/index.ts
git commit -m "feat: add recharts, analytics types, and API client"
Task 7: Inline Step Feedback Component
Files:
- Create:
frontend/src/components/session/StepFeedback.tsx - Modify:
frontend/src/pages/TreeNavigationPage.tsx - Modify:
frontend/src/pages/ProceduralNavigationPage.tsx(same pattern)
Step 1: Create StepFeedback component
Create frontend/src/components/session/StepFeedback.tsx:
import { useState, useEffect } from 'react'
import { ThumbsUp, ThumbsDown } from 'lucide-react'
import { analyticsApi } from '@/api'
import { cn } from '@/lib/utils'
interface StepFeedbackProps {
stepId: string
sessionId: string
}
const HINT_KEY = 'rf-step-feedback-hint-dismissed'
export function StepFeedback({ stepId, sessionId }: StepFeedbackProps) {
const [feedback, setFeedback] = useState<boolean | null>(null)
const [submitting, setSubmitting] = useState(false)
const [showHint, setShowHint] = useState(false)
useEffect(() => {
if (!localStorage.getItem(HINT_KEY)) {
setShowHint(true)
}
}, [])
const handleFeedback = async (wasHelpful: boolean) => {
if (submitting) return
setSubmitting(true)
try {
const newValue = feedback === wasHelpful ? null : wasHelpful
if (newValue !== null) {
await analyticsApi.submitStepFeedback(stepId, sessionId, newValue)
}
setFeedback(newValue)
if (showHint) {
setShowHint(false)
localStorage.setItem(HINT_KEY, '1')
}
} catch {
// Silently fail — feedback is non-critical
} finally {
setSubmitting(false)
}
}
return (
<div className="flex items-center gap-2">
{showHint && (
<span className="text-xs text-muted-foreground">Rate this step to help improve flows</span>
)}
<button
onClick={() => handleFeedback(true)}
disabled={submitting}
className={cn(
'rounded-md p-1.5 transition-colors',
feedback === true
? 'text-emerald-400 bg-emerald-400/10'
: 'text-muted-foreground hover:text-emerald-400 hover:bg-accent'
)}
title="Helpful"
>
<ThumbsUp size={14} />
</button>
<button
onClick={() => handleFeedback(false)}
disabled={submitting}
className={cn(
'rounded-md p-1.5 transition-colors',
feedback === false
? 'text-red-400 bg-red-400/10'
: 'text-muted-foreground hover:text-red-400 hover:bg-accent'
)}
title="Not helpful"
>
<ThumbsDown size={14} />
</button>
</div>
)
}
Step 2: Integrate into TreeNavigationPage
In TreeNavigationPage.tsx, find where step content is rendered (after the step description/help_text, before decision options or continue button). Add:
import { StepFeedback } from '@/components/session/StepFeedback'
// Inside the step rendering, after content and before options:
{currentNode && session && (
<div className="mt-3 flex justify-end">
<StepFeedback stepId={currentNode.id} sessionId={session.id} />
</div>
)}
Apply the same pattern to ProceduralNavigationPage.tsx for procedural flow steps.
Step 3: Verify build
cd frontend && npm run build
Step 4: Commit
git add frontend/src/components/session/StepFeedback.tsx frontend/src/pages/TreeNavigationPage.tsx frontend/src/pages/ProceduralNavigationPage.tsx
git commit -m "feat: add inline step thumbs up/down feedback during sessions"
Task 8: CSAT Modal Component
Files:
- Create:
frontend/src/components/session/CSATModal.tsx - Modify:
frontend/src/pages/TreeNavigationPage.tsx - Modify:
frontend/src/pages/ProceduralNavigationPage.tsx
Step 1: Create CSATModal
Create frontend/src/components/session/CSATModal.tsx:
import { useState } from 'react'
import { Star } from 'lucide-react'
import { Modal } from '@/components/common/Modal'
import { analyticsApi } from '@/api'
import { cn } from '@/lib/utils'
interface CSATModalProps {
isOpen: boolean
onClose: () => void
sessionId: string
}
const RATED_SESSIONS_KEY = 'rf-rated-sessions'
function getRatedSessions(): string[] {
try {
return JSON.parse(localStorage.getItem(RATED_SESSIONS_KEY) || '[]')
} catch {
return []
}
}
function markSessionRated(sessionId: string) {
const rated = getRatedSessions()
rated.push(sessionId)
localStorage.setItem(RATED_SESSIONS_KEY, JSON.stringify(rated.slice(-100)))
}
export function hasBeenRated(sessionId: string): boolean {
return getRatedSessions().includes(sessionId)
}
export function CSATModal({ isOpen, onClose, sessionId }: CSATModalProps) {
const [rating, setRating] = useState(0)
const [hoveredRating, setHoveredRating] = useState(0)
const [comment, setComment] = useState('')
const [submitting, setSubmitting] = useState(false)
const handleSubmit = async () => {
if (rating === 0 || submitting) return
setSubmitting(true)
try {
await analyticsApi.rateSession(sessionId, rating, comment || undefined)
markSessionRated(sessionId)
onClose()
} catch {
// Silently fail
onClose()
} finally {
setSubmitting(false)
}
}
const handleSkip = () => {
markSessionRated(sessionId)
onClose()
}
return (
<Modal isOpen={isOpen} onClose={handleSkip} title="How was this flow?">
<div className="space-y-4">
<p className="text-sm text-muted-foreground">
Your feedback helps flow authors improve troubleshooting paths.
</p>
{/* Star rating */}
<div className="flex items-center justify-center gap-1">
{[1, 2, 3, 4, 5].map((value) => (
<button
key={value}
onClick={() => setRating(value)}
onMouseEnter={() => setHoveredRating(value)}
onMouseLeave={() => setHoveredRating(0)}
className="p-1 transition-colors"
>
<Star
size={28}
className={cn(
'transition-colors',
(hoveredRating || rating) >= value
? 'fill-yellow-400 text-yellow-400'
: 'fill-none text-muted-foreground'
)}
/>
</button>
))}
</div>
{/* Comment */}
<textarea
value={comment}
onChange={(e) => setComment(e.target.value)}
placeholder="Any comments? (optional)"
maxLength={500}
rows={3}
className="w-full rounded-lg border border-border bg-card px-3 py-2 text-sm text-foreground placeholder:text-muted-foreground focus:border-primary focus:outline-none focus:ring-1 focus:ring-primary/20 resize-none"
/>
{/* Actions */}
<div className="flex items-center justify-between">
<button onClick={handleSkip} className="text-sm text-muted-foreground hover:text-foreground transition-colors">
Skip
</button>
<button
onClick={handleSubmit}
disabled={rating === 0 || submitting}
className="rounded-lg bg-gradient-brand px-4 py-2 text-sm font-semibold text-white shadow-lg shadow-primary/20 hover:opacity-90 transition-opacity disabled:opacity-50"
>
{submitting ? 'Submitting...' : 'Submit'}
</button>
</div>
</div>
</Modal>
)
}
Step 2: Integrate into session completion flow
In TreeNavigationPage.tsx, after the SessionOutcomeModal submit handler resolves:
import { CSATModal, hasBeenRated } from '@/components/session/CSATModal'
// Add state
const [csatOpen, setCsatOpen] = useState(false)
// In the outcome submit handler, after successful completion:
const handleSubmitOutcome = async (data) => {
await sessionsApi.complete(session.id, data)
// ... existing logic ...
if (!hasBeenRated(session.id)) {
setCsatOpen(true)
}
}
// In JSX, after SessionOutcomeModal:
{session && (
<CSATModal
isOpen={csatOpen}
onClose={() => setCsatOpen(false)}
sessionId={session.id}
/>
)}
Apply the same pattern to ProceduralNavigationPage.tsx.
Step 3: Verify build
cd frontend && npm run build
Step 4: Commit
git add frontend/src/components/session/CSATModal.tsx frontend/src/pages/TreeNavigationPage.tsx frontend/src/pages/ProceduralNavigationPage.tsx
git commit -m "feat: add CSAT rating modal after session completion"
Task 9: Team Analytics Page
Files:
- Create:
frontend/src/pages/TeamAnalyticsPage.tsx - Modify:
frontend/src/router.tsx - Modify:
frontend/src/components/layout/Sidebar.tsx
Step 1: Create the page
Create frontend/src/pages/TeamAnalyticsPage.tsx:
Build a page with:
- Header: "Team Analytics" + period dropdown (
<select>with 7d/30d/90d) - Stat cards row: Total Sessions, Completion Rate (%), Avg Duration, Active Engineers
- Time-series area chart using Recharts
<AreaChart>with stacked outcomes (resolved=emerald, escalated=red, workaround=yellow, unresolved=slate) - Two-column layout below:
- Left: Top Flows table (name, sessions, completion rate, avg duration)
- Right: Top Engineers table (name, sessions, completion rate, avg duration)
Use the analyticsApi.getTeamAnalytics() method. Loading state with <Loader2> spinner.
Guard with permissions: if not team_admin/super_admin, show "You don't have access to team analytics" message with a link to /analytics/me.
Recharts imports needed:
import { AreaChart, Area, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer } from 'recharts'
Chart color constants:
const CHART_COLORS = {
resolved: '#34d399',
escalated: '#f87171',
workaround: '#fbbf24',
unresolved: '#94a3b8',
}
Step 2: Add route
In frontend/src/router.tsx, add lazy import and route:
const TeamAnalyticsPage = lazy(() => import('@/pages/TeamAnalyticsPage'))
// Inside AppLayout children:
{ path: 'analytics', element: <Suspense fallback={<PageLoader />}><TeamAnalyticsPage /></Suspense> },
Step 3: Add nav item
In frontend/src/components/layout/Sidebar.tsx, add between Sessions and Exports (in both collapsed and expanded branches):
import { BarChart3 } from 'lucide-react'
<NavItem href="/analytics" icon={BarChart3} label="Analytics" />
Step 4: Verify build
cd frontend && npm run build
Step 5: Commit
git add frontend/src/pages/TeamAnalyticsPage.tsx frontend/src/router.tsx frontend/src/components/layout/Sidebar.tsx
git commit -m "feat: add Team Analytics page with charts and leaderboards"
Task 10: Personal Analytics Page
Files:
- Create:
frontend/src/pages/MyAnalyticsPage.tsx - Modify:
frontend/src/router.tsx
Step 1: Create the page
Create frontend/src/pages/MyAnalyticsPage.tsx:
Similar structure to TeamAnalyticsPage but:
- Header: "My Analytics"
- Stat cards: My Sessions, My Completion Rate, My Avg Duration, My Outcomes
- Sessions-per-day line chart (Recharts
<LineChart>) - Two-column layout:
- Left: My Top Flows table
- Right: Outcome distribution donut chart (Recharts
<PieChart>with<Pie>)
Use analyticsApi.getPersonalAnalytics(). No permission guard — any user can access.
Step 2: Add route
const MyAnalyticsPage = lazy(() => import('@/pages/MyAnalyticsPage'))
{ path: 'analytics/me', element: <Suspense fallback={<PageLoader />}><MyAnalyticsPage /></Suspense> },
Step 3: Update TeamAnalyticsPage — add a link/tab to switch between Team and Personal views.
Step 4: Verify build
cd frontend && npm run build
Step 5: Commit
git add frontend/src/pages/MyAnalyticsPage.tsx frontend/src/router.tsx
git commit -m "feat: add My Analytics page with personal stats and charts"
Task 11: Flow Analytics Panel
Files:
- Create:
frontend/src/components/analytics/FlowAnalyticsPanel.tsx - Modify:
frontend/src/pages/TreeEditorPage.tsxor tree detail view
Step 1: Create the panel
Create frontend/src/components/analytics/FlowAnalyticsPanel.tsx:
A collapsible or tabbed panel showing:
- Summary stat row: Usage, Completion Rate, Avg Duration, CSAT score
- Mini area chart for sessions-over-time (compact, 200px height)
- Step feedback table: step name, helpful rate bar (green fill proportional to %), thumbs counts
- Recent comments list: star rating, comment text, user name, date
Use analyticsApi.getFlowAnalytics(treeId).
Step 2: Integrate into tree views
Add as a tab or expandable section in the tree editor or tree detail page. The exact integration point depends on the current page structure — look for a natural place to add an "Analytics" tab alongside existing content.
Step 3: Verify build
cd frontend && npm run build
Step 4: Commit
git add frontend/src/components/analytics/FlowAnalyticsPanel.tsx frontend/src/pages/TreeEditorPage.tsx
git commit -m "feat: add Flow Analytics panel with step feedback and CSAT"
Task 12: Final Integration & Polish
Step 1: Run full backend test suite
cd backend && python -m pytest --override-ini="addopts=" -v
Fix any failures.
Step 2: Run frontend build
cd frontend && npm run build
Fix any type errors.
Step 3: Test end-to-end manually
- Start a session, verify inline thumbs appear on steps
- Complete a session, verify CSAT modal appears
- Submit a rating, verify it persists
- Visit
/analyticsas team admin, verify charts render - Visit
/analytics/meas engineer, verify personal stats
Step 4: Final commit
git add -A
git commit -m "fix: analytics integration polish and test fixes"