From d51e95cdfa1bfdcc47aa90c984fb44d15df07bbf Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 15:18:46 -0400 Subject: [PATCH 01/34] docs(plans): add escalation-mode wedge design + test plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Captures the GTM thesis, premises, reduced-scope engineering plan, locked UI specs, and embedded review report for the Escalation Mode wedge — output of /office-hours, /plan-eng-review, /plan-design-review, and /codex review. Codex review surfaced two corrections we applied: - two-metric framing (manual baseline vs in-product time-to-first-action) - claim role gate moved in-scope (was deferred TODO) TODO updates: peer-tech escalation + claim role gate captured (the latter then moved in-scope by the codex pass). Co-Authored-By: Claude Opus 4.7 --- .ai/TODO.md | 6 + ...2026-04-27-escalation-mode-wedge-design.md | 494 ++++++++++++++++++ ...6-04-27-escalation-mode-wedge-test-plan.md | 33 ++ 3 files changed, 533 insertions(+) create mode 100644 docs/plans/2026-04-27-escalation-mode-wedge-design.md create mode 100644 docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md diff --git a/.ai/TODO.md b/.ai/TODO.md index b7d5270f..3f5ab56d 100644 --- a/.ai/TODO.md +++ b/.ai/TODO.md @@ -15,3 +15,9 @@ - [ ] **Per-test transactional rollback in `test_db` fixture.** Bigger engineering than xdist (which we already shipped). Instead of `DROP SCHEMA public CASCADE` per test, wrap each test in a savepoint and rollback at teardown. ~30-40% additional speedup on top of xdist for test-DB-heavy tests. Real refactor; only worth it if the suite gets significantly larger or runs more frequently. - [ ] **Consider `pytest-testmon` for PR-time test selection.** Tracks which tests touched which source files and only re-runs affected ones. Best for small PRs touching ~few files. Adds cache-invalidation complexity; only worth it if the suite stays painfully long even after xdist. - [ ] **AssistantChatPage `currentChatRef` guard is a silent return** — `handleSend`, `handleTaskSubmit`, `selectChat`, `refreshFacts`, `refreshActiveFix`, and `refreshPreview` all bail with `if (currentChatRef.current !== sentForChatId) return` when stale. This is by design for chat switching, but it also silently masked the prefill-ref bug fixed in PR #153 — the user just saw "no AI response" with no log, no toast, no Sentry event. Either (a) log a `console.warn`/Sentry breadcrumb on the mismatch path so future drift is visible, or (b) split "expected stale" (chat switch) from "unexpected stale" (ref never updated) so only the latter alerts. Pair with an audit of every `currentChatRef.current = ...` assignment vs every `setActiveChatId(...)` call to make sure they're paired everywhere. + +- [ ] **Allow peer-tech to escalate a colleague's session.** Today `POST /ai-sessions/{session_id}/handoff` in [endpoints/session_handoffs.py:48](backend/app/api/endpoints/session_handoffs.py#L48) filters by `AISession.user_id == current_user.id`, so only the session owner can escalate. Real MSP shops have peer hand-offs: Junior A is on lunch, Junior B sees the session is stuck and should be able to escalate it. Auth tweak: switch from session-owner check to `require_engineer_or_admin` + same-account scope. Add a `handed_off_by` audit column (already exists on `SessionHandoff`) so the original-owner-vs-actual-escalator distinction is preserved. Surfaced from /plan-eng-review on the Escalation-Mode wedge plan; v1 wedge demo doesn't need this (solo-founder pilot), but capture for v2 once 3+ pilots are live and a peer-claim need surfaces. + +- [ ] **Mobile/responsive design for EscalationQueue + handoff-context screen.** Pre-PMF wedge demo targets desktop only — MSP techs work on laptops/desktops in shop environments. Once 3+ paying customers exist and a tech requests mobile (likely on-call use case), spec the responsive behavior: stacked card layout below `sm:` breakpoint, full-bleed handoff-context overlay on mobile, swipe-to-claim gesture instead of Pick Up button. Surfaced from /plan-design-review on the Escalation-Mode wedge plan. + +- [ ] **(MOVED IN-SCOPE for Escalation Mode v1, 2026-04-27)** ~~Add role gate to handoff claim endpoint.~~ Codex review correctly flagged this as wedge-relevant (the race-condition story depends on auth gating). Now part of the Escalation Mode v1 build, not a deferred TODO. diff --git a/docs/plans/2026-04-27-escalation-mode-wedge-design.md b/docs/plans/2026-04-27-escalation-mode-wedge-design.md new file mode 100644 index 00000000..25a05701 --- /dev/null +++ b/docs/plans/2026-04-27-escalation-mode-wedge-design.md @@ -0,0 +1,494 @@ +# Design: ResolutionFlow GTM — Escalation-Mode-First Wedge + +Generated by /office-hours on 2026-04-26 +Branch: main +Repo: chihlasm/resolutionflow +Status: APPROVED +Mode: Startup + +## Problem Statement + +ResolutionFlow is a multi-tenant SaaS troubleshooting platform for MSPs, currently +in Go-to-Market Validation (pre-PMF). The backend is feature-complete (55+ endpoints, +100+ tests, FlowPilot telemetry baseline accruing). The product has users but no +paying customers. + +The blocker is not engineering completeness. The blocker is the absence of a sharp +GTM story tied to a number a buyer can verify. The session reframed the wedge twice +before landing on the real one. + +**What ResolutionFlow actually is:** the structuring layer between conversational AI +and the way MSP techs work tickets. AI is great at producing answers; it is bad at +producing workflow-shaped output. ResolutionFlow gives the tech the AI they already +trust (Claude/GPT) but organizes the output into actionable structured steps, +records the session, captures customer-specific context, and turns the result into +PSA-formatted ticket notes — and optionally a runbook — without the tech writing +anything. + +**Positioning line:** "the senior engineer looking over your shoulder." + +## Demand Evidence + +The founder is the first user. Senior Systems Engineer at an MSP, losing ~20 +hours/week to cross-domain interruptions (systems engineer pulled into networking +problems and vice versa). At least 4 interruptions per day, with the time cost +concentrated in the gap between AI-conversation output and MSP-ticket workflow. + +This is solving-your-own-problem demand evidence — strongest possible signal at +this stage. The 20 hrs/week figure is the founder's own time, not a hypothetical. +Every MSP shop with a senior tech and a junior tech has a version of this problem. + +Telemetry signal (Phase 0.5 baseline accruing): captured flows pile up but are not +being re-used. This says capture works, retrieval doesn't — which means the +"hours-saved-via-re-use" number isn't yet generatable from existing data. The +GTM-grade ROI story needs a different metric until re-use lands: minutes recovered +per escalation, generated by Approach A below. + +## Status Quo + +MSP techs today resolve tickets via three workarounds: + +1. **AI in a tab.** Junior tech opens Claude or ChatGPT, pastes the problem, gets a + wall of prose, parses it into action items in their head, executes, repeats. AI + does the diagnostic work. The tech does all the structure-extraction and + ticket-note-writing afterward. + +2. **Tribal knowledge.** Junior tech pings senior in Slack. Senior tech is + interrupted (4+ times/day per the founder's own data). Context handoff is verbal + and lossy. + +3. **Stale runbooks.** Half-maintained Notion / IT Glue / SharePoint pages that + nobody trusts because they're 18 months out of date and don't match the current + customer environment. + +The cost of these workarounds for the founder personally: ~20 hours per week of +senior-tech time lost. For a 5-tech MSP, the equivalent is 1 full FTE worth of +senior-engineer hours leaking into context-switching and tab-hopping. + +## Target User & Narrowest Wedge + +**Target user:** Senior Systems Engineer at a small-to-mid MSP (5-20 techs). The +founder is exemplar #1. Buying authority is shared between senior tech (champion) +and MSP owner (signs the check). + +**Narrowest paid wedge:** Escalation Mode. Single sharp feature. When a junior tech +escalates a ticket they were working in FlowPilot, the senior tech opens the ticket +and sees the entire structured session state — every step the junior tried, every +dead end, every command output — instead of starting with "tell me what you tried" +for five minutes. + +Why this is the wedge: + +- **Two metrics, not one** (revised after /codex review 2026-04-27): + - **Manual baseline** (the Assignment, weeks 0-2): senior tech stopwatches the + next 5 escalations. T1 (first diagnostic action) − T0 (open ticket) under + today's verbal-handoff workflow. This is the "what you currently lose" number. + - **In-product metric** (telemetry, week 3+): time-to-first-action after claim, + derived from `ai_session_step` rows where `created_at > SessionHandoff.claimed_at` + AND `user_id = SessionHandoff.claimed_by`. This is the "what it is now with + structured handoff" number. + - **The savings claim** = manual baseline − in-product metric. Quote both + explicitly in pilot conversations. Do NOT roll the in-product number alone + into "minutes recovered" — that's an apples-to-oranges miscount Codex caught + in the cross-model review. +- **Single-feature demo:** a 2-minute Loom shows the magic moment — junior hits + escalate, senior window opens with full structured context. No theory required. +- **Cross-buyer story:** sells to senior tech (less interruption) AND owner (junior + techs resolve faster, take more accounts). +- **Hours-saved math is simple:** 4-5 minutes per escalation × 15-30 escalations + per week per senior tech = 1-2 hours/week recovered per senior. At $80-150/hr + fully-loaded senior tech cost, the tool pays for itself with one customer. + +## Constraints + +- **One-founder shop.** Cannot run three concurrent product narratives. Sequence + matters more than scope. +- **Pre-PMF runway implied.** 4-8 week build cycles before talking to a buyer are + expensive. Approach A's 1-2 week timeline is the binding constraint. +- **Existing architecture is mostly aligned.** FlowPilot, unified_chat_service, + FlowProposal, ConnectWise PSA integration — most of the pieces exist. Risk is + positioning and UX, not capability. +- **PSA copilot competition is real.** ConnectWise / Autotask / Halo are racing to + ship AI features. The wedge has to be sharp because we lose on distribution. + +## Premises + +The five load-bearing claims this design rests on, all confirmed in session: + +1. **Diagnostic AI is commoditized.** ResolutionFlow does not compete on + "AI solves the ticket faster." That race is over. ChatGPT/Claude already won. +2. **The structuring layer is the wedge.** AI conversational output is too dense + and unstructured for active troubleshooting. ResolutionFlow's value is + organizing that output into actionable, separable, recorded steps. +3. **Escalation context is the killer feature.** "Junior hits escalate, senior gets + full structured context in 30 seconds instead of 5 minutes" is the sharpest + demoable moment in the entire product surface. +4. **First paying customer is bottom-up, prosumer-flavored.** Senior tech at a + small MSP, $20-50/seat/month, monthly billing. Owner-targeted enterprise + pricing waits until 5+ paying shops establish baseline ROI numbers. +5. **Distribution is MSP communities, not paid SaaS ads.** r/msp, MSPGeek, RocketMSP, + PSA marketplace listings. The channel matches the buyer. + +## Approaches Considered + +### Approach A: Escalation Mode first (REDUCED SCOPE per /plan-eng-review) + +Lead the GTM with the killer feature. Polish the escalate-with-context handoff: +junior tech mid-session hits escalate, senior tech window opens with full +structured session state. 2-min demo Loom. Pilot with **3 MSPs** in the founder's +network (capped at 3 to preserve build capacity for B). Metric: minutes recovered +per escalation. + +**SCOPE REDUCTION (2026-04-27 eng review):** ~80% of Approach A is already built. +The original 2-3 week estimate assumed greenfield. Codebase audit confirms: + +| What the doc said "build" | What actually exists | +|---|---| +| Session-state serialization | `ai_session.escalation_package` (JSONB), `SessionHandoff.snapshot` | +| Senior-tech inbox | [EscalationQueuePage.tsx](frontend/src/pages/EscalationQueuePage.tsx) + [EscalationQueue.tsx](frontend/src/components/flowpilot/EscalationQueue.tsx) | +| Claim workflow | [handoff_manager.py:123 claim_session()](backend/app/services/handoff_manager.py#L123) | +| API surface | [session_handoffs.py](backend/app/api/endpoints/session_handoffs.py) — POST /handoff, /claim, GET queue | +| AI assessment for senior | `_generate_ai_assessment()` in handoff_manager | +| PSA round-trip | `escalation_package_markdown`, `escalation_package_external_id` | + +**Real engineering scope (~6-9 days):** + +1. **Notification dual-path** (4-5 days). `notification_sent` flag is a dead column — + never written. Wire two channels in `handoff_manager.create_handoff`: + - **Email** (existing `EmailService.send_notification_email`) — handles offline seniors. + - **WebSocket / SSE push** to the EscalationQueue for live demo magic moment. + - Set `notification_sent=true` after dispatch confirmation. + - Graceful degradation: handoff still created if notification raises (regression test required). + +2. **Hero metric endpoint** (~2 hours). New `GET /api/v1/analytics/escalation-metrics`, + account-scoped, role-gated to `require_engineer_or_admin`. Computes + *minutes recovered per escalation* by querying: + ``` + ai_session_step.created_at (first row by senior_tech_user_id where created_at > SessionHandoff.claimed_at) + minus + SessionHandoff.claimed_at + ``` + Returns a rolling-30-day average per account. No schema change. + +3. **UX polish on EscalationQueue + receiving-engineer view** (2-3 days). Confirm the + magic-moment screen lands when senior clicks claim. Add an unread indicator on + the queue. Wire optimistic insert when SSE event arrives. + +4. **Loom + landing page copy** (1-2 days). Non-engineering. Outside this plan's scope + but required for the GTM in week 3. + +**Test plan:** 100% coverage of new paths — 13 tests including 4 e2e and 1 regression +(graceful-degradation when notification dispatch raises). Test plan artifact at +`~/.gstack/projects/chihlasm-resolutionflow/abc-main-eng-review-test-plan-20260427-000000.md`. + +**Risk:** Low. Single feature, single metric, architecture-aligned. The dual-path +notification is the only mildly novel surface; both halves use existing infra. + +**Reuses:** `services/handoff_manager.py`, `services/escalation_package_generator.py`, +`models/session_handoff.py`, `models/ai_session.py`, `services/notification_service.py`, +`models/notification_log.py`, EmailService, EscalationQueuePage + EscalationQueue. + +### UI Specifications (locked by /plan-design-review 2026-04-27) + +**Magic-moment screen** (new, after Pick Up click): dedicated handoff-context view that +loads BEFORE the regular FlowPilot session view, then dissolves on first senior action. +Four sections, single frame: + +1. **Problem summary** (top, 2-3 lines): junior's framing. Bricolage Grotesque h2. +2. **What's been tried** (left or middle column): structured list of `dead_ends_flagged[]` + and `steps_attempted[]` from `escalation_package` JSONB. Card-flat surface, IBM Plex. +3. **AI assessment** (right column): `ai_assessment_data` rendered as 3 fields — + `likely_cause`, `suggested_steps[]`, `confidence`. accent-dim badge for confidence. +4. **Start here** (primary CTA, electric-blue, ≥44px touch target): opens FlowPilot + session at the most-likely-next-step. Senior typing or clicking anywhere triggers + 200ms fade-out and FlowPilot view fades in. Re-openable via "Show handoff context" + ghost button in FlowPilot toolbar. + +**Hero metric ("minutes recovered per escalation"):** lives in TWO places: +- **Queue stat-card** (above EscalationQueue list on /escalations): compact, "X.X hrs + saved this month" + "click for details" affordance. Refreshes on queue load. +- **Dedicated `/analytics/escalations` page** (owner-facing): trend chart (4-week + rolling), per-tech breakdown, per-problem-domain segmentation. Engineer-or-admin + role-gated. + +**Real-time arrival visual** (when WebSocket pushes a new escalation): +- New card slides in from above the list, 200ms ease-out CSS transition. +- Browser tab title prefixes with " (1) " / " (N) " when tab is backgrounded; clears + on focus. +- No sound. MUST respect `prefers-reduced-motion: reduce` (slide-in collapses to + instant fade-in). + +**Unread state:** subtle 6px dot in top-right corner of card for escalations the +current senior has never opened. Dot fades on first hover or click. + +**Race-condition (two seniors click Pick Up simultaneously):** loser sees a toast +"Already claimed by [name] 2s ago" via existing `@/lib/toast`; the card flashes the +winner's name in the meta row for 1s, then dissolves from the loser's view via +optimistic update + WebSocket reconciliation. + +**Unread state (Codex correction 2026-04-27):** dot indicator clears on **open, +claim, or explicit dismiss** — NOT on hover. Hover-to-clear is a bad proxy for +acknowledgment because incidental mouse movement creates false clears. + +**Notification routing (Codex finding 2026-04-27):** v1 fans out the email + push +to **all engineer-or-admin role users in the same account_id as the SessionHandoff**. +No on-call/round-robin logic in v1. If pilots ask for routing, capture as v2 TODO. +The first senior to claim wins; everyone else's notification self-resolves on +WebSocket reconciliation. + +**Notification delivery model (Codex correction 2026-04-27):** drop the +`notification_sent: bool` flag from v1. Replace with per-channel delivery rows +in a new `notification_log` table (already exists — reuse, don't add a new model) +keyed by `(handoff_id, channel, recipient_user_id, status)` where status ∈ +{queued, sent, failed, suppressed}. This makes partial-success and per-channel +retry visible. If the existing `notification_log` schema doesn't match, defer +the per-channel persistence to a v2 TODO and v1 logs delivery attempts to the +existing telemetry stream instead. Do NOT keep the dead boolean. + +**"Start here" CTA (Codex correction 2026-04-27):** opens the FlowPilot session +at the **latest known state** (the AI's most recent agent_message + the current +pending_task_lane). Surface `ai_assessment_data.suggested_steps[]` as a list of +chips below the chat input — clicking a chip prefills the input. Do NOT invent a +"jump to most-likely-next-step" capability that doesn't exist in the session model. + +**`/claim` role gate (Codex correction 2026-04-27, IN-SCOPE for v1):** add +`require_engineer_or_admin` dep on POST `/handoffs/{id}/claim`. Originally +deferred to TODO during eng review; Codex correctly flagged it as wedge-relevant +because the race-condition story depends on auth gating. ~30 min change. Removed +from TODO.md. + +**A11y requirements (mandatory before pilot ship):** +- Keyboard: Tab order through queue cards; Enter on focused card opens it; Pick Up + button is a reachable target; Esc closes the handoff-context overlay. +- ARIA: `role="region"` + `aria-live="polite"` on the queue list (announces arrivals); + `aria-label="N escalations awaiting pickup"` on the heading; the slide-in animation + must not announce twice (debounce live-region updates). +- Pick Up button: bump from `py-2` to `py-2.5` to clear the 44px touch-target floor. +- Color contrast: confidence-badge text on accent-dim background must be ≥4.5:1 + (verify against DESIGN-SYSTEM.md tokens). + +**DS token discipline:** every new piece must use `card-flat`, `accent-dim`/`accent-text`, +`text-muted-foreground`, `bg-card`/`bg-elevated`, IBM Plex / Bricolage / JetBrains, +explicit `transition` property lists (never `transition: all`). No glass, no blur, +no gradient surfaces. Electric-blue accent reserved for interactive elements only. + +**Mobile responsive:** deferred to post-pilot TODO. Pre-PMF wedge target is desktop; +MSP techs work on laptops/desktops in shop environments. + +**Deferred to TODO.md (out of scope for v1 wedge):** +- Peer-tech escalates colleague's session (currently session-owner-only) +- Role gate on POST /claim (currently any authenticated user in account) + +### Approach B: Full Structured Resolution loop (split B1 + B2) + +End-to-end demo: tech opens FlowPilot, structure appears in side panel as AI +responds, ticket notes auto-populate at end, optional runbook capture for reusable +patterns. Tells the full "senior engineer over your shoulder" story. + +**B1 — Side panel + PSA-formatted ticket notes** (ships first): +- Structured side panel that surfaces parsed AI markers as live actionable steps + while the conversation runs. +- PSA-formatted ticket-notes exporter (ConnectWise first; Autotask/Halo later). +- Effort: M (~3 weeks). + +**B2 — Runbook offer-and-save** (gated on pilot demand): +- "Save this resolution as a flow?" prompt at session end, with auto-drafted + runbook from the structured session state. +- Effort: S (~1 week). Don't build until at least 2 pilot customers explicitly + ask for it. + +- **Risk:** Medium. The structured-output panel quality is the whole demo. If it + looks dumb, the demo dies. +- **Reuses:** FlowPilot, unified_chat_service, FlowProposal, ConnectWise PSA + integration. + +### Approach C: Senior-Tech Time-Saved Counter + +Continuous measurement layer underneath A and B. Every session contributes an +estimated minutes-saved number. Owner-facing dashboard quotes "this month your +shop saved N hours of senior-tech time." Sells to MSP owner with verifiable ROI. + +- **Effort:** S (~1 week + ongoing measurement methodology refinement). +- **Risk:** Medium-low. Methodology has to be defensible. If numbers look + made-up, trust dies fast. +- **Reuses:** FlowPilot telemetry, session metadata, account-scoped analytics. + +## Recommended Approach + +**A first (1-2 weeks), then B (3-4 weeks after A ships), with C running underneath +both as a continuous backdrop.** + +Sequence rationale: + +- **A is the sharpest possible 2-minute demo.** Single feature, single metric, + buyer-verifiable in their own data. Get it in front of 5 MSPs in week 3. +- **B is the depth play.** Once Approach A has produced first-pilot signal, + Approach B's full structured-resolution loop becomes the "what we ship next" that + retains pilots and converts them to paid. +- **C compounds across both.** Every session under A or B contributes to the + time-saved counter. By week 6 there are real numbers to put in front of an MSP + owner — turning a senior-tech-led pilot into an owner-signed contract. + +This sequence is non-negotiable. Building B before A is the classic pre-PMF trap of +perfecting product before validating GTM. Building C alone is measurement without a +demo to anchor it. + +## Pricing + +**Pilot pricing (first 3-5 customers): $39/seat/month, monthly billing, +month-to-month.** Anchored against IT Glue (~$29/tech), Hudu (~$25/tech), +Liongard (~$3/endpoint). The premium over IT Glue/Hudu reflects the active-session +value (vs. their static-runbook value) — 30% above the runbook-only category. + +Customer #6+ pricing is an Open Question (revisit after 3 pilots produce real +hours-saved data; price up if the per-seat ROI is over $200/seat/mo). + +## Open Questions + +1. **Free-tier shape.** Should the time-saved counter be free forever as a + distribution lever, with paid for the structuring + escalation? Land-and-expand + pattern. Decide after 3 pilot conversions. +2. **PSA-marketplace timing.** ConnectWise Marketplace listing requires partnership + onboarding (~6-week cycle). Submit application week 5; expect listing live by + week 11. Don't gate launch on it. +3. **Customer #6+ pricing.** Revisit after 3 pilot customers produce verifiable + hours-saved numbers. + +## Deferred (YAGNI until 10 paying customers) + +- HIPAA / SOC2 audit positioning. Pre-PMF is too early; revisit when a regulated- + vertical MSP asks for it explicitly. +- Multi-PSA depth (Autotask, Halo). ConnectWise alone covers ~40% of the SMB MSP + market and is sufficient for first 5-10 customers. +- Cross-tenant pattern detection. The data-flywheel-across-shops play is at least + 6 months out; building it before single-shop ROI is proven is premature. + +## Success Criteria (revised for realism) + +- **Week 3:** Approach A shipped. 3 MSPs in active free pilot (cap at 3 to + preserve B1 build capacity). +- **Weeks 3-6:** Pilot management dominates. B1 build is paused; founder runs + pilot calls, captures bug reports, iterates UX. Stripe seat-based billing is + set up in week 5. +- **Week 6:** First verbal commit from a pilot customer. Verified + minutes-recovered-per-escalation number from at least 2 pilots. +- **Week 8:** First paid customer (procurement cycles run 4-6 weeks even at small + MSPs; 2 weeks from verbal commit to signed contract is realistic). Time-saved + counter (Approach C) producing dashboard-quality data. +- **Week 11:** B1 (side panel + PSA notes) shipped. 3-5 paying customers. First + MSP-owner-led conversation. ConnectWise Marketplace listing live. +- **Quarter end:** $5K MRR or 10 paying customers, whichever comes first. Loom + demos posted publicly to r/msp and MSPGeek. + +## Distribution Plan (week-by-week cadence) + +- **Week 3:** Escalation Mode demo Loom posted. r/msp launch post. +- **Week 4:** MSPGeek Discord AMA scheduled. RocketMSP newsletter pitch sent. +- **Week 5:** ConnectWise Marketplace listing application submitted. Stripe + billing live for paid conversion. +- **Week 6:** First "guest on Inside MSP podcast" outreach. Second r/msp post + (case study from a pilot, anonymized). +- **Week 7-8:** Pilot conversion calls. First paying customer. +- **Week 9-11:** B1 ships. Owner-targeted demo Loom. Second podcast outreach. + +**Founder-led pilot:** The first 3-5 customers come from the founder's existing +MSP network. Treat them as design partners; expect to ship feature requests +weekly during pilot. Cap at 3 active pilots until B1 ships. + +**Tech audience channels:** r/msp, r/sysadmin, MSPGeek Discord, RocketMSP +newsletter, Inside MSP podcast. +**Owner audience channels:** ConnectWise Marketplace, MSP-focused Substacks, +RIA Vendor Roundup. + +CI/CD: existing Railway auto-deploy via GitHub mirror. No new pipeline needed. + +## Dependencies + +- **Session-state serialization (Approach A blocker).** Schema design + migration + is the longest-lead engineering task. 3-5 days budget. Do this first. +- **Stripe seat-based billing (week 5 task).** No billing infrastructure exists + today. ~3-5 days of work for monthly subscriptions + invoice flow. Block on + this before week-8 first-paid milestone. +- **ConnectWise PSA integration depth.** Sufficient for ticket-notes auto-export + (Approach B1). Autotask and Halo wait until first 5 paying ConnectWise + customers. +- **Authentication.** Existing JWT + role hierarchy is sufficient for senior-tech + inbox view; no new auth work needed. + +## Risks and Kill-Switch + +- **Risk: Session-state serialization design churn.** If the schema needs to + change after pilot feedback, every saved session has to migrate. Mitigation: + keep schema versioned and forward-compatible from day 1. +- **Risk: Pilot-to-paid conversion slower than 4-6 weeks.** MSP procurement is + notoriously slow. Mitigation: get verbal commits in writing; price as + month-to-month with no annual contract to lower the buying friction. +- **Risk: ConnectWise ships an equivalent feature in their 2026.x release.** + Mitigation: lead the marketing on "we're independent of your PSA" — works with + any PSA, not just ConnectWise. The founder's PSA-agnostic FlowPilot is an + asset here. +- **Kill-switch criterion:** if 0 of 3 pilots produce a verifiable + hours-saved-per-week number above 1.0 by week 8, **revisit the wedge**. The + product may need to pivot to deterministic-ops territory (Read 1 from the + session) or be repositioned. Don't sink another quarter into the current GTM + story without this number. + +## The Assignment + +**This week, before any code:** + +Time-track the next 5 escalations in your shop manually. For each, capture: +1. Time the senior tech opens the ticket +2. Time the senior tech takes their first diagnostic action (not counting the + verbal "tell me what you tried" warm-up) +3. The delta — that's the wasted time per escalation today + +Average those 5 numbers. **That's the hero stat in your first sales conversation:** +"Senior techs at our shop wasted N minutes per escalation just getting up to +speed. We built the thing that takes that to zero." + +Don't try to pull this from telemetry — the doc itself notes that retrieval/re-use +data isn't queryable yet. Manual stopwatch on the next 5 escalations is the +fastest path to a defensible number. + +This is the assignment because it forces the GTM story into the same time-zone as +the build, and it's a one-day effort that compounds for every conversation +afterward. + +## What I noticed about how you think + +- You contradicted my framing twice in the same session and the second + contradiction was sharper than the first. Most founders agree with the + diagnostic and walk out with a polished version of what they came in with. You + said "I'm just questioning if flows are even the way to go" — and that + sentence reset the entire wedge. That's craft. + +- "The senior engineer looking over your shoulder" came out of you spontaneously, + not as a prepared pitch. That's the line. Use it. It survives because it's + emotional truth (every junior tech has had this, every senior tech has been + this), not constructed marketing copy. + +- You're solving your own problem with your own time. 20 hrs/week isn't a + hypothetical user pain — it's your Tuesday. Founders who solve their own pain + ship sharper products because the feedback loop is instant. + +- The escalation feature emerged from your description, not mine. I was busy + cataloging documentation pains. You said "junior to senior escalation? no + worries there either" almost as an afterthought. That afterthought is the wedge. + Pay attention to which features you describe casually versus which you push hard + on — the casual ones are sometimes where the truth lives. + +## GSTACK REVIEW REPORT + +| Review | Trigger | Why | Runs | Status | Findings | +|--------|---------|-----|------|--------|----------| +| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | not run | +| Codex Review | `/codex review` | Independent 2nd opinion | 1 | INFO | 12 findings, 6 applied, 1 partial, 5 rejected | +| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR (PLAN) | 2 issues, 0 critical gaps, scope reduced | +| Design Review | `/plan-design-review` | UI/UX gaps | 1 | CLEAR (FULL) | score 6/10 → 9/10, 8 decisions | +| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not run | + +- **CODEX:** 12 findings reviewed. Applied: 2-metric framing (#2), notification routing spec (#3), per-channel delivery model (#4), unread-state fix (#11), Start-here CTA reframe (#9), claim role gate moved in-scope (#8). Rejected: full scope reduction to PSA-brief-only (#6/7/12 — user kept queue UI as demo hero). Partial: scope concern (#5) acknowledged in eng review's email-first/polling-fallback. Misread: #1, #10. +- **CROSS-MODEL:** Claude (eng + design reviews) and Codex agree on 6/12 findings. The major disagreement was scope — Codex argued for cutting the queue UI, user rejected. Both agree on metric definition, notification routing, claim auth gating. +- **UNRESOLVED:** 0 +- **VERDICT:** ENG + DESIGN CLEARED, CODEX REVIEWED — ready to implement. diff --git a/docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md b/docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md new file mode 100644 index 00000000..c37be618 --- /dev/null +++ b/docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md @@ -0,0 +1,33 @@ +# Test Plan +Generated by /plan-eng-review on 2026-04-27 +Branch: main +Repo: chihlasm/resolutionflow + +## Affected Pages/Routes + +- `/escalations` ([EscalationQueuePage.tsx](frontend/src/pages/EscalationQueuePage.tsx)) — senior-tech inbox view; verify queue list, real-time arrival, click-through +- `/pilot/:session_id` (FlowPilotSessionPage) — verify post-claim load shows full escalation context (snapshot, ai_assessment, escalation_package) +- `GET /api/v1/analytics/escalation-metrics` (NEW) — verify hero metric calculation, account-scoping, role gate + +## Key Interactions to Verify + +- Junior tech clicks **Escalate** in active FlowPilot session → handoff is created → notification fires → senior sees escalation in queue within 30 seconds +- Senior tech clicks **Claim** in queue → session reactivates → senior is redirected into FlowPilot session view → ai_assessment + snapshot are visible +- Senior types first message in chat after claim → metric query starts attributing time-to-first-action +- MSP owner opens analytics page → "minutes recovered per escalation" widget shows current month's rolling average + +## Edge Cases + +- **Two seniors race to claim** the same handoff → one wins, the other gets a "Already claimed by [name]" message +- **Senior is offline** when escalation fires → email arrives via existing `EmailService.send_notification_email` +- **WebSocket disconnects mid-session** → frontend reconnects; missed events backfilled by re-fetching the queue +- **Notification dispatch raises** (SMTP down, WebSocket fanout fails) → handoff is still created (graceful degradation) +- **Senior takes non-chat action first** (e.g., posts directly to PSA) → metric falls back to PSA writeback timestamp or remains null; doc the chosen behavior +- **Account-scoped multi-tenancy** → senior at MSP A cannot see escalations from MSP B (Phase 4 RLS) +- **Role gate on metric endpoint** → only `engineer_or_admin` can hit `/escalation-metrics` + +## Critical Paths + +1. **Magic-moment demo flow** (the entire Loom): junior escalate → senior notification → senior claim → session view → first action recorded → metric updates +2. **Email fallback** when senior is offline — must not silently drop +3. **Regression: handoff creation succeeds even if notification dispatch raises** — graceful degradation is mandatory -- 2.49.1 From 52f6d0308fc7ebfb073d6d5fd5367322b5c23a4b Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 15:25:46 -0400 Subject: [PATCH 02/34] feat(analytics): add escalation time-to-first-action metric endpoint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GET /api/v1/analytics/flowpilot/escalations?period={7d,30d,90d} Computes the in-product wedge metric for Escalation Mode: average / median / p95 seconds between SessionHandoff.claimed_at and the first ai_session_step created on the same session after that timestamp. Account-scoped, role-gated to engineer-or-admin. The metric is intentionally NOT called "minutes recovered" — that's the two-metric framing locked by /codex review: this in-product number must be paired with manual baseline (the verbal-handoff stopwatch from The Assignment) to produce the savings claim. Schema's `metric_definition` field surfaces the disclaimer in every response so callers don't oversell it. Implementation notes: - Uses correlated scalar subquery for first-step-after-claim per handoff, aggregates avg/median/p95 in Python (~1k rows/account/month is well within budget; cleaner than percentile_cont gymnastics in SQL) - Excludes unclaimed handoffs (claimed_at IS NULL) - Counts claimed-but-no-action handoffs in n_handoffs_claimed but not in n_handoffs_with_action — surfaces the conversion-rate signal - Floors negative deltas at 0 to handle clock-drift edge cases Tests cover happy path, zero-data, claimed-but-no-action accounting, period window filtering, multi-handoff aggregation, multi-tenant isolation (Phase 4 RLS landmine pattern), viewer-role 403 gate, and period validation. 9 tests, all green. No regressions in existing handoff_manager / session_handoffs suites. First piece of the Approach A wedge build per docs/plans/2026-04-27-escalation-mode-wedge-design.md. Unblocks the queue stat-card and the analytics page. Co-Authored-By: Claude Opus 4.7 --- .../app/api/endpoints/flowpilot_analytics.py | 113 +++++- backend/app/schemas/flowpilot_analytics.py | 23 ++ .../test_flowpilot_analytics_escalations.py | 363 ++++++++++++++++++ 3 files changed, 498 insertions(+), 1 deletion(-) create mode 100644 backend/tests/test_flowpilot_analytics_escalations.py diff --git a/backend/app/api/endpoints/flowpilot_analytics.py b/backend/app/api/endpoints/flowpilot_analytics.py index 66a322bb..870b434f 100644 --- a/backend/app/api/endpoints/flowpilot_analytics.py +++ b/backend/app/api/endpoints/flowpilot_analytics.py @@ -3,8 +3,10 @@ Endpoints: GET /analytics/flowpilot?period=30d — Main dashboard data GET /analytics/flowpilot/knowledge-gaps — Knowledge gap report + GET /analytics/flowpilot/escalations?period=30d — Escalation handoff metrics """ import logging +import statistics from datetime import datetime, timezone, timedelta from typing import Annotated, Optional @@ -13,10 +15,17 @@ from sqlalchemy import select, func, case, cast, Date, extract from sqlalchemy.ext.asyncio import AsyncSession from app.core.rate_limit import limiter -from app.api.deps import get_current_active_user, get_db, require_team_admin +from app.api.deps import ( + get_current_active_user, + get_db, + require_engineer_or_admin, + require_team_admin, +) from app.models.user import User from app.models.tree import Tree from app.models.ai_session import AISession +from app.models.ai_session_step import AISessionStep +from app.models.session_handoff import SessionHandoff from app.models.flow_proposal import FlowProposal from app.models.psa_activity_log import PsaActivityLog from app.models.psa_post_log import PsaPostLog @@ -36,6 +45,7 @@ from app.schemas.flowpilot_analytics import ( EnhancedPsaMetrics, PsaFunnel, PsaDailyTrend, + EscalationMetrics, ) from app.services.knowledge_gap_service import get_knowledge_gaps, KnowledgeGapReport @@ -727,3 +737,104 @@ async def get_enhanced_psa_metrics( push_funnel=push_funnel, daily_trend=daily_trend, ) + + +# ─── Escalation Mode metrics (wedge stat for /escalations queue + analytics page) +# +# Pulls all (handoff.claimed_at, first_step_after_claim.created_at) pairs in the +# window and aggregates avg/median/p95 of the delta in Python. Pilot scale +# (~1k rows max per account per month) makes this cheaper and clearer than +# Postgres percentile_cont gymnastics. +# +# IMPORTANT: this is the in-product metric only. The "minutes recovered" +# sales claim requires manual baseline measurement (see The Assignment in +# docs/plans/2026-04-27-escalation-mode-wedge-design.md). + + +@router.get("/escalations", response_model=EscalationMetrics) +@limiter.limit("30/minute") +async def get_escalation_metrics( + request: Request, + current_user: Annotated[User, Depends(get_current_active_user)], + db: Annotated[AsyncSession, Depends(get_db)], + _: None = Depends(require_engineer_or_admin), + period: str = Query("30d", pattern="^(7d|30d|90d)$"), +) -> EscalationMetrics: + """Time-to-first-action after escalation claim, account-scoped. + + Returns: + n_handoffs_claimed: handoffs in window that were claimed by a senior. + n_handoffs_with_action: subset where the senior took at least one + action (an ai_session_step row created after claimed_at). + avg/median/p95_seconds_to_first_action: aggregates of + (first_step.created_at - claimed_at) in seconds. + + Excludes handoffs where claimed_at IS NULL (never claimed) and handoffs + where no ai_session_step was created after the claim. Both are + counted — n_handoffs_claimed includes "no action yet" handoffs so the + conversion rate is visible. + """ + if not current_user.account_id: + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, detail="No account" + ) + + account_id = current_user.account_id + period_start = _get_period_start(period) + + # First-action timestamp per handoff via correlated scalar subquery. + first_action_subq = ( + select(func.min(AISessionStep.created_at)) + .where( + AISessionStep.session_id == SessionHandoff.session_id, + AISessionStep.created_at > SessionHandoff.claimed_at, + ) + .correlate(SessionHandoff) + .scalar_subquery() + ) + + rows = ( + await db.execute( + select( + SessionHandoff.claimed_at, + first_action_subq.label("first_action_at"), + ).where( + SessionHandoff.account_id == account_id, + SessionHandoff.claimed_at.isnot(None), + SessionHandoff.claimed_at >= period_start, + ) + ) + ).all() + + n_handoffs_claimed = len(rows) + deltas: list[float] = [] + for claimed_at, first_action_at in rows: + if first_action_at is None: + continue + delta_s = (first_action_at - claimed_at).total_seconds() + # Floor at zero — clock drift between rows could in theory yield a + # tiny negative if a step's created_at races claimed_at. Surface as + # 0s rather than absurd negative deltas. + if delta_s < 0: + delta_s = 0.0 + deltas.append(delta_s) + + n_handoffs_with_action = len(deltas) + if n_handoffs_with_action == 0: + return EscalationMetrics( + period=period, + n_handoffs_claimed=n_handoffs_claimed, + n_handoffs_with_action=0, + ) + + sorted_deltas = sorted(deltas) + p95_idx = max(0, int(round(0.95 * (n_handoffs_with_action - 1)))) + + return EscalationMetrics( + period=period, + n_handoffs_claimed=n_handoffs_claimed, + n_handoffs_with_action=n_handoffs_with_action, + avg_seconds_to_first_action=round(statistics.fmean(deltas), 2), + median_seconds_to_first_action=round(statistics.median(deltas), 2), + p95_seconds_to_first_action=round(sorted_deltas[p95_idx], 2), + ) diff --git a/backend/app/schemas/flowpilot_analytics.py b/backend/app/schemas/flowpilot_analytics.py index b3155283..410f5141 100644 --- a/backend/app/schemas/flowpilot_analytics.py +++ b/backend/app/schemas/flowpilot_analytics.py @@ -124,3 +124,26 @@ class FlowPilotDashboard(BaseModel): confidence_breakdown: ConfidenceBreakdown knowledge_coverage: KnowledgeCoverage psa_metrics: PsaMetrics | None = None + + +class EscalationMetrics(BaseModel): + """In-product time-to-first-action metric for the Escalation Mode wedge. + + NOTE: this is the *in-product* metric (post-claim time-to-first-action). The + "minutes recovered" sales claim requires a manual baseline measurement of the + pre-Escalation-Mode verbal-handoff time. See + docs/plans/2026-04-27-escalation-mode-wedge-design.md for the two-metric + framing — do not roll this number alone into "minutes recovered." + """ + + period: str + n_handoffs_claimed: int + n_handoffs_with_action: int + avg_seconds_to_first_action: float | None = None + median_seconds_to_first_action: float | None = None + p95_seconds_to_first_action: float | None = None + metric_definition: str = ( + "elapsed_seconds(first ai_session_step in session where " + "created_at > SessionHandoff.claimed_at) — measures post-claim activity " + "lag, NOT verbal-handoff savings. Pair with manual baseline." + ) diff --git a/backend/tests/test_flowpilot_analytics_escalations.py b/backend/tests/test_flowpilot_analytics_escalations.py new file mode 100644 index 00000000..18b30212 --- /dev/null +++ b/backend/tests/test_flowpilot_analytics_escalations.py @@ -0,0 +1,363 @@ +"""Tests for GET /analytics/flowpilot/escalations — Escalation Mode wedge metric. + +Covers the in-product time-to-first-action measurement that powers the queue +stat-card and the analytics page. The savings claim itself comes from the +manual baseline (the Assignment); these tests only cover what the in-product +endpoint returns. +""" +from datetime import datetime, timedelta, timezone +from uuid import UUID as PyUUID + +import pytest +from httpx import AsyncClient +from sqlalchemy import select + +from app.models.ai_session import AISession +from app.models.ai_session_step import AISessionStep +from app.models.session_handoff import SessionHandoff +from app.models.user import User + + +URL = "/api/v1/analytics/flowpilot/escalations" + + +# ─── Helpers ────────────────────────────────────────────────────────────────── + + +async def _make_session(db, *, user_id, account_id) -> AISession: + s = AISession( + user_id=user_id, + account_id=account_id, + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="escalated", + confidence_tier="discovery", + conversation_messages=[], + ) + db.add(s) + await db.flush() + return s + + +async def _make_handoff( + db, + *, + session_id, + account_id, + user_id, + claimed_at: datetime | None, + claimed_by=None, +) -> SessionHandoff: + h = SessionHandoff( + session_id=session_id, + account_id=account_id, + handed_off_by=user_id, + intent="escalate", + snapshot={"branch_map": "stub"}, + priority="normal", + claimed_at=claimed_at, + claimed_by=claimed_by, + ) + db.add(h) + await db.flush() + return h + + +async def _make_step(db, *, session_id, account_id, created_at: datetime) -> AISessionStep: + """Insert an ai_session_step row with an explicit created_at. + + SQLAlchemy's default would set created_at to now(); the metric query keys + off this column so the tests need to control it directly. + """ + step = AISessionStep( + session_id=session_id, + account_id=account_id, + step_order=1, + step_type="note", + content={"text": "first action"}, + confidence_at_step=0.5, + input_tokens=0, + output_tokens=0, + is_fork_point=False, + was_free_text=False, + was_skipped=False, + created_at=created_at, + ) + db.add(step) + await db.flush() + return step + + +# ─── Tests ──────────────────────────────────────────────────────────────────── + + +@pytest.mark.asyncio +async def test_returns_zero_metrics_when_no_handoffs( + client: AsyncClient, auth_headers, test_user +): + """Empty account → n_handoffs_claimed=0, all stats None, 200 OK.""" + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 200 + body = response.json() + assert body["period"] == "30d" + assert body["n_handoffs_claimed"] == 0 + assert body["n_handoffs_with_action"] == 0 + assert body["avg_seconds_to_first_action"] is None + assert body["median_seconds_to_first_action"] is None + assert body["p95_seconds_to_first_action"] is None + # Disclaimer is part of the contract — pilots reading the API should see it. + assert "manual baseline" in body["metric_definition"].lower() + + +@pytest.mark.asyncio +async def test_happy_path_single_handoff_with_action( + client: AsyncClient, auth_headers, test_user, test_db +): + """One claimed handoff + a step 90s later → avg=median=p95=90.0.""" + user_id = PyUUID(test_user["user_data"]["id"]) + account_id = PyUUID(test_user["user_data"]["account_id"]) + + claimed_at = datetime.now(timezone.utc) - timedelta(hours=2) + first_action_at = claimed_at + timedelta(seconds=90) + + session = await _make_session(test_db, user_id=user_id, account_id=account_id) + await _make_handoff( + test_db, + session_id=session.id, + account_id=account_id, + user_id=user_id, + claimed_at=claimed_at, + claimed_by=user_id, + ) + await _make_step( + test_db, + session_id=session.id, + account_id=account_id, + created_at=first_action_at, + ) + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 200 + body = response.json() + assert body["n_handoffs_claimed"] == 1 + assert body["n_handoffs_with_action"] == 1 + assert body["avg_seconds_to_first_action"] == 90.0 + assert body["median_seconds_to_first_action"] == 90.0 + assert body["p95_seconds_to_first_action"] == 90.0 + + +@pytest.mark.asyncio +async def test_handoff_claimed_but_no_action( + client: AsyncClient, auth_headers, test_user, test_db +): + """Claimed handoff with no post-claim step → counted in n_handoffs_claimed + but not in n_handoffs_with_action; aggregates remain None.""" + user_id = PyUUID(test_user["user_data"]["id"]) + account_id = PyUUID(test_user["user_data"]["account_id"]) + claimed_at = datetime.now(timezone.utc) - timedelta(minutes=5) + + session = await _make_session(test_db, user_id=user_id, account_id=account_id) + await _make_handoff( + test_db, + session_id=session.id, + account_id=account_id, + user_id=user_id, + claimed_at=claimed_at, + claimed_by=user_id, + ) + # Pre-claim step (created_at < claimed_at) — must NOT count. + await _make_step( + test_db, + session_id=session.id, + account_id=account_id, + created_at=claimed_at - timedelta(seconds=30), + ) + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 200 + body = response.json() + assert body["n_handoffs_claimed"] == 1 + assert body["n_handoffs_with_action"] == 0 + assert body["avg_seconds_to_first_action"] is None + + +@pytest.mark.asyncio +async def test_unclaimed_handoffs_excluded( + client: AsyncClient, auth_headers, test_user, test_db +): + """Handoffs with claimed_at IS NULL are excluded entirely.""" + user_id = PyUUID(test_user["user_data"]["id"]) + account_id = PyUUID(test_user["user_data"]["account_id"]) + + session = await _make_session(test_db, user_id=user_id, account_id=account_id) + await _make_handoff( + test_db, + session_id=session.id, + account_id=account_id, + user_id=user_id, + claimed_at=None, + ) + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 200 + assert response.json()["n_handoffs_claimed"] == 0 + + +@pytest.mark.asyncio +async def test_period_window_excludes_old_handoffs( + client: AsyncClient, auth_headers, test_user, test_db +): + """A handoff claimed >7d ago must not appear in ?period=7d.""" + user_id = PyUUID(test_user["user_data"]["id"]) + account_id = PyUUID(test_user["user_data"]["account_id"]) + + old_claimed_at = datetime.now(timezone.utc) - timedelta(days=10) + session = await _make_session(test_db, user_id=user_id, account_id=account_id) + await _make_handoff( + test_db, + session_id=session.id, + account_id=account_id, + user_id=user_id, + claimed_at=old_claimed_at, + claimed_by=user_id, + ) + await _make_step( + test_db, + session_id=session.id, + account_id=account_id, + created_at=old_claimed_at + timedelta(seconds=60), + ) + await test_db.commit() + + # 7d window: excluded + r7 = await client.get(URL, headers=auth_headers, params={"period": "7d"}) + assert r7.status_code == 200 + assert r7.json()["n_handoffs_claimed"] == 0 + + # 90d window: included + r90 = await client.get(URL, headers=auth_headers, params={"period": "90d"}) + assert r90.status_code == 200 + assert r90.json()["n_handoffs_claimed"] == 1 + assert r90.json()["n_handoffs_with_action"] == 1 + + +@pytest.mark.asyncio +async def test_aggregate_stats_for_multiple_handoffs( + client: AsyncClient, auth_headers, test_user, test_db +): + """Three handoffs with deltas 30/60/180s → avg=90, median=60, p95≈180.""" + user_id = PyUUID(test_user["user_data"]["id"]) + account_id = PyUUID(test_user["user_data"]["account_id"]) + + base = datetime.now(timezone.utc) - timedelta(hours=3) + deltas = [30, 60, 180] + for i, delta in enumerate(deltas): + s = await _make_session(test_db, user_id=user_id, account_id=account_id) + claimed_at = base + timedelta(minutes=i * 10) + await _make_handoff( + test_db, + session_id=s.id, + account_id=account_id, + user_id=user_id, + claimed_at=claimed_at, + claimed_by=user_id, + ) + await _make_step( + test_db, + session_id=s.id, + account_id=account_id, + created_at=claimed_at + timedelta(seconds=delta), + ) + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + body = response.json() + assert response.status_code == 200 + assert body["n_handoffs_claimed"] == 3 + assert body["n_handoffs_with_action"] == 3 + assert body["avg_seconds_to_first_action"] == 90.0 + assert body["median_seconds_to_first_action"] == 60.0 + assert body["p95_seconds_to_first_action"] == 180.0 + + +@pytest.mark.asyncio +async def test_account_isolation_requesting_user_only_sees_own_account( + client: AsyncClient, auth_headers, test_user, test_db +): + """A handoff in another account must not appear in this user's response. + + Critical: the Phase 4 RLS pattern can fail silently if account_id is wrong. + This test would catch an account-scoped query that accidentally returned + cross-tenant rows. + """ + from app.models.account import Account + + other_account = Account(name="Other MSP", display_code="OTHER001") + test_db.add(other_account) + await test_db.flush() + + other_user = User( + email="other@example.com", + password_hash="x", + name="Other Tech", + role="engineer", + account_id=other_account.id, + account_role="owner", + ) + test_db.add(other_user) + await test_db.flush() + + s = await _make_session( + test_db, user_id=other_user.id, account_id=other_account.id + ) + claimed_at = datetime.now(timezone.utc) - timedelta(hours=1) + await _make_handoff( + test_db, + session_id=s.id, + account_id=other_account.id, + user_id=other_user.id, + claimed_at=claimed_at, + claimed_by=other_user.id, + ) + await _make_step( + test_db, + session_id=s.id, + account_id=other_account.id, + created_at=claimed_at + timedelta(seconds=45), + ) + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 200 + body = response.json() + # The other account's handoff must NOT leak into this account's response. + assert body["n_handoffs_claimed"] == 0 + assert body["n_handoffs_with_action"] == 0 + + +@pytest.mark.asyncio +async def test_viewer_role_is_blocked( + client: AsyncClient, test_user, auth_headers, test_db +): + """Downgrade the test user to 'viewer' and confirm the endpoint 403s.""" + user_id = PyUUID(test_user["user_data"]["id"]) + user = ( + await test_db.execute(select(User).where(User.id == user_id)) + ).scalar_one() + user.account_role = "viewer" + await test_db.commit() + + response = await client.get(URL, headers=auth_headers) + assert response.status_code == 403 + assert "engineer" in response.json()["detail"].lower() + + +@pytest.mark.asyncio +async def test_invalid_period_rejected(client: AsyncClient, auth_headers): + """period=1d is not in {7d,30d,90d} — must 422.""" + response = await client.get(URL, headers=auth_headers, params={"period": "1d"}) + assert response.status_code == 422 -- 2.49.1 From 7a5b853b3b1cd3be830e3b72461f0508d34d1c92 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 15:46:59 -0400 Subject: [PATCH 03/34] feat(api): role-gate handoff claim to engineer-or-admin POST /ai-sessions/{id}/handoffs/{hid}/claim previously required only an authenticated user, so a viewer-role account user could claim escalations. Codex review flagged this as wedge-relevant: the Escalation Mode race- condition story (two seniors clicking Pick Up simultaneously) depends on auth gating for audit integrity. Originally captured as a deferred TODO during /plan-eng-review, then moved in-scope by /codex review. Swap the dep to require_engineer_or_admin. One-line change. Two new tests: - viewer_role gets 403 with "Engineer or admin access required" - engineer/owner role still succeeds and claimed_at + claimed_by populate Existing handoff create + queue tests unaffected. Co-Authored-By: Claude Opus 4.7 --- backend/app/api/endpoints/session_handoffs.py | 12 ++- backend/tests/test_session_handoffs_api.py | 89 +++++++++++++++++++ 2 files changed, 98 insertions(+), 3 deletions(-) diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 513eefc6..2e3ec65f 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -13,7 +13,7 @@ from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession -from app.api.deps import get_current_active_user, get_db +from app.api.deps import get_current_active_user, get_db, require_engineer_or_admin from app.models.user import User from app.models.ai_session import AISession from app.models.session_handoff import SessionHandoff @@ -86,10 +86,16 @@ async def list_handoffs( async def claim_handoff( session_id: UUID, handoff_id: UUID, - current_user: Annotated[User, Depends(get_current_active_user)], + current_user: Annotated[User, Depends(require_engineer_or_admin)], db: Annotated[AsyncSession, Depends(get_db)], ) -> HandoffResponse: - """Claim a handed-off session.""" + """Claim a handed-off session. + + Role-gated to engineer/admin/owner — viewers cannot claim. The race-condition + story (two seniors clicking Pick Up simultaneously) depends on auth gating + for audit integrity. Codex review flagged this as wedge-relevant; locked + in-scope for Escalation Mode v1. + """ manager = HandoffManager(db) try: handoff = await manager.claim_session( diff --git a/backend/tests/test_session_handoffs_api.py b/backend/tests/test_session_handoffs_api.py index 26a47988..6edaac1e 100644 --- a/backend/tests/test_session_handoffs_api.py +++ b/backend/tests/test_session_handoffs_api.py @@ -1,8 +1,12 @@ """API endpoint tests for session handoffs.""" +from uuid import UUID as PyUUID + import pytest from httpx import AsyncClient +from sqlalchemy import select from app.models.ai_session import AISession +from app.models.user import User @pytest.mark.asyncio @@ -58,3 +62,88 @@ async def test_get_queue(client: AsyncClient, test_user, auth_headers, test_db): assert resp.status_code == 200 data = resp.json() assert len(data) >= 1 + + +@pytest.mark.asyncio +async def test_claim_blocked_for_viewer_role( + client: AsyncClient, test_user, auth_headers, test_db +): + """POST /handoffs/{id}/claim must 403 for viewer-role users. + + Codex review flagged the missing role gate as wedge-relevant: the + race-condition story (two seniors clicking Pick Up simultaneously) + requires auth gating for audit integrity. Viewers must not be able + to claim escalations. + """ + # Create a session + handoff as the engineer-role test_user (default = owner). + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + create_resp = await client.post( + f"/api/v1/ai-sessions/{session.id}/handoff", + headers=auth_headers, + json={"intent": "escalate", "engineer_notes": "Need help"}, + ) + assert create_resp.status_code == 201 + handoff_id = create_resp.json()["id"] + + # Downgrade the user to viewer. + user_id = PyUUID(test_user["user_data"]["id"]) + user = ( + await test_db.execute(select(User).where(User.id == user_id)) + ).scalar_one() + user.account_role = "viewer" + await test_db.commit() + + claim_resp = await client.post( + f"/api/v1/ai-sessions/{session.id}/handoffs/{handoff_id}/claim", + headers=auth_headers, + ) + assert claim_resp.status_code == 403 + assert "engineer" in claim_resp.json()["detail"].lower() + + +@pytest.mark.asyncio +async def test_claim_allowed_for_engineer_role( + client: AsyncClient, test_user, auth_headers, test_db +): + """POST /handoffs/{id}/claim succeeds for engineer-or-admin roles.""" + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + create_resp = await client.post( + f"/api/v1/ai-sessions/{session.id}/handoff", + headers=auth_headers, + json={"intent": "escalate", "engineer_notes": "Need help"}, + ) + assert create_resp.status_code == 201 + handoff_id = create_resp.json()["id"] + + # Default test_user role is "owner", which passes engineer-or-admin. + claim_resp = await client.post( + f"/api/v1/ai-sessions/{session.id}/handoffs/{handoff_id}/claim", + headers=auth_headers, + ) + assert claim_resp.status_code == 200 + assert claim_resp.json()["claimed_by"] == test_user["user_data"]["id"] + assert claim_resp.json()["claimed_at"] is not None -- 2.49.1 From 07d0db9579f81e9292c5e91e140543bda3beb292 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 15:58:05 -0400 Subject: [PATCH 04/34] feat(handoff): email engineer-or-admin teammates on escalation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First half of the Escalation Mode notification dual-path. WebSocket/SSE push is the second half (next commit) — email handles offline seniors, push handles online ones for the magic-moment demo. HandoffManager.dispatch_escalation_notifications: - Pulls active engineer/admin/owner-role users in the same account_id (excludes the escalator + viewers + soft-deleted) - Sends via existing EmailService.send_notification_email, concurrent via asyncio.gather; per-message failures don't block the rest - Wrapped in try/except: any exception is logged + swallowed. Handoff creation is authoritative; notification is advisory. This is the graceful-degradation regression both eng + codex reviews flagged as critical (handoff must succeed even if SMTP is down). Endpoint wiring (POST /ai-sessions/{id}/handoff): - Dispatch fires AFTER db.commit() — never email about a rolled-back handoff. Trust-erosion bug if we got that wrong. - Only fires for intent=escalate. Park is private to the escalator. Tests (4 new): - emails-engineer-recipients-in-account: viewer excluded, escalator excluded, only the engineer/admin teammates get the message - skipped-for-park-intent: park doesn't fan out - graceful-degradation-when-email-raises: RuntimeError from the email service does NOT bubble out of dispatch - endpoint-dispatches-on-escalate: end-to-end wiring through POST Per-channel delivery records (replacing the dead `notification_sent` boolean per Codex correction) is a v1.x story — for now application logs are the audit trail. See docs/plans/2026-04-27-escalation-mode-wedge-design.md. 20 tests green across handoff_manager + session_handoffs_api + flowpilot_analytics_escalations. No regressions. Co-Authored-By: Claude Opus 4.7 --- backend/app/api/endpoints/session_handoffs.py | 7 + backend/app/services/handoff_manager.py | 100 +++++++++ backend/tests/test_handoff_manager.py | 208 ++++++++++++++++++ 3 files changed, 315 insertions(+) diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 2e3ec65f..5e444bd2 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -63,6 +63,13 @@ async def create_handoff( raise HTTPException(status_code=400, detail=str(e)) await db.commit() + + # Best-effort notification dispatch AFTER commit so we never email about + # a rolled-back handoff. Failures are swallowed inside the manager — + # handoff creation is authoritative; notifications are advisory. + if handoff.intent == "escalate": + await manager.dispatch_escalation_notifications(handoff) + return HandoffResponse.model_validate(handoff) diff --git a/backend/app/services/handoff_manager.py b/backend/app/services/handoff_manager.py index c79461ba..fedc8a74 100644 --- a/backend/app/services/handoff_manager.py +++ b/backend/app/services/handoff_manager.py @@ -4,6 +4,7 @@ Creates handoff snapshots, AI assessments (for escalations), claim workflow, and queue queries. Dual-writes to ai_sessions.escalation_package for backward compatibility with the existing escalation queue. """ +import asyncio import logging from datetime import datetime, timezone from typing import Any @@ -12,9 +13,12 @@ from uuid import UUID from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession +from app.core.config import settings +from app.core.email import EmailService from app.models.ai_session import AISession from app.models.session_branch import SessionBranch from app.models.session_handoff import SessionHandoff +from app.models.user import User logger = logging.getLogger(__name__) @@ -87,6 +91,102 @@ class HandoffManager: await self.db.flush() return handoff + async def dispatch_escalation_notifications( + self, handoff: SessionHandoff + ) -> int: + """Email engineer-or-admin users in the account about a new escalation. + + Call this AFTER `db.commit()` has succeeded — sending email for a + rolled-back handoff is the kind of trust-erosion bug that makes pilot + customers stop trusting the tool. Returns the number of recipients + successfully emailed (best-effort, not authoritative). + + Failures are logged but never raise: the wedge demo's reliability + story is "handoff creation always succeeds; notification is best-effort," + not "handoff creation depends on the email service being up." This is + the graceful-degradation regression the eng + codex reviews both + flagged as critical. + + Per-channel delivery records (Codex correction on the dead + `notification_sent` boolean) are a v1.x story — for now the + application logs are the audit trail. + """ + if handoff.intent != "escalate": + return 0 + + try: + recipients = ( + await self.db.execute( + select(User).where( + User.account_id == handoff.account_id, + User.id != handoff.handed_off_by, + User.account_role.in_(("owner", "admin", "engineer")), + User.is_active.is_(True), + User.deleted_at.is_(None), + ) + ) + ).scalars().all() + + if not recipients: + logger.info( + "No notification recipients for handoff %s in account %s", + handoff.id, + handoff.account_id, + ) + return 0 + + # Pull session for the email subject. Fall back to a generic title + # if the session is gone (e.g. cascade delete mid-dispatch). + session_result = await self.db.execute( + select(AISession).where(AISession.id == handoff.session_id) + ) + session = session_result.scalar_one_or_none() + problem = ( + session.problem_summary if session and session.problem_summary + else "an active session" + ) + + title = f"New escalation: {problem}" + notes = (handoff.engineer_notes or "").strip() + body = ( + "A teammate has escalated a session and is asking for help.\n\n" + f"Reason: {notes if notes else 'No reason provided.'}\n" + f"Priority: {handoff.priority}" + ) + link_url = ( + f"{settings.FRONTEND_URL.rstrip('/')}/escalations" + if settings.FRONTEND_URL + else None + ) + + results = await asyncio.gather( + *[ + EmailService.send_notification_email( + to_email=r.email, + title=title, + body=body, + link_url=link_url, + ) + for r in recipients + ], + return_exceptions=True, + ) + sent = sum(1 for r in results if r is True) + logger.info( + "Escalation notifications dispatched for handoff %s: %d/%d recipients", + handoff.id, + sent, + len(recipients), + ) + return sent + + except Exception: + logger.exception( + "Escalation notification dispatch failed for handoff %s", + handoff.id, + ) + return 0 + async def _generate_snapshot(self, session: AISession) -> dict[str, Any]: """Generate a snapshot of the session state at handoff time.""" snapshot: dict[str, Any] = { diff --git a/backend/tests/test_handoff_manager.py b/backend/tests/test_handoff_manager.py index 6e1e530e..fc4644be 100644 --- a/backend/tests/test_handoff_manager.py +++ b/backend/tests/test_handoff_manager.py @@ -1,8 +1,12 @@ """Integration tests for HandoffManager service.""" +from unittest.mock import AsyncMock, patch + import pytest from httpx import AsyncClient from app.models.ai_session import AISession +from app.models.user import User +from app.services.handoff_manager import HandoffManager @pytest.mark.asyncio @@ -113,3 +117,207 @@ async def test_claim_session(client: AsyncClient, test_user, test_admin, auth_he await test_db.refresh(session) assert session.status == "active" + + +# ─── Notification dispatch ──────────────────────────────────────────────────── + + +@pytest.mark.asyncio +async def test_dispatch_emails_engineer_recipients_in_account( + client: AsyncClient, test_user, auth_headers, test_db +): + """dispatch_escalation_notifications emails every engineer/admin in the + account except the escalator.""" + # Add a second user (engineer role) in the same account. + teammate = User( + email="teammate@example.com", + password_hash="x", + name="Teammate", + role="engineer", + account_id=test_user["user_data"]["account_id"], + account_role="engineer", + ) + test_db.add(teammate) + await test_db.flush() + + # Add a viewer-role user — must NOT receive a notification. + viewer = User( + email="viewer@example.com", + password_hash="x", + name="Viewer", + role="engineer", + account_id=test_user["user_data"]["account_id"], + account_role="viewer", + ) + test_db.add(viewer) + await test_db.flush() + + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "vpn down"}, + problem_summary="VPN won't connect after Win update", + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="Stuck on auth handshake", + user_id=test_user["user_data"]["id"], + ) + await test_db.commit() + + with patch( + "app.services.handoff_manager.EmailService.send_notification_email", + new=AsyncMock(return_value=True), + ) as send: + sent = await manager.dispatch_escalation_notifications(handoff) + + assert sent == 1 # only the engineer-role teammate + recipients = {call.kwargs["to_email"] for call in send.call_args_list} + assert recipients == {"teammate@example.com"} + assert viewer.email not in recipients + assert test_user["email"] not in recipients # not self-notified + + title = send.call_args_list[0].kwargs["title"] + assert "VPN won't connect after Win update" in title + + +@pytest.mark.asyncio +async def test_dispatch_skipped_for_park_intent( + client: AsyncClient, test_user, auth_headers, test_db +): + """park-intent handoffs are private (waiting for client logs etc) — no + team-wide email.""" + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "x"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="park", + engineer_notes="waiting on customer", + user_id=test_user["user_data"]["id"], + ) + await test_db.commit() + + with patch( + "app.services.handoff_manager.EmailService.send_notification_email", + new=AsyncMock(return_value=True), + ) as send: + sent = await manager.dispatch_escalation_notifications(handoff) + + assert sent == 0 + assert send.call_count == 0 + + +@pytest.mark.asyncio +async def test_dispatch_graceful_degradation_when_email_raises( + client: AsyncClient, test_user, auth_headers, test_db +): + """If the email service raises (auth misconfig, network, etc.), dispatch + must NOT raise. Handoff creation has already committed; emailing is + best-effort. Codex-flagged regression.""" + teammate = User( + email="t@example.com", + password_hash="x", + name="T", + role="engineer", + account_id=test_user["user_data"]["account_id"], + account_role="engineer", + ) + test_db.add(teammate) + await test_db.flush() + + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "x"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="help", + user_id=test_user["user_data"]["id"], + ) + await test_db.commit() + + with patch( + "app.services.handoff_manager.EmailService.send_notification_email", + new=AsyncMock(side_effect=RuntimeError("SMTP down")), + ): + # Must not raise. + sent = await manager.dispatch_escalation_notifications(handoff) + assert sent == 0 + + +@pytest.mark.asyncio +async def test_create_handoff_endpoint_dispatches_on_escalate( + client: AsyncClient, test_user, auth_headers, test_db +): + """End-to-end: POST /handoff with intent=escalate triggers + dispatch_escalation_notifications after commit. Verifies the wiring in + the endpoint, not just the manager method.""" + teammate = User( + email="t2@example.com", + password_hash="x", + name="T2", + role="engineer", + account_id=test_user["user_data"]["account_id"], + account_role="engineer", + ) + test_db.add(teammate) + await test_db.commit() + + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "x"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + with patch( + "app.services.handoff_manager.EmailService.send_notification_email", + new=AsyncMock(return_value=True), + ) as send: + resp = await client.post( + f"/api/v1/ai-sessions/{session.id}/handoff", + headers=auth_headers, + json={"intent": "escalate", "engineer_notes": "Need help"}, + ) + assert resp.status_code == 201 + assert send.call_count == 1 + assert send.call_args.kwargs["to_email"] == "t2@example.com" -- 2.49.1 From 9f0bfd44f950232a23306e7591e49e0990352355 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 16:00:34 -0400 Subject: [PATCH 05/34] feat(escalations): mount time-to-first-action stat-card on /escalations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Surfaces the new GET /analytics/flowpilot/escalations endpoint as a card above the EscalationQueue list. Closes the loop from yesterday's metric endpoint commit — seniors and owners see the wedge stat the moment they open the queue, which is the daily-reps version of the GTM ROI story. Pieces: - EscalationMetrics TS interface mirroring the backend Pydantic model (incl. metric_definition disclaimer field) - flowpilotAnalyticsApi.getEscalationMetrics(period) client method - EscalationMetricCard component: * loading skeleton, error state, zero-data empty state * avg + median + n_with_action/n_claimed conversion rate * humanized seconds → "Ns" / "N.N min" formatting * inline disclaimer reminding callers this is in-product time-to- first-action only, NOT the savings claim — pair with manual baseline (per /codex review's two-metric correction) - Wired into EscalationQueuePage above EscalationQueue DS-aligned: card-flat, accent-dim usage held to interactive elements, text-muted-foreground for secondary copy, font-heading on the headline number, explicit transition properties (no `transition: all`). Respects prefers-reduced-motion implicitly (only animation is the loading pulse, which Tailwind's animate-pulse already gates). tsc -b clean. No new tests in this commit — component is a thin state-machine over an axios call; integration coverage comes from the existing backend tests + the e2e Playwright work in the plan. Co-Authored-By: Claude Opus 4.7 --- frontend/src/api/flowpilotAnalytics.ts | 16 ++- .../flowpilot/EscalationMetricCard.tsx | 130 ++++++++++++++++++ frontend/src/components/flowpilot/index.ts | 1 + frontend/src/pages/EscalationQueuePage.tsx | 4 +- frontend/src/types/flowpilot-analytics.ts | 13 ++ 5 files changed, 162 insertions(+), 2 deletions(-) create mode 100644 frontend/src/components/flowpilot/EscalationMetricCard.tsx diff --git a/frontend/src/api/flowpilotAnalytics.ts b/frontend/src/api/flowpilotAnalytics.ts index 0f4ccca4..27552bee 100644 --- a/frontend/src/api/flowpilotAnalytics.ts +++ b/frontend/src/api/flowpilotAnalytics.ts @@ -1,5 +1,12 @@ import apiClient from './client' -import type { FlowPilotDashboard, KnowledgeGapReport, CoverageResponse, FlowQualityResponse, EnhancedPsaMetrics } from '@/types/flowpilot-analytics' +import type { + FlowPilotDashboard, + KnowledgeGapReport, + CoverageResponse, + FlowQualityResponse, + EnhancedPsaMetrics, + EscalationMetrics, +} from '@/types/flowpilot-analytics' export const flowpilotAnalyticsApi = { async getDashboard(period: string = '30d'): Promise { @@ -36,6 +43,13 @@ export const flowpilotAnalyticsApi = { }) return response.data }, + + async getEscalationMetrics(period: string = '30d'): Promise { + const response = await apiClient.get('/analytics/flowpilot/escalations', { + params: { period }, + }) + return response.data + }, } export default flowpilotAnalyticsApi diff --git a/frontend/src/components/flowpilot/EscalationMetricCard.tsx b/frontend/src/components/flowpilot/EscalationMetricCard.tsx new file mode 100644 index 00000000..78b97d34 --- /dev/null +++ b/frontend/src/components/flowpilot/EscalationMetricCard.tsx @@ -0,0 +1,130 @@ +import { useEffect, useState } from 'react' +import { Clock, TrendingUp, AlertCircle } from 'lucide-react' +import { flowpilotAnalyticsApi } from '@/api' +import type { EscalationMetrics } from '@/types/flowpilot-analytics' + +interface EscalationMetricCardProps { + period?: string +} + +function formatSeconds(s: number | null): string { + if (s === null) return '—' + if (s < 60) return `${Math.round(s)}s` + const mins = s / 60 + if (mins < 10) return `${mins.toFixed(1)} min` + return `${Math.round(mins)} min` +} + +/** + * Shows the in-product time-to-first-action metric above the EscalationQueue. + * + * NOTE: this is the in-product metric only. The "minutes recovered" sales + * claim requires a manual baseline measurement (see The Assignment in + * docs/plans/2026-04-27-escalation-mode-wedge-design.md). Frame the number + * as "time-to-first-action with structured handoff," not "minutes saved." + */ +export function EscalationMetricCard({ period = '30d' }: EscalationMetricCardProps) { + const [metrics, setMetrics] = useState(null) + const [error, setError] = useState(null) + const [isLoading, setIsLoading] = useState(true) + + useEffect(() => { + let cancelled = false + + const load = async () => { + setIsLoading(true) + setError(null) + try { + const data = await flowpilotAnalyticsApi.getEscalationMetrics(period) + if (!cancelled) setMetrics(data) + } catch { + if (!cancelled) setError('Failed to load metric') + } finally { + if (!cancelled) setIsLoading(false) + } + } + load() + return () => { + cancelled = true + } + }, [period]) + + if (isLoading) { + return ( +
+
+
+ ) + } + + if (error) { + return ( +
+ + {error} +
+ ) + } + + if (!metrics || metrics.n_handoffs_claimed === 0) { + return ( +
+

+ Time to first action ({period}) +

+

+ No claimed escalations yet. Once your team starts using Pick Up, + we'll measure how fast they get into resolution. +

+
+ ) + } + + const avgLabel = formatSeconds(metrics.avg_seconds_to_first_action) + const medianLabel = formatSeconds(metrics.median_seconds_to_first_action) + const conversionRate = + metrics.n_handoffs_claimed > 0 + ? Math.round( + (metrics.n_handoffs_with_action / metrics.n_handoffs_claimed) * 100, + ) + : 0 + + return ( +
+
+ + Time to first action — last {period} +
+ +
+
+ + {avgLabel} + + avg +
+
+ {medianLabel} median +
+
+ + {metrics.n_handoffs_with_action} + + /{metrics.n_handoffs_claimed} claimed escalations + + ({conversionRate}% reached first action) + +
+
+ +

+ + + In-product measurement only. The savings claim requires a manual + baseline of pre-Escalation-Mode handoff time. See your team's + Assignment for the baseline number. + +

+
+ ) +} diff --git a/frontend/src/components/flowpilot/index.ts b/frontend/src/components/flowpilot/index.ts index 3fe5cc4e..0cdb9db0 100644 --- a/frontend/src/components/flowpilot/index.ts +++ b/frontend/src/components/flowpilot/index.ts @@ -9,6 +9,7 @@ export { AISessionListItem } from './AISessionListItem' export { SessionTicketCard } from './SessionTicketCard' export { EscalateModal } from './EscalateModal' export { EscalationQueue } from './EscalationQueue' +export { EscalationMetricCard } from './EscalationMetricCard' export { SessionBriefing } from './SessionBriefing' export { ProposalCard } from './ProposalCard' export { ProposalDetail } from './ProposalDetail' diff --git a/frontend/src/pages/EscalationQueuePage.tsx b/frontend/src/pages/EscalationQueuePage.tsx index cddbff18..5ae5a20e 100644 --- a/frontend/src/pages/EscalationQueuePage.tsx +++ b/frontend/src/pages/EscalationQueuePage.tsx @@ -1,6 +1,6 @@ import { useState } from 'react' import { AlertTriangle } from 'lucide-react' -import { EscalationQueue } from '@/components/flowpilot' +import { EscalationQueue, EscalationMetricCard } from '@/components/flowpilot' export default function EscalationQueuePage() { const [count, setCount] = useState(null) @@ -21,6 +21,8 @@ export default function EscalationQueuePage() {
+ + ) diff --git a/frontend/src/types/flowpilot-analytics.ts b/frontend/src/types/flowpilot-analytics.ts index f1446f43..b5767060 100644 --- a/frontend/src/types/flowpilot-analytics.ts +++ b/frontend/src/types/flowpilot-analytics.ts @@ -134,3 +134,16 @@ export interface EnhancedPsaMetrics { push_funnel: PsaFunnel daily_trend: PsaDailyTrend[] } + +// Escalation Mode wedge metric — in-product time-to-first-action. +// Pair with a manual baseline measurement for the savings claim. +// See docs/plans/2026-04-27-escalation-mode-wedge-design.md. +export interface EscalationMetrics { + period: string + n_handoffs_claimed: number + n_handoffs_with_action: number + avg_seconds_to_first_action: number | null + median_seconds_to_first_action: number | null + p95_seconds_to_first_action: number | null + metric_definition: string +} -- 2.49.1 From a283d0d3fdc45949a0d24e9348b5b3de006b43f4 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 16:38:14 -0400 Subject: [PATCH 06/34] docs(ai): refresh handoff state mid-flight on Escalation Mode build Capture the in-flight state of the Escalation Mode wedge build so the next session (or Codex resume) picks up cleanly without re-deriving context: - CURRENT_TASK now describes the wedge, what's done across the 5 commits on this branch, what remains (WebSocket push, magic-moment screen, analytics page, e2e), and the two-metric framing readers MUST internalize before quoting numbers - HANDOFF resume point is the WebSocket/SSE push (live-arrival half of the notification dual-path); includes suggested first slice + watch-outs (no user_id on ai_session_step, denormalized account_id, peer-escalation still gated to session owner) - Both files reference the design doc and the kill-switch criterion No code change. Co-Authored-By: Claude Opus 4.7 --- .ai/CURRENT_TASK.md | 44 +++++++++++++++++++++++++++--------- .ai/HANDOFF.md | 54 +++++++++++++++++++++++++++++++-------------- 2 files changed, 71 insertions(+), 27 deletions(-) diff --git a/.ai/CURRENT_TASK.md b/.ai/CURRENT_TASK.md index 6879f473..5e8d9314 100644 --- a/.ai/CURRENT_TASK.md +++ b/.ai/CURRENT_TASK.md @@ -1,20 +1,42 @@ # CURRENT_TASK.md -**Task:** No active task — pick from [`TODO.md`](TODO.md). +**Task:** Build **Escalation Mode** — the wedge for ResolutionFlow's GTM (first paying-customer push). When a junior tech escalates a FlowPilot session, the senior tech sees structured handoff context in seconds instead of running a 5-minute verbal "tell me what you tried" call. -**Status:** ready for next pickup. +**Status:** in-flight on `feat/escalation-mode` (currently `feat/escalation-metric-endpoint`). Backend metric + role gate + email notification shipped. Frontend stat-card mounted. **Next:** WebSocket/SSE push (live-arrival half of the dual-path) and the magic-moment handoff-context screen. -## Recommended next moves +**Plan:** [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md). Reviewed by `/office-hours`, `/plan-eng-review`, `/plan-design-review`, `/codex review`. Eng + Design CLEARED. Codex's two-metric correction + claim-role-gate + per-channel notification model all applied to the plan and the code. -1. **Promote `CI / e2e (pull_request)` to required on `main`.** Two consecutive PR runs (#150 and #153) have now finished green on the e2e job. That was the threshold the prior CI-recovery task set for promoting it. Branch protection update only — no code change. -2. **Pick a backlog item.** Top of `TODO.md` "Up next" is the `data-testid` e2e-stability work (PR #152 spent five one-line selector updates chasing UI churn — adding stable test IDs to a small set of high-value elements would make those tests immune to copy/route renames). The new `currentChatRef` silent-return audit added in #153's session is in Backlog and is a natural pairing with the bug fix that was just shipped. +**Test plan artifact:** [`docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md`](../docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md) — primary input for `/qa` once the build is feature-complete. + +## Done so far on `feat/escalation-metric-endpoint` + +| Commit | What it ships | +|---|---| +| `d51e95c` | Plan + test-plan artifacts checked in | +| `52f6d03` | `GET /analytics/flowpilot/escalations` — in-product time-to-first-action; account-scoped, engineer-or-admin gated; 9 tests including multi-tenant isolation | +| `7a5b853` | Role-gate POST `/handoffs/{id}/claim` to engineer-or-admin (was viewer-claimable); 2 tests | +| `07d0db9` | `HandoffManager.dispatch_escalation_notifications` — emails engineer/admin teammates on intent=escalate; graceful-degradation regression test; 4 tests | +| `9f0bfd4` | `EscalationMetricCard` mounted above the queue list; consumes the new endpoint; matches DESIGN-SYSTEM tokens | + +20 backend tests green across handoff_manager + session_handoffs_api + flowpilot_analytics_escalations. Frontend `tsc -b` clean. Nothing pushed yet. + +## Remaining work on this branch + +1. **WebSocket/SSE push** for live escalation arrival in the queue — the second half of the notification dual-path. Senior already on the queue page sees a new card slide in within ~1s of the junior hitting Escalate. ~3-4 days of work split across multiple commits (connection manager, auth-scoped fan-out, frontend EventSource handling, reconnect, slide-in animation, tab-title flash). +2. **Magic-moment handoff-context screen** — 4-section view (problem summary / what's been tried / AI assessment / Start here CTA) that loads on Pick Up before dissolving into the regular FlowPilot session view. ~1.5-2 days. +3. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. +4. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. + +## Two-metric framing — read this before quoting numbers to anyone + +The in-product endpoint measures *post-claim time-to-first-action*. The "minutes recovered" sales claim is `manual_baseline − in_product_metric`. Manual baseline comes from the founder's stopwatch on the next 5 escalations (The Assignment in the design doc). Don't roll the in-product number alone into "minutes recovered" — that's the apples-to-oranges miscount Codex caught. + +## Kill-switch + +Week 8: if 0 of 3 pilots produce a verifiable hours-saved-per-week number above 1.0, revisit the wedge. The design doc names the alternative (deterministic-ops territory) for context, but don't pivot before the data lands. ## Previous task — closed out -**Task:** Land PR #153 — fix the `AssistantChatPage` prefill `currentChatRef` bug that silently dropped AI follow-up responses in the task lane. +**Task:** Land PR #153 — fix the `AssistantChatPage` prefill `currentChatRef` bug. **Status:** complete (2026-04-26). Merged as `68fcdc6` on `main`. E2e regression test now in the suite. -**Status:** complete (2026-04-26). - -- PR #153 merged as commit `68fcdc6` on `main`. Backend, frontend, and e2e all green on the merged SHA after the env-var fix. -- E2e CI needed a stub `ANTHROPIC_API_KEY` in the workflow so the AI-gated `POST /api/v1/ai-sessions` endpoint stops returning 503; the Playwright `page.route` stub still intercepts the actual `/chat` call in the browser, so no real Anthropic traffic occurs. -- Regression test `frontend/e2e/assistant-chat-prefill.spec.ts` is part of the e2e suite going forward. +**Background CI item, not blocking:** promoting `CI / e2e (pull_request)` to required on `main`. Two consecutive green PR runs (#150 and #153) cleared the threshold. Ops-only. diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index 54190f44..4c96a7ab 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -2,27 +2,49 @@ # HANDOFF.md -**Last updated:** 2026-04-26 04:55 EDT +**Last updated:** 2026-04-27 EDT -**Active task:** None — pick from [`TODO.md`](TODO.md). See [`CURRENT_TASK.md`](CURRENT_TASK.md) for recommended next moves. +**Active task:** **Escalation Mode** wedge build. See [`CURRENT_TASK.md`](CURRENT_TASK.md) for the full status; this file holds the resume point only. -**Branch:** `main` is the home position. Recent merges: PR #150 (CI recovery, `87bb20b`), PR #153 (prefill `currentChatRef` fix, `68fcdc6`). +**Branch:** `feat/escalation-metric-endpoint` — five commits stacked on top of `main` (`c0ed6d9`). Nothing pushed yet. + +``` +9f0bfd4 feat(escalations): mount time-to-first-action stat-card on /escalations +07d0db9 feat(handoff): email engineer-or-admin teammates on escalation +7a5b853 feat(api): role-gate handoff claim to engineer-or-admin +52f6d03 feat(analytics): add escalation time-to-first-action metric endpoint +d51e95c docs(plans): add escalation-mode wedge design + test plan +``` + +## Resume point + +Pick up the **WebSocket/SSE push** — the live-arrival half of the notification dual-path. Email is already wired (commit `07d0db9`); push is the second channel that makes the demo's "30-second magic moment" undeniable when the receiving senior is online and on the queue page. + +Suggested first slice: a thin server-side SSE endpoint scoped to `current_user.account_id`, fan out from `HandoffManager.dispatch_escalation_notifications` (alongside email), and hook the frontend `EscalationQueue` to subscribe and prepend new cards with the locked 200ms slide-in. Reconnect logic, tab-title flash, and `prefers-reduced-motion` respect are part of this slice per the locked UI spec in the design doc. + +After the dual-path is feature-complete, the **magic-moment handoff-context screen** is next (4 sections, dissolves into the FlowPilot session view on first action). ## Where things stand -- CI is healthy on `main`: backend, frontend, and e2e all green on the latest commits. -- Branch protection on `main`: PR-only merges, force-push blocked, **`CI / frontend (pull_request)` required**, **`CI / backend (pull_request)` required**, `CI / e2e (pull_request)` not yet required. -- Two consecutive PR runs (#150, #153) finished green on e2e. The "promote e2e to required" gate from the prior task is now satisfiable. -- Backend AI-gated endpoints (`POST /ai-sessions`, `/chat`, `/respond`, etc.) call `_require_ai_enabled()` and return 503 if no provider key is set. The e2e CI job now sets a stub `ANTHROPIC_API_KEY` so any future test that exercises those flows can rely on it; tests should still stub the actual AI calls in the browser via `page.route` so no real Anthropic traffic occurs. - -## Immediate next steps - -1. (Optional, ops-only) Promote `CI / e2e (pull_request)` to required on `main` in Gitea branch protection. -2. Pick the next backlog item from `TODO.md`. Top of "Up next" is the `data-testid` e2e-stability audit; the new `currentChatRef` silent-return audit (added to backlog in this session) is a natural pairing with the bug fix that just shipped. +- CI on `main` still healthy. Branch protection: `CI / frontend (pull_request)` required, `CI / backend (pull_request)` required, `CI / e2e (pull_request)` not yet required (ops-only follow-up — two consecutive green runs cleared the threshold). +- 20 backend tests green on this branch (handoff_manager, session_handoffs_api, flowpilot_analytics_escalations). Frontend `tsc -b` clean. Branch has not been pushed; no CI runs yet. +- The plan doc at [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md) is the source of truth for every UI / metric / scope decision. The embedded **GSTACK REVIEW REPORT** at the bottom shows Eng + Design CLEARED and Codex INFO with the disposition of all 12 of its findings. ## Useful breadcrumbs -- The fix that just landed: [`frontend/src/pages/AssistantChatPage.tsx`](../frontend/src/pages/AssistantChatPage.tsx) — `currentChatRef.current = session.session_id` after `setActiveChatId` in the dashboard prefill effect. -- Regression test: [`frontend/e2e/assistant-chat-prefill.spec.ts`](../frontend/e2e/assistant-chat-prefill.spec.ts). -- E2e env convention: [`.gitea/workflows/ci.yml`](../.gitea/workflows/ci.yml) — `ANTHROPIC_API_KEY` is stubbed in the e2e job env. Tests that exercise AI-gated endpoints should stub the actual AI calls in the browser, not rely on a real key. -- Silent-return follow-up entry: [`.ai/TODO.md`](TODO.md), Backlog section. +- New endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics` at the bottom of the file. +- Notification dispatch: [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`. Wired in [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) **after** `db.commit()` so a rolled-back handoff never emails. +- Frontend stat-card: [`frontend/src/components/flowpilot/EscalationMetricCard.tsx`](../frontend/src/components/flowpilot/EscalationMetricCard.tsx). Renders `n_with_action / n_claimed`, avg + median, and the metric_definition disclaimer. +- Two-metric framing — required reading before quoting any number to a pilot. The in-product endpoint measures *post-claim time-to-first-action*; the savings claim is `manual_baseline − in_product`. Manual baseline comes from the founder's stopwatch on the next 5 escalations (The Assignment in the design doc). +- The `notification_sent` boolean is intentionally NOT being written. Per Codex's correction it should be replaced by per-channel delivery records; v1.x story. For now, application logs are the audit trail. +- Two TODOs added during this session: peer-tech escalation (deferred to v2) and the (already moved-in-scope) claim role gate. See [`TODO.md`](TODO.md). + +## Watch-outs + +- `ai_session_step` has NO `user_id` column — the metric query keys "first action by senior" off `session_id + created_at > claimed_at`, which is fine because session activity post-claim IS the senior's activity (the session is reactivated under `escalated_to_id`). If a future change adds `user_id` to `ai_session_step`, the metric query can become more precise. +- `account_id` is denormalized on `ai_session_step` (Phase 4 RLS pattern). The metric query and any new SSE subscription scoping must use it directly, not join through `ai_sessions`. +- POST `/handoff` still requires the session owner to be the escalator (`AISession.user_id == current_user.id`). Peer-tech escalation is captured as a v2 TODO. Don't widen this without a UX decision. + +## Kill-switch (week 8) + +If 0 of 3 pilots produce a verifiable hours-saved-per-week number above 1.0, revisit the wedge. The design doc names the alternative direction (deterministic-ops territory) but data lands first. -- 2.49.1 From 87bd0b7c569691d7fa09b0fb3c0ea6d144fc13b8 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 19:29:07 -0400 Subject: [PATCH 07/34] WIP: SSE pub/sub for live escalation arrivals (paused for Codex review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First half of the WebSocket/SSE push slice. Paused mid-flight to hand the branch to Codex for outside-voice review before stacking more commits on top. See .ai/HANDOFF.md for the full pause context + what to look at. What's here: - backend/app/core/escalation_bus.py — module-level singleton in-memory pub/sub keyed by account_id. asyncio.Queue per subscriber with 64-event maxsize and drop-on-full semantics. Designed to be swappable for Redis pub/sub when Railway scales past single-replica. - backend/app/api/endpoints/session_handoffs.py — GET /api/v1/ai-sessions/escalations/stream SSE endpoint. Auth via require_engineer_or_admin. 25s heartbeat. Account-scoped subscribe bound to current_user.account_id. - backend/app/services/handoff_manager.py — dispatch_escalation_notifications now publishes a `handoff_created` event to the bus BEFORE the email fan-out, in a try/except so a bus failure can't block email delivery. - backend/tests/test_escalation_bus.py — 7 unit tests, all green standalone (0.14s). Cross-tenant isolation, drop-on-full, no-subscribers. - backend/tests/test_handoff_manager.py — +1 dispatcher integration test (publishes to bus, payload shape). - backend/tests/test_session_handoffs_api.py — +2 endpoint tests (viewer blocked, ready event handshake). [gstack-context] Decisions: - SSE over WebSocket (one-way, browser EventSource semantics, fewer moving parts behind Railway proxy) - In-memory bus over Redis for v1 pilot (3 MSPs, single replica) - Drop-on-full subscriber queue rather than back-pressure publishers - Bus publish ahead of email send, both wrapped in try/except so neither can break handoff creation - Frontend will be a fetch-based ReadableStream reader matching the existing streamDocumentation pattern, not native EventSource (custom-header auth) Remaining (post-Codex): - Frontend SSE subscription in EscalationQueue.tsx (slide-in, reconnect, tab-title flash, prefers-reduced-motion) - Magic-moment handoff-context screen - Re-run the full backend test suite to verify the SSE + dispatcher integration tests (bus units already green standalone) Tried: - Running the full test suite repeatedly without xdist; the per-test DROP SCHEMA + recreate fixture made wall-clock prohibitive when multiple stale runs collided on the same Postgres test schema. Resolution: -n auto next time. [/gstack-context] Co-Authored-By: Claude Opus 4.7 --- backend/app/api/endpoints/session_handoffs.py | 90 ++++++++++++++- backend/app/core/escalation_bus.py | 97 ++++++++++++++++ backend/app/services/handoff_manager.py | 24 ++++ backend/tests/test_escalation_bus.py | 106 ++++++++++++++++++ backend/tests/test_handoff_manager.py | 52 +++++++++ backend/tests/test_session_handoffs_api.py | 43 +++++++ 6 files changed, 408 insertions(+), 4 deletions(-) create mode 100644 backend/app/core/escalation_bus.py create mode 100644 backend/tests/test_escalation_bus.py diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 5e444bd2..5b62a3c5 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -1,19 +1,24 @@ """Handoff endpoints — unified park/escalate. - POST /ai-sessions/{id}/handoff — Create handoff + POST /ai-sessions/{id}/handoff — Create handoff GET /ai-sessions/{id}/handoffs — Handoff history POST /ai-sessions/{id}/handoffs/{hid}/claim — Claim session - GET /ai-sessions/queue — Team queue + GET /ai-sessions/queue — Team queue + GET /ai-sessions/escalations/stream — SSE: live escalation arrivals """ +import asyncio +import json import logging -from typing import Annotated +from typing import Annotated, AsyncGenerator from uuid import UUID -from fastapi import APIRouter, Depends, HTTPException, status +from fastapi import APIRouter, Depends, HTTPException, Request, status +from fastapi.responses import StreamingResponse from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from app.api.deps import get_current_active_user, get_db, require_engineer_or_admin +from app.core.escalation_bus import bus as escalation_bus from app.models.user import User from app.models.ai_session import AISession from app.models.session_handoff import SessionHandoff @@ -127,3 +132,80 @@ async def get_queue( team_id=current_user.team_id, account_id=current_user.account_id, ) + + +# ─── Live escalation arrivals (SSE) ────────────────────────────────────────── +# +# Streams `handoff_created` events to subscribers in the same account_id as +# the new handoff. Connected EscalationQueue instances prepend the new card +# with the locked 200ms slide-in. Account-scoped: cross-tenant leakage is +# prevented at the bus.publish boundary (only handoff.account_id subscribers +# are notified) and re-enforced here by binding the subscription to +# current_user.account_id. +# +# Heartbeat: a `: keepalive\n\n` SSE comment every 25s keeps the connection +# alive through Railway / nginx default 60s idle timeouts. Reconnect policy +# is on the client (browser EventSource auto-reconnects; our fetch-based +# reader retries with backoff). + + +_HEARTBEAT_INTERVAL_S = 25 +_QUEUE_GET_TIMEOUT_S = 25 # < heartbeat so heartbeat fires reliably + + +@queue_router.get("/escalations/stream") +async def stream_escalations( + request: Request, + current_user: Annotated[User, Depends(require_engineer_or_admin)], +): + """SSE stream of new escalation arrivals for the current user's account. + + Role-gated to engineer/admin/owner so viewers can't subscribe (matches + the queue + claim role surface). One open connection per browser tab is + expected; the bus handles fan-out. + """ + if not current_user.account_id: + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, detail="No account" + ) + + account_id = current_user.account_id + + async def event_generator() -> AsyncGenerator[str, None]: + queue = await escalation_bus.subscribe(account_id) + try: + # Initial hello so the client knows the stream is live. + yield ( + "event: ready\n" + f"data: {json.dumps({'account_id': str(account_id)})}\n\n" + ) + + while True: + if await request.is_disconnected(): + break + try: + event = await asyncio.wait_for( + queue.get(), timeout=_QUEUE_GET_TIMEOUT_S + ) + except asyncio.TimeoutError: + # Heartbeat keeps the connection alive through proxies. + yield ": keepalive\n\n" + continue + + event_type = event.get("type", "message") + yield ( + f"event: {event_type}\n" + f"data: {json.dumps(event)}\n\n" + ) + finally: + await escalation_bus.unsubscribe(account_id, queue) + + return StreamingResponse( + event_generator(), + media_type="text/event-stream", + headers={ + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "X-Accel-Buffering": "no", + }, + ) diff --git a/backend/app/core/escalation_bus.py b/backend/app/core/escalation_bus.py new file mode 100644 index 00000000..bf623950 --- /dev/null +++ b/backend/app/core/escalation_bus.py @@ -0,0 +1,97 @@ +"""In-memory pub/sub bus for live escalation events. + +Single-process, non-durable. When a handoff fires, every connected SSE +subscriber for the same `account_id` receives the event. Subscribers come +and go as senior techs open and close the EscalationQueue page. + +Pre-PMF scale (3 pilots × 5-20 techs/MSP = ~15-60 concurrent subscribers +total, single Railway replica) makes in-memory the right call. When the +deployment scales horizontally, swap this for Redis pub/sub or similar — +the public surface (`publish` / `subscribe`) is intentionally narrow so +the swap is local. + +Events are JSON-serializable dicts. `publish()` is non-blocking (drops the +event if a subscriber's queue is full rather than back-pressuring the +caller). `subscribe()` MUST be paired with `unsubscribe()` in a finally +block, or you leak queues. +""" +from __future__ import annotations + +import asyncio +import logging +from typing import Any +from uuid import UUID + +logger = logging.getLogger(__name__) + + +# Bound how many unconsumed events can sit in a subscriber's queue before +# we start dropping. 64 is generous for the queue-page use case; if a +# subscriber is that far behind, they're probably gone or stuck. +_QUEUE_MAXSIZE = 64 + + +class EscalationBus: + """Account-scoped pub/sub for escalation arrival events.""" + + def __init__(self) -> None: + self._subscribers: dict[UUID, set[asyncio.Queue[dict[str, Any]]]] = {} + self._lock = asyncio.Lock() + + async def subscribe(self, account_id: UUID) -> asyncio.Queue[dict[str, Any]]: + """Register a new subscriber queue for an account. + + Caller must invoke `unsubscribe(account_id, queue)` when the + consumer disconnects. + """ + queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue( + maxsize=_QUEUE_MAXSIZE + ) + async with self._lock: + self._subscribers.setdefault(account_id, set()).add(queue) + return queue + + async def unsubscribe( + self, account_id: UUID, queue: asyncio.Queue[dict[str, Any]] + ) -> None: + async with self._lock: + subs = self._subscribers.get(account_id) + if subs is None: + return + subs.discard(queue) + if not subs: + self._subscribers.pop(account_id, None) + + async def publish(self, account_id: UUID, event: dict[str, Any]) -> int: + """Fan event out to every subscriber for `account_id`. + + Returns the number of subscribers that successfully received the + event. Drops the event for any subscriber whose queue is full + (logs at warning level). + """ + async with self._lock: + subs = list(self._subscribers.get(account_id, ())) + if not subs: + return 0 + delivered = 0 + for queue in subs: + try: + queue.put_nowait(event) + delivered += 1 + except asyncio.QueueFull: + logger.warning( + "EscalationBus: dropped event for full subscriber queue " + "(account_id=%s, event=%s)", + account_id, + event.get("type", "?"), + ) + return delivered + + def subscriber_count(self, account_id: UUID) -> int: + """Diagnostic — number of active subscribers for an account.""" + return len(self._subscribers.get(account_id, ())) + + +# Module-level singleton. FastAPI imports this; `subscribe()` and `publish()` +# are coroutine-safe via the internal Lock. +bus = EscalationBus() diff --git a/backend/app/services/handoff_manager.py b/backend/app/services/handoff_manager.py index fedc8a74..bc3717f9 100644 --- a/backend/app/services/handoff_manager.py +++ b/backend/app/services/handoff_manager.py @@ -15,6 +15,7 @@ from sqlalchemy.ext.asyncio import AsyncSession from app.core.config import settings from app.core.email import EmailService +from app.core.escalation_bus import bus as escalation_bus from app.models.ai_session import AISession from app.models.session_branch import SessionBranch from app.models.session_handoff import SessionHandoff @@ -114,6 +115,29 @@ class HandoffManager: if handoff.intent != "escalate": return 0 + # Publish to the in-memory bus first so connected senior-tech inboxes + # see the new card slide in within ~1s of escalate. This path is + # fire-and-forget (no IO, just memory) so it can sit ahead of the + # email fan-out. + try: + await escalation_bus.publish( + handoff.account_id, + { + "type": "handoff_created", + "handoff_id": str(handoff.id), + "session_id": str(handoff.session_id), + "priority": handoff.priority, + "engineer_notes": handoff.engineer_notes or "", + "created_at": handoff.created_at.isoformat() + if handoff.created_at + else None, + }, + ) + except Exception: + logger.exception( + "EscalationBus publish failed for handoff %s", handoff.id + ) + try: recipients = ( await self.db.execute( diff --git a/backend/tests/test_escalation_bus.py b/backend/tests/test_escalation_bus.py new file mode 100644 index 00000000..50d10f3c --- /dev/null +++ b/backend/tests/test_escalation_bus.py @@ -0,0 +1,106 @@ +"""Unit tests for the in-memory escalation pub/sub bus.""" +import asyncio +from uuid import uuid4 + +import pytest + +from app.core.escalation_bus import EscalationBus + + +@pytest.mark.asyncio +async def test_publish_with_no_subscribers_returns_zero(): + bus = EscalationBus() + delivered = await bus.publish(uuid4(), {"type": "handoff_created"}) + assert delivered == 0 + + +@pytest.mark.asyncio +async def test_subscribe_then_publish_delivers_event(): + bus = EscalationBus() + account = uuid4() + queue = await bus.subscribe(account) + try: + delivered = await bus.publish(account, {"type": "handoff_created", "id": "x"}) + assert delivered == 1 + event = await asyncio.wait_for(queue.get(), timeout=1.0) + assert event == {"type": "handoff_created", "id": "x"} + finally: + await bus.unsubscribe(account, queue) + + +@pytest.mark.asyncio +async def test_two_subscribers_same_account_both_receive(): + bus = EscalationBus() + account = uuid4() + q1 = await bus.subscribe(account) + q2 = await bus.subscribe(account) + try: + delivered = await bus.publish(account, {"type": "x"}) + assert delivered == 2 + e1 = await asyncio.wait_for(q1.get(), timeout=1.0) + e2 = await asyncio.wait_for(q2.get(), timeout=1.0) + assert e1 == e2 == {"type": "x"} + finally: + await bus.unsubscribe(account, q1) + await bus.unsubscribe(account, q2) + + +@pytest.mark.asyncio +async def test_subscriber_in_other_account_does_not_receive(): + """Cross-tenant isolation is the whole point — sanity check it directly.""" + bus = EscalationBus() + account_a = uuid4() + account_b = uuid4() + q_a = await bus.subscribe(account_a) + q_b = await bus.subscribe(account_b) + try: + delivered = await bus.publish(account_a, {"type": "x"}) + assert delivered == 1 + + e_a = await asyncio.wait_for(q_a.get(), timeout=1.0) + assert e_a == {"type": "x"} + + # B's queue must remain empty. + with pytest.raises(asyncio.TimeoutError): + await asyncio.wait_for(q_b.get(), timeout=0.1) + finally: + await bus.unsubscribe(account_a, q_a) + await bus.unsubscribe(account_b, q_b) + + +@pytest.mark.asyncio +async def test_unsubscribe_drops_subscriber_count_to_zero(): + bus = EscalationBus() + account = uuid4() + q = await bus.subscribe(account) + assert bus.subscriber_count(account) == 1 + await bus.unsubscribe(account, q) + assert bus.subscriber_count(account) == 0 + + +@pytest.mark.asyncio +async def test_publish_drops_events_when_subscriber_queue_is_full(): + """A stuck subscriber must not back-pressure publishers.""" + bus = EscalationBus() + account = uuid4() + queue = await bus.subscribe(account) + try: + # Stuff the queue past capacity (maxsize is 64) without consuming. + for _ in range(65): + await bus.publish(account, {"type": "x"}) + # Sanity: queue holds at most maxsize. + assert queue.qsize() <= 64 + # Publishes after capacity didn't raise — they were dropped silently. + finally: + await bus.unsubscribe(account, queue) + + +@pytest.mark.asyncio +async def test_unsubscribe_unknown_queue_is_noop(): + """Defensive: unsubscribe on an account/queue that isn't registered + should not raise — finally blocks rely on this.""" + bus = EscalationBus() + account = uuid4() + fake_queue: asyncio.Queue = asyncio.Queue() + # Should not raise. + await bus.unsubscribe(account, fake_queue) diff --git a/backend/tests/test_handoff_manager.py b/backend/tests/test_handoff_manager.py index fc4644be..3a2836a5 100644 --- a/backend/tests/test_handoff_manager.py +++ b/backend/tests/test_handoff_manager.py @@ -278,6 +278,58 @@ async def test_dispatch_graceful_degradation_when_email_raises( assert sent == 0 +@pytest.mark.asyncio +async def test_dispatch_publishes_to_escalation_bus( + client: AsyncClient, test_user, auth_headers, test_db +): + """dispatch_escalation_notifications puts an event on the in-memory bus + so connected SSE subscribers see live arrivals.""" + from app.core.escalation_bus import bus as escalation_bus + + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "x"}, + problem_summary="VPN down", + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.commit() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="please help", + user_id=test_user["user_data"]["id"], + ) + await test_db.commit() + + from uuid import UUID as PyUUID + account_id = PyUUID(test_user["user_data"]["account_id"]) + + queue = await escalation_bus.subscribe(account_id) + try: + with patch( + "app.services.handoff_manager.EmailService.send_notification_email", + new=AsyncMock(return_value=True), + ): + await manager.dispatch_escalation_notifications(handoff) + + import asyncio + event = await asyncio.wait_for(queue.get(), timeout=1.0) + assert event["type"] == "handoff_created" + assert event["handoff_id"] == str(handoff.id) + assert event["session_id"] == str(session.id) + assert event["priority"] == "normal" + finally: + await escalation_bus.unsubscribe(account_id, queue) + + @pytest.mark.asyncio async def test_create_handoff_endpoint_dispatches_on_escalate( client: AsyncClient, test_user, auth_headers, test_db diff --git a/backend/tests/test_session_handoffs_api.py b/backend/tests/test_session_handoffs_api.py index 6edaac1e..6ddc307c 100644 --- a/backend/tests/test_session_handoffs_api.py +++ b/backend/tests/test_session_handoffs_api.py @@ -113,6 +113,49 @@ async def test_claim_blocked_for_viewer_role( assert "engineer" in claim_resp.json()["detail"].lower() +@pytest.mark.asyncio +async def test_escalations_stream_blocked_for_viewer( + client: AsyncClient, test_user, auth_headers, test_db +): + """SSE stream is role-gated to engineer-or-admin (matches queue/claim).""" + user_id = PyUUID(test_user["user_data"]["id"]) + user = ( + await test_db.execute(select(User).where(User.id == user_id)) + ).scalar_one() + user.account_role = "viewer" + await test_db.commit() + + resp = await client.get( + "/api/v1/ai-sessions/escalations/stream", headers=auth_headers + ) + assert resp.status_code == 403 + + +@pytest.mark.asyncio +async def test_escalations_stream_returns_sse_content_type( + client: AsyncClient, test_user, auth_headers, test_db +): + """Engineer/owner can open the SSE stream and gets text/event-stream + plus an initial `ready` event. Read just enough bytes to confirm the + handshake — the full pub/sub flow is covered by the bus + dispatcher + tests separately.""" + async with client.stream( + "GET", + "/api/v1/ai-sessions/escalations/stream", + headers=auth_headers, + ) as resp: + assert resp.status_code == 200 + assert resp.headers["content-type"].startswith("text/event-stream") + # First chunk must contain the ready event. + first = b"" + async for chunk in resp.aiter_bytes(): + first += chunk + if b"event: ready" in first and b"\n\n" in first: + break + assert b"event: ready" in first + assert b'"account_id"' in first + + @pytest.mark.asyncio async def test_claim_allowed_for_engineer_role( client: AsyncClient, test_user, auth_headers, test_db -- 2.49.1 From ba46fc5644c53189229203be20d522ea1b6eb909 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 19:29:16 -0400 Subject: [PATCH 08/34] docs(ai): pause Escalation Mode build mid-SSE for Codex review Update HANDOFF to reflect: - Build paused after the WIP SSE commit (87bd0b7) - What Codex should look at on the SSE bus + endpoint + dispatch wiring - Resume point post-review: re-run tests with -n auto, then frontend SSE subscription, then magic-moment screen - Test-suite watch-out: per-test DROP SCHEMA fixture means concurrent pytest runs on the same DB collide; always one-suite-at-a-time or -n auto with conftest's per-worker DB isolation No code change. Co-Authored-By: Claude Opus 4.7 --- .ai/HANDOFF.md | 57 +++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index 4c96a7ab..fbf41e8a 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -2,13 +2,34 @@ # HANDOFF.md -**Last updated:** 2026-04-27 EDT +**Last updated:** 2026-04-27 EDT (paused mid-build for Codex review) **Active task:** **Escalation Mode** wedge build. See [`CURRENT_TASK.md`](CURRENT_TASK.md) for the full status; this file holds the resume point only. -**Branch:** `feat/escalation-metric-endpoint` — five commits stacked on top of `main` (`c0ed6d9`). Nothing pushed yet. +**Branch:** `feat/escalation-metric-endpoint` — six commits stacked on `main` (`c0ed6d9`). Working tree has UNCOMMITTED WIP for the SSE push. + +## Status — paused for Codex review + +Build is paused mid-flight on the SSE push. Hand the branch (and the WIP) to Codex for an outside-voice pass before stacking more commits, fixing tests, or pushing. Reasons: local backend test loop got tangled (multiple stale pytest processes contended on the same Postgres test schema; the suite design rebuilds the schema per test which doesn't tolerate concurrent runs well), and the SSE work is the kind of cross-layer surface a second pair of eyes is most valuable on. + +What Codex should look at: +1. The new SSE endpoint at [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations` — and the in-memory pub/sub bus at [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). +2. Whether the bus's single-process / non-durable design is acceptable for the v1 pilot (Railway single-replica) and what the swap-to-Redis story should look like. +3. The dispatch wiring in [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications` now publishes to the bus before the email fan-out. Race / ordering / failure-mode review. +4. Auth on the SSE stream — same `require_engineer_or_admin` dep as `/queue` and `/claim`. Browsers can't send custom headers via the native `EventSource` API; the planned frontend uses a fetch-based `ReadableStream` reader (matching the existing `streamDocumentation` pattern in [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts)). Verify that's the right call vs. a query-token scheme. +5. Whether the bus's "drop-on-full-queue" semantic is acceptable, given a stuck subscriber would silently miss live-arrival cards (they'd still see them on next page load via REST `/queue`). + +## Resume point (after Codex review) + +1. **Get the test suite back to green.** Stale pytest zombies in the container were cleared (PIDs 1790034, 1844996, 1883167, 1916565, 1935830, 2009437, 2009449 — all dead, parent uvicorn-reload didn't reap them; PID slots remain but no live processes). Re-run with `pytest -n auto` to keep wall-clock manageable. Files: `tests/test_escalation_bus.py` (7 tests), the 4 new dispatch + SSE tests in `tests/test_handoff_manager.py` and `tests/test_session_handoffs_api.py`. +2. **Frontend SSE subscription** in `EscalationQueue.tsx` — fetch-based reader, prepend new cards with the locked 200ms slide-in, reconnect with backoff, tab-title flash when backgrounded, respect `prefers-reduced-motion`. Then ship the magic-moment handoff-context screen (4 sections, dissolves into FlowPilot session view). +3. Push the branch + open a draft PR. + +## Stack ``` +WIP (uncommitted): SSE bus + endpoint + dispatcher publish + 7 bus tests + 1 dispatcher test + 2 SSE endpoint tests +a283d0d docs(ai): refresh handoff state mid-flight on Escalation Mode build 9f0bfd4 feat(escalations): mount time-to-first-action stat-card on /escalations 07d0db9 feat(handoff): email engineer-or-admin teammates on escalation 7a5b853 feat(api): role-gate handoff claim to engineer-or-admin @@ -16,34 +37,28 @@ d51e95c docs(plans): add escalation-mode wedge design + test plan ``` -## Resume point - -Pick up the **WebSocket/SSE push** — the live-arrival half of the notification dual-path. Email is already wired (commit `07d0db9`); push is the second channel that makes the demo's "30-second magic moment" undeniable when the receiving senior is online and on the queue page. - -Suggested first slice: a thin server-side SSE endpoint scoped to `current_user.account_id`, fan out from `HandoffManager.dispatch_escalation_notifications` (alongside email), and hook the frontend `EscalationQueue` to subscribe and prepend new cards with the locked 200ms slide-in. Reconnect logic, tab-title flash, and `prefers-reduced-motion` respect are part of this slice per the locked UI spec in the design doc. - -After the dual-path is feature-complete, the **magic-moment handoff-context screen** is next (4 sections, dissolves into the FlowPilot session view on first action). - ## Where things stand -- CI on `main` still healthy. Branch protection: `CI / frontend (pull_request)` required, `CI / backend (pull_request)` required, `CI / e2e (pull_request)` not yet required (ops-only follow-up — two consecutive green runs cleared the threshold). -- 20 backend tests green on this branch (handoff_manager, session_handoffs_api, flowpilot_analytics_escalations). Frontend `tsc -b` clean. Branch has not been pushed; no CI runs yet. -- The plan doc at [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md) is the source of truth for every UI / metric / scope decision. The embedded **GSTACK REVIEW REPORT** at the bottom shows Eng + Design CLEARED and Codex INFO with the disposition of all 12 of its findings. +- CI on `main` still healthy. Branch protection: `CI / frontend (pull_request)` required, `CI / backend (pull_request)` required, `CI / e2e (pull_request)` not yet required. +- The 20 tests passing as of `9f0bfd4` are still passing (last green run logged before the SSE work). The newly added SSE tests (7 bus + 1 dispatcher integration + 2 endpoint) HAVE NOT been verified end-to-end this session — they ran clean on the bus suite alone (7/7 in 0.14s) but the DB-backed integration tests were aborted before completing. +- The plan doc at [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md) is the source of truth for every UI / metric / scope decision. The embedded **GSTACK REVIEW REPORT** at the bottom shows Eng + Design CLEARED and Codex INFO from the design-stage pass. ## Useful breadcrumbs -- New endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics` at the bottom of the file. -- Notification dispatch: [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`. Wired in [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) **after** `db.commit()` so a rolled-back handoff never emails. +- Metric endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics` at the bottom. +- Notification dispatch (email + bus publish): [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`. Wired in [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) **after** `db.commit()` so a rolled-back handoff never emails or fans out. +- SSE endpoint (WIP): [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations`. Heartbeat every 25s, account-scoped subscribe, role-gated to engineer-or-admin. +- Pub/sub bus (WIP): [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). Module-level singleton, in-memory, `asyncio.Queue` per subscriber with 64-event maxsize and drop-on-full semantics. - Frontend stat-card: [`frontend/src/components/flowpilot/EscalationMetricCard.tsx`](../frontend/src/components/flowpilot/EscalationMetricCard.tsx). Renders `n_with_action / n_claimed`, avg + median, and the metric_definition disclaimer. -- Two-metric framing — required reading before quoting any number to a pilot. The in-product endpoint measures *post-claim time-to-first-action*; the savings claim is `manual_baseline − in_product`. Manual baseline comes from the founder's stopwatch on the next 5 escalations (The Assignment in the design doc). -- The `notification_sent` boolean is intentionally NOT being written. Per Codex's correction it should be replaced by per-channel delivery records; v1.x story. For now, application logs are the audit trail. -- Two TODOs added during this session: peer-tech escalation (deferred to v2) and the (already moved-in-scope) claim role gate. See [`TODO.md`](TODO.md). +- Two-metric framing — required reading before quoting any number to a pilot. In-product endpoint measures *post-claim time-to-first-action*; the savings claim is `manual_baseline − in_product`. Manual baseline comes from the founder's stopwatch on the next 5 escalations (The Assignment in the design doc). +- The `notification_sent` boolean is intentionally NOT being written. Per Codex's design-stage correction it should be replaced by per-channel delivery records; v1.x story. For now application logs are the audit trail. ## Watch-outs -- `ai_session_step` has NO `user_id` column — the metric query keys "first action by senior" off `session_id + created_at > claimed_at`, which is fine because session activity post-claim IS the senior's activity (the session is reactivated under `escalated_to_id`). If a future change adds `user_id` to `ai_session_step`, the metric query can become more precise. -- `account_id` is denormalized on `ai_session_step` (Phase 4 RLS pattern). The metric query and any new SSE subscription scoping must use it directly, not join through `ai_sessions`. -- POST `/handoff` still requires the session owner to be the escalator (`AISession.user_id == current_user.id`). Peer-tech escalation is captured as a v2 TODO. Don't widen this without a UX decision. +- `ai_session_step` has NO `user_id` column — the metric query keys "first action by senior" off `session_id + created_at > claimed_at`. Fine for v1 because session activity post-claim IS the senior's activity (session reactivates under `escalated_to_id`). +- `account_id` is denormalized on `ai_session_step` (Phase 4 RLS pattern). Use it directly; don't join through `ai_sessions`. +- POST `/handoff` still requires the session owner to be the escalator (`AISession.user_id == current_user.id`). Peer-tech escalation is a v2 TODO. +- The test suite uses `DROP SCHEMA public CASCADE` + `CREATE SCHEMA public` per test (see [`backend/tests/conftest.py:144`](../backend/tests/conftest.py#L144)). Concurrent pytest runs against the same test DB collide. Always run one suite at a time, or via `-n auto` xdist with the per-worker-DB isolation already in conftest. ## Kill-switch (week 8) -- 2.49.1 From bc15952857d99d7a03f0d24ced579cf2f91b592c Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 19:47:43 -0400 Subject: [PATCH 09/34] fix(tests): stabilize escalation SSE backend tests Co-Authored-By: Codex --- .ai/HANDOFF.md | 76 ++++++++----------- .ai/SESSION_LOG.md | 12 +++ backend/app/api/endpoints/session_handoffs.py | 5 +- backend/app/core/escalation_bus.py | 28 ++++--- backend/tests/test_escalation_bus.py | 15 ++++ backend/tests/test_handoff_manager.py | 20 +++++ backend/tests/test_session_handoffs_api.py | 70 ++++++++++++----- 7 files changed, 153 insertions(+), 73 deletions(-) diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index fbf41e8a..8ed216bf 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -2,64 +2,50 @@ # HANDOFF.md -**Last updated:** 2026-04-27 EDT (paused mid-build for Codex review) +**Last updated:** 2026-04-27 EDT **Active task:** **Escalation Mode** wedge build. See [`CURRENT_TASK.md`](CURRENT_TASK.md) for the full status; this file holds the resume point only. -**Branch:** `feat/escalation-metric-endpoint` — six commits stacked on `main` (`c0ed6d9`). Working tree has UNCOMMITTED WIP for the SSE push. +**Branch:** `feat/escalation-metric-endpoint` — SSE backend WIP is now test-stabilized locally. Working tree should be clean after the handoff commit. -## Status — paused for Codex review +## Status -Build is paused mid-flight on the SSE push. Hand the branch (and the WIP) to Codex for an outside-voice pass before stacking more commits, fixing tests, or pushing. Reasons: local backend test loop got tangled (multiple stale pytest processes contended on the same Postgres test schema; the suite design rebuilds the schema per test which doesn't tolerate concurrent runs well), and the SSE work is the kind of cross-layer surface a second pair of eyes is most valuable on. +Previous session diagnosed the slow-test issue and fixed the backend test loop. -What Codex should look at: -1. The new SSE endpoint at [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations` — and the in-memory pub/sub bus at [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). -2. Whether the bus's single-process / non-durable design is acceptable for the v1 pilot (Railway single-replica) and what the swap-to-Redis story should look like. -3. The dispatch wiring in [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications` now publishes to the bus before the email fan-out. Race / ordering / failure-mode review. -4. Auth on the SSE stream — same `require_engineer_or_admin` dep as `/queue` and `/claim`. Browsers can't send custom headers via the native `EventSource` API; the planned frontend uses a fetch-based `ReadableStream` reader (matching the existing `streamDocumentation` pattern in [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts)). Verify that's the right call vs. a query-token scheme. -5. Whether the bus's "drop-on-full-queue" semantic is acceptable, given a stuck subscriber would silently miss live-arrival cards (they'd still see them on next page load via REST `/queue`). +Root causes: +- Multiple stale pytest processes were still alive inside `resolutionflow_backend`, despite the prior handoff saying they were dead. They held `resolutionflow_test` transactions open and caused later tests to block on `DROP SCHEMA public CASCADE`. +- `test_escalations_stream_returns_sse_content_type` used HTTPX `ASGITransport` against an infinite SSE stream. That transport buffers the entire response body before returning, so the test waited forever and held the auth DB dependency transaction open. +- Escalation handoff tests created `intent="escalate"` handoffs without stubbing `_generate_ai_assessment()`, so they waited on the real AI path instead of testing handoff behavior. +- The bus keyed subscribers by raw `account_id`; string UUIDs and `UUID` objects for the same account did not match. -## Resume point (after Codex review) +Fixes made: +- `stream_escalations` now uses `Depends(require_engineer_or_admin, scope="function")` so auth DB dependencies are released before the long-lived stream body. +- The SSE handshake test now calls `stream_escalations()` directly and consumes only the first generator yield, avoiding HTTPX's infinite-stream buffering behavior. +- Handoff manager/API tests stub `_generate_ai_assessment()` with an `AsyncMock`. +- `EscalationBus` normalizes string/UUID account IDs at subscribe/publish/unsubscribe/subscriber_count boundaries, with a regression test. -1. **Get the test suite back to green.** Stale pytest zombies in the container were cleared (PIDs 1790034, 1844996, 1883167, 1916565, 1935830, 2009437, 2009449 — all dead, parent uvicorn-reload didn't reap them; PID slots remain but no live processes). Re-run with `pytest -n auto` to keep wall-clock manageable. Files: `tests/test_escalation_bus.py` (7 tests), the 4 new dispatch + SSE tests in `tests/test_handoff_manager.py` and `tests/test_session_handoffs_api.py`. -2. **Frontend SSE subscription** in `EscalationQueue.tsx` — fetch-based reader, prepend new cards with the locked 200ms slide-in, reconnect with backoff, tab-title flash when backgrounded, respect `prefers-reduced-motion`. Then ship the magic-moment handoff-context screen (4 sections, dissolves into FlowPilot session view). -3. Push the branch + open a draft PR. +Verified: +- `pytest tests/test_escalation_bus.py tests/test_handoff_manager.py tests/test_session_handoffs_api.py tests/test_flowpilot_analytics_escalations.py --override-ini=addopts= -q --durations=20` → `31 passed in 46.95s` +- Same subset with `-n auto` → `31 passed in 17.80s` +- No remaining pytest processes or `resolutionflow%test%` Postgres sessions after the run. -## Stack +## Resume point -``` -WIP (uncommitted): SSE bus + endpoint + dispatcher publish + 7 bus tests + 1 dispatcher test + 2 SSE endpoint tests -a283d0d docs(ai): refresh handoff state mid-flight on Escalation Mode build -9f0bfd4 feat(escalations): mount time-to-first-action stat-card on /escalations -07d0db9 feat(handoff): email engineer-or-admin teammates on escalation -7a5b853 feat(api): role-gate handoff claim to engineer-or-admin -52f6d03 feat(analytics): add escalation time-to-first-action metric endpoint -d51e95c docs(plans): add escalation-mode wedge design + test plan -``` - -## Where things stand - -- CI on `main` still healthy. Branch protection: `CI / frontend (pull_request)` required, `CI / backend (pull_request)` required, `CI / e2e (pull_request)` not yet required. -- The 20 tests passing as of `9f0bfd4` are still passing (last green run logged before the SSE work). The newly added SSE tests (7 bus + 1 dispatcher integration + 2 endpoint) HAVE NOT been verified end-to-end this session — they ran clean on the bus suite alone (7/7 in 0.14s) but the DB-backed integration tests were aborted before completing. -- The plan doc at [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md) is the source of truth for every UI / metric / scope decision. The embedded **GSTACK REVIEW REPORT** at the bottom shows Eng + Design CLEARED and Codex INFO from the design-stage pass. +1. Continue the **Frontend SSE subscription** in `EscalationQueue.tsx`: fetch-based reader, prepend new cards with the locked 200ms slide-in, reconnect with backoff, tab-title flash when backgrounded, respect `prefers-reduced-motion`. +2. Then ship the **magic-moment handoff-context screen**: 4 sections (problem summary / what's been tried / AI assessment / Start here CTA), loads on Pick Up, then dissolves into regular FlowPilot session view. +3. Push the branch and open a draft PR when the frontend/live-arrival slice is ready. ## Useful breadcrumbs -- Metric endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics` at the bottom. -- Notification dispatch (email + bus publish): [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`. Wired in [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) **after** `db.commit()` so a rolled-back handoff never emails or fans out. -- SSE endpoint (WIP): [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations`. Heartbeat every 25s, account-scoped subscribe, role-gated to engineer-or-admin. -- Pub/sub bus (WIP): [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). Module-level singleton, in-memory, `asyncio.Queue` per subscriber with 64-event maxsize and drop-on-full semantics. -- Frontend stat-card: [`frontend/src/components/flowpilot/EscalationMetricCard.tsx`](../frontend/src/components/flowpilot/EscalationMetricCard.tsx). Renders `n_with_action / n_claimed`, avg + median, and the metric_definition disclaimer. -- Two-metric framing — required reading before quoting any number to a pilot. In-product endpoint measures *post-claim time-to-first-action*; the savings claim is `manual_baseline − in_product`. Manual baseline comes from the founder's stopwatch on the next 5 escalations (The Assignment in the design doc). -- The `notification_sent` boolean is intentionally NOT being written. Per Codex's design-stage correction it should be replaced by per-channel delivery records; v1.x story. For now application logs are the audit trail. +- SSE endpoint: [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations`. +- Pub/sub bus: [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). In-memory, account-scoped, non-durable, 64-event per-subscriber queue, drop-on-full. +- Notification dispatch: [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`, called after `db.commit()` in the handoff endpoint. +- Frontend streaming reference: [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts) — `streamDocumentation` uses fetch + `ReadableStream`, which remains the right pattern because native `EventSource` cannot send auth headers. +- Metric endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics`. ## Watch-outs -- `ai_session_step` has NO `user_id` column — the metric query keys "first action by senior" off `session_id + created_at > claimed_at`. Fine for v1 because session activity post-claim IS the senior's activity (session reactivates under `escalated_to_id`). -- `account_id` is denormalized on `ai_session_step` (Phase 4 RLS pattern). Use it directly; don't join through `ai_sessions`. -- POST `/handoff` still requires the session owner to be the escalator (`AISession.user_id == current_user.id`). Peer-tech escalation is a v2 TODO. -- The test suite uses `DROP SCHEMA public CASCADE` + `CREATE SCHEMA public` per test (see [`backend/tests/conftest.py:144`](../backend/tests/conftest.py#L144)). Concurrent pytest runs against the same test DB collide. Always run one suite at a time, or via `-n auto` xdist with the per-worker-DB isolation already in conftest. - -## Kill-switch (week 8) - -If 0 of 3 pilots produce a verifiable hours-saved-per-week number above 1.0, revisit the wedge. The design doc names the alternative direction (deterministic-ops territory) but data lands first. +- Do not reintroduce `client.stream()`/ASGITransport tests for infinite SSE responses; test the generator directly or use a real server-level test. +- `DROP SCHEMA public CASCADE` per test is still the dominant cost: DB-backed tests spend ~1.7-2.8s in setup. Use `-n auto` for focused backend loops. +- The bus is acceptable for v1 pilot scale only because Railway is single-replica. Redis pub/sub is the obvious swap when horizontal scaling appears. +- Synchronous `_generate_ai_assessment()` during escalation creation remains product-latency risk; tests are now isolated from it, but the UX path should be watched as the magic-moment screen is built. diff --git a/.ai/SESSION_LOG.md b/.ai/SESSION_LOG.md index 04cf1e06..7c0ee399 100644 --- a/.ai/SESSION_LOG.md +++ b/.ai/SESSION_LOG.md @@ -12,6 +12,18 @@ --- +## 2026-04-27 19:50 EDT — Codex — Stabilize Escalation Mode SSE backend tests + +- Diagnosed slow backend tests on `feat/escalation-metric-endpoint`. Multiple stale pytest processes were still alive inside `resolutionflow_backend` and held `resolutionflow_test` transactions open, blocking later per-test schema resets on `DROP SCHEMA public CASCADE`. +- Reproduced a deterministic hang in `test_escalations_stream_returns_sse_content_type`: HTTPX `ASGITransport` buffers the full response body before returning, so an infinite SSE response never yielded the initial chunk and kept the auth DB dependency transaction open. +- Fixed `stream_escalations` to release auth dependencies before the long-lived stream body with `Depends(..., scope="function")`. +- Reworked the SSE handshake test to call `stream_escalations()` directly and consume one generator yield, then close it; kept viewer role-gate coverage through the API client. +- Stubbed `_generate_ai_assessment()` in handoff manager/API tests so escalation handoff tests no longer wait on the real AI path. +- Normalized account IDs inside `EscalationBus` so string UUIDs and `UUID` objects hit the same subscriber bucket; added a regression test. +- Verified focused backend subset: serial `31 passed in 46.95s`; xdist `31 passed in 17.80s`. Confirmed no lingering pytest processes or test DB sessions afterward. +- Left for next session: continue frontend SSE subscription in `EscalationQueue.tsx`, then the magic-moment handoff-context screen. +- Files touched: `backend/app/api/endpoints/session_handoffs.py`, `backend/app/core/escalation_bus.py`, `backend/tests/test_escalation_bus.py`, `backend/tests/test_handoff_manager.py`, `backend/tests/test_session_handoffs_api.py`, `.ai/HANDOFF.md`, `.ai/SESSION_LOG.md`. + ## 2026-04-26 03:50 EDT — Claude Code — Ship AssistantChatPage prefill `currentChatRef` fix; close out PR #150 - User reported a troubleshooting-session bug: after answering a subset of task-lane questions and clicking *Send N of M Responses*, no AI response appeared. Traced to `AssistantChatPage`: the dashboard prefill effect set `activeChatId` after creating a new chat session but never updated `currentChatRef.current`. The `currentChatRef.current !== sentForChatId` guard in `handleSend` and `handleTaskSubmit` then bailed silently on every later request and discarded the AI's reply. The user message was already pushed to the chat before the await, so the user saw their answers but nothing else. diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 5b62a3c5..ce74e008 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -156,7 +156,10 @@ _QUEUE_GET_TIMEOUT_S = 25 # < heartbeat so heartbeat fires reliably @queue_router.get("/escalations/stream") async def stream_escalations( request: Request, - current_user: Annotated[User, Depends(require_engineer_or_admin)], + current_user: Annotated[ + User, + Depends(require_engineer_or_admin, scope="function"), + ], ): """SSE stream of new escalation arrivals for the current user's account. diff --git a/backend/app/core/escalation_bus.py b/backend/app/core/escalation_bus.py index bf623950..8102cae3 100644 --- a/backend/app/core/escalation_bus.py +++ b/backend/app/core/escalation_bus.py @@ -38,39 +38,46 @@ class EscalationBus: self._subscribers: dict[UUID, set[asyncio.Queue[dict[str, Any]]]] = {} self._lock = asyncio.Lock() - async def subscribe(self, account_id: UUID) -> asyncio.Queue[dict[str, Any]]: + @staticmethod + def _normalize_account_id(account_id: UUID | str) -> UUID: + return account_id if isinstance(account_id, UUID) else UUID(str(account_id)) + + async def subscribe(self, account_id: UUID | str) -> asyncio.Queue[dict[str, Any]]: """Register a new subscriber queue for an account. Caller must invoke `unsubscribe(account_id, queue)` when the consumer disconnects. """ + normalized_account_id = self._normalize_account_id(account_id) queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue( maxsize=_QUEUE_MAXSIZE ) async with self._lock: - self._subscribers.setdefault(account_id, set()).add(queue) + self._subscribers.setdefault(normalized_account_id, set()).add(queue) return queue async def unsubscribe( - self, account_id: UUID, queue: asyncio.Queue[dict[str, Any]] + self, account_id: UUID | str, queue: asyncio.Queue[dict[str, Any]] ) -> None: + normalized_account_id = self._normalize_account_id(account_id) async with self._lock: - subs = self._subscribers.get(account_id) + subs = self._subscribers.get(normalized_account_id) if subs is None: return subs.discard(queue) if not subs: - self._subscribers.pop(account_id, None) + self._subscribers.pop(normalized_account_id, None) - async def publish(self, account_id: UUID, event: dict[str, Any]) -> int: + async def publish(self, account_id: UUID | str, event: dict[str, Any]) -> int: """Fan event out to every subscriber for `account_id`. Returns the number of subscribers that successfully received the event. Drops the event for any subscriber whose queue is full (logs at warning level). """ + normalized_account_id = self._normalize_account_id(account_id) async with self._lock: - subs = list(self._subscribers.get(account_id, ())) + subs = list(self._subscribers.get(normalized_account_id, ())) if not subs: return 0 delivered = 0 @@ -82,14 +89,15 @@ class EscalationBus: logger.warning( "EscalationBus: dropped event for full subscriber queue " "(account_id=%s, event=%s)", - account_id, + normalized_account_id, event.get("type", "?"), ) return delivered - def subscriber_count(self, account_id: UUID) -> int: + def subscriber_count(self, account_id: UUID | str) -> int: """Diagnostic — number of active subscribers for an account.""" - return len(self._subscribers.get(account_id, ())) + normalized_account_id = self._normalize_account_id(account_id) + return len(self._subscribers.get(normalized_account_id, ())) # Module-level singleton. FastAPI imports this; `subscribe()` and `publish()` diff --git a/backend/tests/test_escalation_bus.py b/backend/tests/test_escalation_bus.py index 50d10f3c..5f77aa58 100644 --- a/backend/tests/test_escalation_bus.py +++ b/backend/tests/test_escalation_bus.py @@ -68,6 +68,21 @@ async def test_subscriber_in_other_account_does_not_receive(): await bus.unsubscribe(account_b, q_b) +@pytest.mark.asyncio +async def test_publish_normalizes_string_uuid_account_id(): + """ORM-created objects can briefly carry string UUIDs in-memory.""" + bus = EscalationBus() + account = uuid4() + queue = await bus.subscribe(account) + try: + delivered = await bus.publish(str(account), {"type": "x"}) + assert delivered == 1 + event = await asyncio.wait_for(queue.get(), timeout=1.0) + assert event == {"type": "x"} + finally: + await bus.unsubscribe(str(account), queue) + + @pytest.mark.asyncio async def test_unsubscribe_drops_subscriber_count_to_zero(): bus = EscalationBus() diff --git a/backend/tests/test_handoff_manager.py b/backend/tests/test_handoff_manager.py index 3a2836a5..dc54a82d 100644 --- a/backend/tests/test_handoff_manager.py +++ b/backend/tests/test_handoff_manager.py @@ -9,6 +9,26 @@ from app.models.user import User from app.services.handoff_manager import HandoffManager +@pytest.fixture(autouse=True) +def stub_ai_assessment(): + """Keep handoff tests focused on handoff behavior, not external AI calls.""" + with patch.object( + HandoffManager, + "_generate_ai_assessment", + new=AsyncMock( + return_value=( + "Stub escalation assessment", + { + "likely_cause": "Stub", + "suggested_steps": [], + "confidence": "medium", + }, + ) + ), + ): + yield + + @pytest.mark.asyncio async def test_create_park_handoff(client: AsyncClient, test_user, auth_headers, test_db): """Parking a session creates a handoff with snapshot.""" diff --git a/backend/tests/test_session_handoffs_api.py b/backend/tests/test_session_handoffs_api.py index 6ddc307c..64682c2d 100644 --- a/backend/tests/test_session_handoffs_api.py +++ b/backend/tests/test_session_handoffs_api.py @@ -1,12 +1,41 @@ """API endpoint tests for session handoffs.""" +from unittest.mock import AsyncMock, patch from uuid import UUID as PyUUID import pytest from httpx import AsyncClient from sqlalchemy import select +from app.api.endpoints.session_handoffs import stream_escalations +from app.core.escalation_bus import bus as escalation_bus from app.models.ai_session import AISession from app.models.user import User +from app.services.handoff_manager import HandoffManager + + +class _ConnectedRequest: + async def is_disconnected(self) -> bool: + return False + + +@pytest.fixture(autouse=True) +def stub_ai_assessment(): + """Endpoint tests should not wait on the external AI assessment path.""" + with patch.object( + HandoffManager, + "_generate_ai_assessment", + new=AsyncMock( + return_value=( + "Stub escalation assessment", + { + "likely_cause": "Stub", + "suggested_steps": [], + "confidence": "medium", + }, + ) + ), + ): + yield @pytest.mark.asyncio @@ -137,23 +166,30 @@ async def test_escalations_stream_returns_sse_content_type( ): """Engineer/owner can open the SSE stream and gets text/event-stream plus an initial `ready` event. Read just enough bytes to confirm the - handshake — the full pub/sub flow is covered by the bus + dispatcher - tests separately.""" - async with client.stream( - "GET", - "/api/v1/ai-sessions/escalations/stream", - headers=auth_headers, - ) as resp: - assert resp.status_code == 200 - assert resp.headers["content-type"].startswith("text/event-stream") - # First chunk must contain the ready event. - first = b"" - async for chunk in resp.aiter_bytes(): - first += chunk - if b"event: ready" in first and b"\n\n" in first: - break - assert b"event: ready" in first - assert b'"account_id"' in first + handshake — the full pub/sub flow is covered by the bus + dispatcher tests + separately. + + Do not use `client.stream()` here: HTTPX's ASGITransport buffers the whole + response body before returning, which hangs forever for an infinite SSE + stream. + """ + user_id = PyUUID(test_user["user_data"]["id"]) + user = ( + await test_db.execute(select(User).where(User.id == user_id)) + ).scalar_one() + + resp = await stream_escalations(_ConnectedRequest(), current_user=user) + assert resp.media_type == "text/event-stream" + + body_iterator = resp.body_iterator + try: + first = await anext(body_iterator) + finally: + await body_iterator.aclose() + + assert "event: ready" in first + assert '"account_id"' in first + assert escalation_bus.subscriber_count(user.account_id) == 0 @pytest.mark.asyncio -- 2.49.1 From fff8338bf2063be3ee001bc45f6e90f8b2d3c96d Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 19:55:31 -0400 Subject: [PATCH 10/34] docs(ai): track escalation assessment latency follow-up Co-Authored-By: Codex --- .ai/TODO.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.ai/TODO.md b/.ai/TODO.md index 3f5ab56d..f6e91d74 100644 --- a/.ai/TODO.md +++ b/.ai/TODO.md @@ -16,6 +16,8 @@ - [ ] **Consider `pytest-testmon` for PR-time test selection.** Tracks which tests touched which source files and only re-runs affected ones. Best for small PRs touching ~few files. Adds cache-invalidation complexity; only worth it if the suite stays painfully long even after xdist. - [ ] **AssistantChatPage `currentChatRef` guard is a silent return** — `handleSend`, `handleTaskSubmit`, `selectChat`, `refreshFacts`, `refreshActiveFix`, and `refreshPreview` all bail with `if (currentChatRef.current !== sentForChatId) return` when stale. This is by design for chat switching, but it also silently masked the prefill-ref bug fixed in PR #153 — the user just saw "no AI response" with no log, no toast, no Sentry event. Either (a) log a `console.warn`/Sentry breadcrumb on the mismatch path so future drift is visible, or (b) split "expected stale" (chat switch) from "unexpected stale" (ref never updated) so only the latter alerts. Pair with an audit of every `currentChatRef.current = ...` assignment vs every `setActiveChatId(...)` call to make sure they're paired everywhere. +- [ ] **Make escalation AI assessment non-blocking or latency-bounded.** `HandoffManager.create_handoff(intent="escalate")` currently calls `_generate_ai_assessment()` synchronously before the handoff commit. Tests now stub this path, but the product path can still make the junior tech's Escalate action wait on model/network latency. For v1, either set a strict timeout with graceful fallback or move assessment generation behind the committed handoff and let the handoff-context screen render partial state until the assessment arrives. + - [ ] **Allow peer-tech to escalate a colleague's session.** Today `POST /ai-sessions/{session_id}/handoff` in [endpoints/session_handoffs.py:48](backend/app/api/endpoints/session_handoffs.py#L48) filters by `AISession.user_id == current_user.id`, so only the session owner can escalate. Real MSP shops have peer hand-offs: Junior A is on lunch, Junior B sees the session is stuck and should be able to escalate it. Auth tweak: switch from session-owner check to `require_engineer_or_admin` + same-account scope. Add a `handed_off_by` audit column (already exists on `SessionHandoff`) so the original-owner-vs-actual-escalator distinction is preserved. Surfaced from /plan-eng-review on the Escalation-Mode wedge plan; v1 wedge demo doesn't need this (solo-founder pilot), but capture for v2 once 3+ pilots are live and a peer-claim need surfaces. - [ ] **Mobile/responsive design for EscalationQueue + handoff-context screen.** Pre-PMF wedge demo targets desktop only — MSP techs work on laptops/desktops in shop environments. Once 3+ paying customers exist and a tech requests mobile (likely on-call use case), spec the responsive behavior: stacked card layout below `sm:` breakpoint, full-bleed handoff-context overlay on mobile, swipe-to-claim gesture instead of Pick Up button. Surfaced from /plan-design-review on the Escalation-Mode wedge plan. -- 2.49.1 From 9bdd9959a84d57b32c41a0ad8355b7b6062888b4 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 20:03:14 -0400 Subject: [PATCH 11/34] fix(handoff): bound escalation assessment latency Co-Authored-By: Codex --- .ai/HANDOFF.md | 4 +- .ai/SESSION_LOG.md | 3 +- .ai/TODO.md | 2 - backend/app/core/config.py | 1 + backend/app/services/handoff_manager.py | 22 ++++++++++- backend/tests/test_handoff_manager.py | 50 +++++++++++++++++++++++++ 6 files changed, 77 insertions(+), 5 deletions(-) diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index 8ed216bf..e7654915 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -23,10 +23,12 @@ Fixes made: - The SSE handshake test now calls `stream_escalations()` directly and consumes only the first generator yield, avoiding HTTPX's infinite-stream buffering behavior. - Handoff manager/API tests stub `_generate_ai_assessment()` with an `AsyncMock`. - `EscalationBus` normalizes string/UUID account IDs at subscribe/publish/unsubscribe/subscriber_count boundaries, with a regression test. +- Follow-up fix: escalation AI assessment is now latency-bounded by `ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS` (default 5s). If it times out, handoff creation proceeds with no assessment instead of blocking on the model/network path. Verified: - `pytest tests/test_escalation_bus.py tests/test_handoff_manager.py tests/test_session_handoffs_api.py tests/test_flowpilot_analytics_escalations.py --override-ini=addopts= -q --durations=20` → `31 passed in 46.95s` - Same subset with `-n auto` → `31 passed in 17.80s` +- After the assessment-timeout fix: same subset with `-n auto` → `32 passed in 17.77s` - No remaining pytest processes or `resolutionflow%test%` Postgres sessions after the run. ## Resume point @@ -48,4 +50,4 @@ Verified: - Do not reintroduce `client.stream()`/ASGITransport tests for infinite SSE responses; test the generator directly or use a real server-level test. - `DROP SCHEMA public CASCADE` per test is still the dominant cost: DB-backed tests spend ~1.7-2.8s in setup. Use `-n auto` for focused backend loops. - The bus is acceptable for v1 pilot scale only because Railway is single-replica. Redis pub/sub is the obvious swap when horizontal scaling appears. -- Synchronous `_generate_ai_assessment()` during escalation creation remains product-latency risk; tests are now isolated from it, but the UX path should be watched as the magic-moment screen is built. +- Escalation assessment can be missing when the 5s timeout fires. The handoff-context UI must render a graceful "assessment unavailable/in progress" state rather than treating it as required. diff --git a/.ai/SESSION_LOG.md b/.ai/SESSION_LOG.md index 7c0ee399..21ad2f09 100644 --- a/.ai/SESSION_LOG.md +++ b/.ai/SESSION_LOG.md @@ -21,8 +21,9 @@ - Stubbed `_generate_ai_assessment()` in handoff manager/API tests so escalation handoff tests no longer wait on the real AI path. - Normalized account IDs inside `EscalationBus` so string UUIDs and `UUID` objects hit the same subscriber bucket; added a regression test. - Verified focused backend subset: serial `31 passed in 46.95s`; xdist `31 passed in 17.80s`. Confirmed no lingering pytest processes or test DB sessions afterward. +- Follow-up in the same session: fixed the product latency risk by adding `ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS` (default 5s) around escalation AI assessment generation. If the optional assessment times out, handoff creation continues with no assessment. Added regression coverage; focused xdist subset now `32 passed in 17.77s`. - Left for next session: continue frontend SSE subscription in `EscalationQueue.tsx`, then the magic-moment handoff-context screen. -- Files touched: `backend/app/api/endpoints/session_handoffs.py`, `backend/app/core/escalation_bus.py`, `backend/tests/test_escalation_bus.py`, `backend/tests/test_handoff_manager.py`, `backend/tests/test_session_handoffs_api.py`, `.ai/HANDOFF.md`, `.ai/SESSION_LOG.md`. +- Files touched: `backend/app/api/endpoints/session_handoffs.py`, `backend/app/core/config.py`, `backend/app/core/escalation_bus.py`, `backend/app/services/handoff_manager.py`, `backend/tests/test_escalation_bus.py`, `backend/tests/test_handoff_manager.py`, `backend/tests/test_session_handoffs_api.py`, `.ai/HANDOFF.md`, `.ai/SESSION_LOG.md`, `.ai/TODO.md`. ## 2026-04-26 03:50 EDT — Claude Code — Ship AssistantChatPage prefill `currentChatRef` fix; close out PR #150 diff --git a/.ai/TODO.md b/.ai/TODO.md index f6e91d74..3f5ab56d 100644 --- a/.ai/TODO.md +++ b/.ai/TODO.md @@ -16,8 +16,6 @@ - [ ] **Consider `pytest-testmon` for PR-time test selection.** Tracks which tests touched which source files and only re-runs affected ones. Best for small PRs touching ~few files. Adds cache-invalidation complexity; only worth it if the suite stays painfully long even after xdist. - [ ] **AssistantChatPage `currentChatRef` guard is a silent return** — `handleSend`, `handleTaskSubmit`, `selectChat`, `refreshFacts`, `refreshActiveFix`, and `refreshPreview` all bail with `if (currentChatRef.current !== sentForChatId) return` when stale. This is by design for chat switching, but it also silently masked the prefill-ref bug fixed in PR #153 — the user just saw "no AI response" with no log, no toast, no Sentry event. Either (a) log a `console.warn`/Sentry breadcrumb on the mismatch path so future drift is visible, or (b) split "expected stale" (chat switch) from "unexpected stale" (ref never updated) so only the latter alerts. Pair with an audit of every `currentChatRef.current = ...` assignment vs every `setActiveChatId(...)` call to make sure they're paired everywhere. -- [ ] **Make escalation AI assessment non-blocking or latency-bounded.** `HandoffManager.create_handoff(intent="escalate")` currently calls `_generate_ai_assessment()` synchronously before the handoff commit. Tests now stub this path, but the product path can still make the junior tech's Escalate action wait on model/network latency. For v1, either set a strict timeout with graceful fallback or move assessment generation behind the committed handoff and let the handoff-context screen render partial state until the assessment arrives. - - [ ] **Allow peer-tech to escalate a colleague's session.** Today `POST /ai-sessions/{session_id}/handoff` in [endpoints/session_handoffs.py:48](backend/app/api/endpoints/session_handoffs.py#L48) filters by `AISession.user_id == current_user.id`, so only the session owner can escalate. Real MSP shops have peer hand-offs: Junior A is on lunch, Junior B sees the session is stuck and should be able to escalate it. Auth tweak: switch from session-owner check to `require_engineer_or_admin` + same-account scope. Add a `handed_off_by` audit column (already exists on `SessionHandoff`) so the original-owner-vs-actual-escalator distinction is preserved. Surfaced from /plan-eng-review on the Escalation-Mode wedge plan; v1 wedge demo doesn't need this (solo-founder pilot), but capture for v2 once 3+ pilots are live and a peer-claim need surfaces. - [ ] **Mobile/responsive design for EscalationQueue + handoff-context screen.** Pre-PMF wedge demo targets desktop only — MSP techs work on laptops/desktops in shop environments. Once 3+ paying customers exist and a tech requests mobile (likely on-call use case), spec the responsive behavior: stacked card layout below `sm:` breakpoint, full-bleed handoff-context overlay on mobile, swipe-to-claim gesture instead of Pick Up button. Surfaced from /plan-design-review on the Escalation-Mode wedge plan. diff --git a/backend/app/core/config.py b/backend/app/core/config.py index 0363bf8e..985bca98 100644 --- a/backend/app/core/config.py +++ b/backend/app/core/config.py @@ -111,6 +111,7 @@ class Settings(BaseSettings): GOOGLE_AI_API_KEY: Optional[str] = None AI_MODEL_GEMINI: str = "gemini-2.5-flash" AI_MODEL_ANTHROPIC: str = "claude-sonnet-4-6" + ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS: int = 5 # Model tier routing — maps action types to model tiers AI_MODEL_TIERS: dict[str, str] = { diff --git a/backend/app/services/handoff_manager.py b/backend/app/services/handoff_manager.py index bc3717f9..270882db 100644 --- a/backend/app/services/handoff_manager.py +++ b/backend/app/services/handoff_manager.py @@ -57,7 +57,9 @@ class HandoffManager: ai_assessment = None ai_assessment_data = None if intent == "escalate": - ai_assessment, ai_assessment_data = await self._generate_ai_assessment(session) + ai_assessment, ai_assessment_data = ( + await self._generate_ai_assessment_with_timeout(session) + ) handoff = SessionHandoff( session_id=session_id, @@ -311,6 +313,24 @@ class HandoffManager: logger.exception("Failed to generate AI assessment") return None, None + async def _generate_ai_assessment_with_timeout( + self, session: AISession + ) -> tuple[str | None, dict[str, Any] | None]: + """Generate optional escalation assessment within the click-path budget.""" + timeout = settings.ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS + try: + return await asyncio.wait_for( + self._generate_ai_assessment(session), + timeout=timeout, + ) + except asyncio.TimeoutError: + logger.warning( + "Escalation AI assessment timed out after %ss for session %s", + timeout, + session.id, + ) + return None, None + async def generate_briefing( self, handoff_id: UUID, claiming_user_id: UUID ) -> str: diff --git a/backend/tests/test_handoff_manager.py b/backend/tests/test_handoff_manager.py index dc54a82d..a2e75c05 100644 --- a/backend/tests/test_handoff_manager.py +++ b/backend/tests/test_handoff_manager.py @@ -1,4 +1,5 @@ """Integration tests for HandoffManager service.""" +import asyncio from unittest.mock import AsyncMock, patch import pytest @@ -101,6 +102,55 @@ async def test_create_escalate_handoff(client: AsyncClient, test_user, auth_head assert "branch_map" in session.escalation_package or "snapshot" in session.escalation_package +@pytest.mark.asyncio +async def test_create_escalate_handoff_does_not_wait_on_slow_ai_assessment( + client: AsyncClient, test_user, auth_headers, test_db, monkeypatch +): + """Escalate should commit a handoff even when optional AI assessment is slow.""" + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.flush() + + async def slow_assessment(self, session): + await asyncio.sleep(0.2) + return "too slow", {"confidence": "medium"} + + monkeypatch.setattr( + "app.services.handoff_manager.settings." + "ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS", + 0.01, + ) + with patch.object( + HandoffManager, + "_generate_ai_assessment", + new=slow_assessment, + ): + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="Need senior help", + user_id=test_user["user_data"]["id"], + ) + + assert handoff.intent == "escalate" + assert handoff.ai_assessment is None + assert handoff.ai_assessment_data is None + + await test_db.refresh(session) + assert session.status == "escalated" + assert session.handoff_count == 1 + + @pytest.mark.asyncio async def test_claim_session(client: AsyncClient, test_user, test_admin, auth_headers, test_db): """Claiming a handoff sets claimed_by and reactivates session.""" -- 2.49.1 From 02d5c6c08cfe148c0a3cceea2ed1178530256c40 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 20:13:40 -0400 Subject: [PATCH 12/34] docs(ai): refresh handoff state for next-session pickup under 200k context MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Default Claude Code model is being switched from Opus 4.7 1M-context to Opus 4.7 (200k). Tighten the per-session pickup docs so they're self-sufficient under the smaller window: - CURRENT_TASK now reflects the post-Codex state: 8 commits on the branch (5 feat + WIP SSE + 2 Codex test/latency fixes + 1 doc refresh), 32/32 backend tests with -n auto, frontend tsc -b clean. Remaining work re-scoped: the SSE backend half is feature-complete and tested, so what's left is the FRONTEND SSE subscription in EscalationQueue.tsx, then the magic-moment handoff-context screen, then push + draft PR. - Session log gets a Claude Code entry covering today's planning → build → pause-for-Codex arc, the design decisions locked into the doc and code, the two TODOs added (peer-tech escalation, mobile responsive), and the model-switch context for the next session. - HANDOFF.md needs no change — Codex's update in 9bdd995 already describes the resume point and watch-outs cleanly. No code change. Co-Authored-By: Claude Opus 4.7 --- .ai/CURRENT_TASK.md | 37 +++++++++++++++++++++---------------- .ai/SESSION_LOG.md | 12 ++++++++++++ 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/.ai/CURRENT_TASK.md b/.ai/CURRENT_TASK.md index 5e8d9314..6dd60035 100644 --- a/.ai/CURRENT_TASK.md +++ b/.ai/CURRENT_TASK.md @@ -2,29 +2,34 @@ **Task:** Build **Escalation Mode** — the wedge for ResolutionFlow's GTM (first paying-customer push). When a junior tech escalates a FlowPilot session, the senior tech sees structured handoff context in seconds instead of running a 5-minute verbal "tell me what you tried" call. -**Status:** in-flight on `feat/escalation-mode` (currently `feat/escalation-metric-endpoint`). Backend metric + role gate + email notification shipped. Frontend stat-card mounted. **Next:** WebSocket/SSE push (live-arrival half of the dual-path) and the magic-moment handoff-context screen. +**Status:** in-flight on `feat/escalation-metric-endpoint`. Backend is **feature-complete and test-stabilized**. **Next:** frontend SSE subscription in `EscalationQueue.tsx`, then the magic-moment handoff-context screen, then push + draft PR. -**Plan:** [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md). Reviewed by `/office-hours`, `/plan-eng-review`, `/plan-design-review`, `/codex review`. Eng + Design CLEARED. Codex's two-metric correction + claim-role-gate + per-channel notification model all applied to the plan and the code. +**Plan:** [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md). Reviewed by `/office-hours`, `/plan-eng-review`, `/plan-design-review`, `/codex review`. Eng + Design CLEARED. Codex's two-metric correction + claim role gate + per-channel notification model + SSE bus diagnostics all applied. -**Test plan artifact:** [`docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md`](../docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md) — primary input for `/qa` once the build is feature-complete. +**Test plan artifact:** [`docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md`](../docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md) — primary input for `/qa` once feature-complete. -## Done so far on `feat/escalation-metric-endpoint` +## Done on `feat/escalation-metric-endpoint` (8 commits, branched from `main` @ `c0ed6d9`) | Commit | What it ships | |---|---| -| `d51e95c` | Plan + test-plan artifacts checked in | -| `52f6d03` | `GET /analytics/flowpilot/escalations` — in-product time-to-first-action; account-scoped, engineer-or-admin gated; 9 tests including multi-tenant isolation | -| `7a5b853` | Role-gate POST `/handoffs/{id}/claim` to engineer-or-admin (was viewer-claimable); 2 tests | -| `07d0db9` | `HandoffManager.dispatch_escalation_notifications` — emails engineer/admin teammates on intent=escalate; graceful-degradation regression test; 4 tests | -| `9f0bfd4` | `EscalationMetricCard` mounted above the queue list; consumes the new endpoint; matches DESIGN-SYSTEM tokens | +| `d51e95c` | Plan + test-plan artifacts | +| `52f6d03` | `GET /analytics/flowpilot/escalations` — in-product time-to-first-action; account-scoped, engineer-or-admin gated | +| `7a5b853` | Role-gate POST `/handoffs/{id}/claim` to engineer-or-admin | +| `07d0db9` | `HandoffManager.dispatch_escalation_notifications` — emails engineer/admin teammates on intent=escalate; graceful-degradation regression | +| `9f0bfd4` | `EscalationMetricCard` mounted above the queue list | +| `a283d0d` | `.ai/` mid-flight refresh | +| `87bd0b7` | **WIP** marker for the SSE backend slice (paused for Codex pass) | +| `bc15952` | Codex: stabilize SSE backend tests — `Depends(..., scope="function")` releases auth DB deps before the long-lived stream body; SSE handshake test calls the generator directly; AI-assessment stub fixture; bus normalizes string vs UUID account_id | +| `fff8338` | Doc-only: track escalation assessment latency follow-up | +| `9bdd995` | Bound escalation assessment latency to `ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS` (default 5s); handoff still creates if assessment times out | -20 backend tests green across handoff_manager + session_handoffs_api + flowpilot_analytics_escalations. Frontend `tsc -b` clean. Nothing pushed yet. +**Test status:** focused subset (`test_escalation_bus`, `test_handoff_manager`, `test_session_handoffs_api`, `test_flowpilot_analytics_escalations`) → `32 passed in 17.77s` with `-n auto`. Frontend `tsc -b` clean. Branch not pushed. ## Remaining work on this branch -1. **WebSocket/SSE push** for live escalation arrival in the queue — the second half of the notification dual-path. Senior already on the queue page sees a new card slide in within ~1s of the junior hitting Escalate. ~3-4 days of work split across multiple commits (connection manager, auth-scoped fan-out, frontend EventSource handling, reconnect, slide-in animation, tab-title flash). -2. **Magic-moment handoff-context screen** — 4-section view (problem summary / what's been tried / AI assessment / Start here CTA) that loads on Pick Up before dissolving into the regular FlowPilot session view. ~1.5-2 days. -3. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. +1. **Frontend SSE subscription** in `EscalationQueue.tsx`. Use a fetch-based `ReadableStream` reader (matching [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts) `streamDocumentation` — native `EventSource` can't send auth headers). Prepend new cards with the locked 200ms slide-in. Reconnect with backoff. Tab-title flash when backgrounded. Respect `prefers-reduced-motion`. +2. **Magic-moment handoff-context screen** — 4-section view (problem summary / what's been tried / AI assessment / Start here CTA) that loads on Pick Up before dissolving into the regular FlowPilot session view. ~1.5-2 days. Must render gracefully when `ai_assessment` is `None` (assessment timed out — see `9bdd995`). +3. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. Optional for v1 demo. 4. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. ## Two-metric framing — read this before quoting numbers to anyone @@ -33,10 +38,10 @@ The in-product endpoint measures *post-claim time-to-first-action*. The "minutes ## Kill-switch -Week 8: if 0 of 3 pilots produce a verifiable hours-saved-per-week number above 1.0, revisit the wedge. The design doc names the alternative (deterministic-ops territory) for context, but don't pivot before the data lands. +Week 8: if 0 of 3 pilots produce a verifiable hours-saved-per-week number above 1.0, revisit the wedge. The design doc names the alternative direction (deterministic-ops territory) for context, but data lands first. ## Previous task — closed out -**Task:** Land PR #153 — fix the `AssistantChatPage` prefill `currentChatRef` bug. **Status:** complete (2026-04-26). Merged as `68fcdc6` on `main`. E2e regression test now in the suite. +**Task:** Land PR #153 — fix the `AssistantChatPage` prefill `currentChatRef` bug. **Status:** complete (2026-04-26). Merged as `68fcdc6` on `main`. -**Background CI item, not blocking:** promoting `CI / e2e (pull_request)` to required on `main`. Two consecutive green PR runs (#150 and #153) cleared the threshold. Ops-only. +**Background CI item, not blocking:** promoting `CI / e2e (pull_request)` to required on `main`. Two consecutive green runs cleared the threshold. Ops-only. diff --git a/.ai/SESSION_LOG.md b/.ai/SESSION_LOG.md index 21ad2f09..aff0f49b 100644 --- a/.ai/SESSION_LOG.md +++ b/.ai/SESSION_LOG.md @@ -12,6 +12,18 @@ --- +## 2026-04-27 EDT — Claude Code — Escalation Mode wedge: design through SSE backend (8 commits) + +- One long session that produced the entire planning artifact stack and most of the backend for the Escalation Mode wedge. Output of `/office-hours` (8 founder-signal session, top-tier YC archetype indicators), `/plan-eng-review` (scope reduced from "2-3 weeks greenfield" to "~6-9 days integration + metric + polish" once the existing handoff_manager surface was inventoried), `/plan-design-review` (6/10 → 9/10 with magic-moment screen, hero metric placement, and real-time arrival visual locked), and `/codex review` (12 findings, 6 applied — two-metric framing, notification routing, claim auth gate moved in-scope, unread-state fix, "Start here" CTA reframe, per-channel delivery model; 5 rejected including the full-scope reduction Codex pushed for). +- Branched `feat/escalation-metric-endpoint` off `main` @ `c0ed6d9`. Stack at session end: `d51e95c` plan + test-plan artifacts; `52f6d03` `GET /analytics/flowpilot/escalations` endpoint with 9 tests including multi-tenant isolation; `7a5b853` claim-endpoint role gate; `07d0db9` email dispatch on escalate with graceful-degradation regression; `9f0bfd4` `EscalationMetricCard` mounted above the queue list; `a283d0d` mid-flight `.ai/` refresh; `87bd0b7` WIP commit for SSE pub/sub bus + endpoint + 7 bus unit tests + 1 dispatcher integration test + 2 endpoint tests; `ba46fc5` paused-for-Codex-review handoff. Codex picked up from `ba46fc5` and added `bc15952` / `fff8338` / `9bdd995` (test stabilization + assessment latency bound). +- Pause was forced by a runaway local test loop: multiple stale `pytest` processes were left inside `resolutionflow_backend` after several aborted runs and contended on the same Postgres test schema. Codex diagnosed and fixed (see entry above). +- Frontend: thin slice — added `getEscalationMetrics` to `flowpilotAnalyticsApi`, the `EscalationMetricCard` component (loading / error / zero-data states + avg + median + conversion-rate + the inline two-metric disclaimer), and mounted it above `EscalationQueue`. `tsc -b` clean. +- Plan-stage UI decisions locked into the design doc and the codebase: dedicated 4-section magic-moment screen on Pick Up that dissolves into FlowPilot; queue stat-card + dedicated owner analytics page for the hero metric (in two places, not one); 200ms slide-in + tab-title flash on real-time arrival, no sound, respects `prefers-reduced-motion`; unread dot clears on open/claim/dismiss, NOT on hover (Codex correction). Claim role gate moved in-scope per Codex (not deferred to TODO). +- Two TODOs added: peer-tech escalation (deferred to v2 once a pilot asks); mobile/responsive design (also v2; pre-PMF wedge demo targets desktop). Claim role gate's TODO entry was struck through in the same session because it shipped in `7a5b853`. +- Plan and test-plan artifacts copied into `docs/plans/` under the `YYYY-MM-DD-name-design.md` / `-test-plan.md` convention so they live alongside the existing project plans, not just in `~/.gstack/projects/`. +- Left for next session: frontend SSE subscription in `EscalationQueue.tsx` (fetch-based ReadableStream — native EventSource can't send auth headers; match `streamDocumentation` in `frontend/src/api/aiSessions.ts`), then the magic-moment handoff-context screen, then push + draft PR. Default Claude Code model is being switched from Opus 4.7 1M-context to Opus 4.7 (200k) for the next session — the resume docs are sized to be self-sufficient under the smaller window. +- Files touched (committed): `docs/plans/2026-04-27-escalation-mode-wedge-design.md`, `docs/plans/2026-04-27-escalation-mode-wedge-test-plan.md`, `backend/app/api/endpoints/flowpilot_analytics.py`, `backend/app/schemas/flowpilot_analytics.py`, `backend/app/api/endpoints/session_handoffs.py`, `backend/app/services/handoff_manager.py`, `backend/app/core/escalation_bus.py` (new), `backend/tests/test_flowpilot_analytics_escalations.py` (new), `backend/tests/test_escalation_bus.py` (new), `backend/tests/test_handoff_manager.py`, `backend/tests/test_session_handoffs_api.py`, `frontend/src/types/flowpilot-analytics.ts`, `frontend/src/api/flowpilotAnalytics.ts`, `frontend/src/components/flowpilot/EscalationMetricCard.tsx` (new), `frontend/src/components/flowpilot/index.ts`, `frontend/src/pages/EscalationQueuePage.tsx`, `.ai/CURRENT_TASK.md`, `.ai/HANDOFF.md`, `.ai/TODO.md`. + ## 2026-04-27 19:50 EDT — Codex — Stabilize Escalation Mode SSE backend tests - Diagnosed slow backend tests on `feat/escalation-metric-endpoint`. Multiple stale pytest processes were still alive inside `resolutionflow_backend` and held `resolutionflow_test` transactions open, blocking later per-test schema resets on `DROP SCHEMA public CASCADE`. -- 2.49.1 From b8627f41803273a00b9771634f704718a4a6a8d7 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 20:57:15 -0400 Subject: [PATCH 13/34] feat(escalations): subscribe EscalationQueue to live SSE arrivals MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the frontend live-arrival slice on top of the test-stabilized SSE backend. Senior techs now see a junior's escalation slide into the queue without refresh. - streamEscalations(handlers, signal) in aiSessions.ts: fetch-based ReadableStream parser (native EventSource cannot send auth headers). Handles SSE frames, partial frames across chunks, : keepalive heartbeats. Dispatches ready and handoff_created. - HandoffCreatedEvent + EscalationStreamHandlers types mirror the bus payload published by HandoffManager.dispatch_escalation_notifications. - EscalationQueue.tsx: AbortController-managed subscription with exponential-backoff reconnect (1s → 30s cap, attempt counter resets on ready). On handoff_created, refetch and diff against previous IDs via sessionsRef; new arrivals prepended (newest-first) above established cards (oldest-first preserved). Slide-in tag held for 800ms so the locked 200ms animation completes. Tab-title flash prefixes (N) while document.hidden, restores on focus / unmount. prefers-reduced-motion swaps slide-in for fade-in. ARIA region + aria-live=polite + aria-label on heading. Pick Up bumped to py-2.5 to clear the 44px touch floor. Verified end-to-end against the running dev stack: subscriber received the ready frame on connect; after posting a handoff via the API, the subscriber received the handoff_created frame with the expected payload — wire format matches the parser. Backend regression: focused subset still 32 passed in 18.91s. Frontend tsc -b clean. Co-Authored-By: Claude Opus 4.7 --- frontend/src/api/aiSessions.ts | 69 +++++ .../components/flowpilot/EscalationQueue.tsx | 270 ++++++++++++++---- frontend/src/types/ai-session.ts | 20 ++ 3 files changed, 303 insertions(+), 56 deletions(-) diff --git a/frontend/src/api/aiSessions.ts b/frontend/src/api/aiSessions.ts index 59d82e24..90531a8d 100644 --- a/frontend/src/api/aiSessions.ts +++ b/frontend/src/api/aiSessions.ts @@ -18,6 +18,8 @@ import type { ChatSessionCreateResponse, ChatMessageRequest, ChatMessageResponse, + HandoffCreatedEvent, + EscalationStreamHandlers, } from '@/types/ai-session' export const aiSessionsApi = { @@ -220,6 +222,73 @@ export const aiSessionsApi = { return response.data }, + // Native EventSource cannot send Authorization headers, so we use fetch + + // ReadableStream and parse SSE frames manually (same pattern as + // `streamDocumentation`). The returned promise resolves on clean stream + // close (server hangs up) and rejects on network/HTTP error so the caller + // can decide whether to reconnect with backoff. + async streamEscalations( + handlers: EscalationStreamHandlers, + signal: AbortSignal, + ): Promise { + const token = localStorage.getItem('access_token') + const baseUrl = import.meta.env.VITE_API_URL || '' + + const response = await fetch( + `${baseUrl}/api/v1/ai-sessions/escalations/stream`, + { + headers: { Authorization: `Bearer ${token}` }, + signal, + }, + ) + + if (!response.ok) { + throw new Error(`Escalation stream failed: HTTP ${response.status}`) + } + + const reader = response.body?.getReader() + if (!reader) { + throw new Error('Escalation stream: no response body') + } + + const decoder = new TextDecoder() + let buffer = '' + + while (true) { + const { done, value } = await reader.read() + if (done) return + + buffer += decoder.decode(value, { stream: true }) + + // SSE frames are separated by blank lines. Hold the trailing partial + // frame in the buffer until the next chunk completes it. + const frames = buffer.split('\n\n') + buffer = frames.pop() ?? '' + + for (const frame of frames) { + if (!frame) continue + let eventType = 'message' + let data = '' + for (const line of frame.split('\n')) { + if (line.startsWith(':')) continue // comment / keepalive + if (line.startsWith('event: ')) eventType = line.slice(7).trim() + else if (line.startsWith('data: ')) data += line.slice(6) + } + if (!data) continue + try { + const parsed = JSON.parse(data) as Record + if (eventType === 'handoff_created' && parsed.type === 'handoff_created') { + handlers.onHandoffCreated?.(parsed as unknown as HandoffCreatedEvent) + } else if (eventType === 'ready') { + handlers.onReady?.() + } + } catch { + // skip malformed frame + } + } + } + }, + async search(q: string, limit: number = 5): Promise { const response = await apiClient.get('/ai-sessions/search', { params: { q, limit }, diff --git a/frontend/src/components/flowpilot/EscalationQueue.tsx b/frontend/src/components/flowpilot/EscalationQueue.tsx index 20e865f1..dbce00aa 100644 --- a/frontend/src/components/flowpilot/EscalationQueue.tsx +++ b/frontend/src/components/flowpilot/EscalationQueue.tsx @@ -1,15 +1,31 @@ -import { useState, useEffect } from 'react' +import { useCallback, useEffect, useMemo, useRef, useState } from 'react' import { useNavigate } from 'react-router-dom' import { AlertTriangle, Clock, Hash, Ticket, Loader2, RefreshCw } from 'lucide-react' import { aiSessionsApi } from '@/api' import type { AISessionSummary } from '@/types/ai-session' import { timeAgo } from '@/lib/timeAgo' +import { cn } from '@/lib/utils' interface EscalationQueueProps { onPickup?: (sessionId: string) => void onCountChange?: (count: number) => void } +// Static list sort: oldest-first. Longest waiting = most urgent. +const sortOldestFirst = (a: AISessionSummary, b: AISessionSummary) => + new Date(a.created_at).getTime() - new Date(b.created_at).getTime() + +// Live-arrival bucket sort: newest-first so the most recent escalation is at +// the very top of the list. +const sortNewestFirst = (a: AISessionSummary, b: AISessionSummary) => + new Date(b.created_at).getTime() - new Date(a.created_at).getTime() + +// How long a freshly-arrived card keeps the slide-in animation class. The +// keyframe itself runs 200ms; this just keeps the class on the DOM long +// enough for the animation to finish before React removes it on the next +// state transition. +const NEW_CARD_HIGHLIGHT_MS = 800 + function waitTimeColor(createdAt: string): string { const hours = (Date.now() - new Date(createdAt).getTime()) / 3_600_000 if (hours >= 4) return '#f87171' // danger @@ -22,29 +38,156 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp const [sessions, setSessions] = useState([]) const [isLoading, setIsLoading] = useState(true) const [error, setError] = useState(null) + // Session IDs that arrived via SSE and should still play the slide-in. + const [newIds, setNewIds] = useState>(new Set()) + // Track count of unseen arrivals while the tab is backgrounded. + const [unseenCount, setUnseenCount] = useState(0) - const loadQueue = async () => { + // Ref mirrors the latest sessions so the SSE handler can diff without + // re-binding on every state change. + const sessionsRef = useRef([]) + useEffect(() => { + sessionsRef.current = sessions + }, [sessions]) + + const prefersReducedMotion = useMemo(() => { + if (typeof window === 'undefined' || !window.matchMedia) return false + return window.matchMedia('(prefers-reduced-motion: reduce)').matches + }, []) + + // ── Tab title flash ── + // Capture the original title once at mount. While unseen > 0, prefix it. + const originalTitleRef = useRef('') + useEffect(() => { + originalTitleRef.current = document.title + return () => { + // Restore on unmount so a leftover "(N) ..." prefix doesn't bleed + // into the next page. + document.title = originalTitleRef.current + } + }, []) + + useEffect(() => { + const base = originalTitleRef.current || document.title + document.title = unseenCount > 0 ? `(${unseenCount}) ${base}` : base + }, [unseenCount]) + + useEffect(() => { + const clearUnseen = () => { + if (!document.hidden) setUnseenCount(0) + } + const onFocus = () => setUnseenCount(0) + document.addEventListener('visibilitychange', clearUnseen) + window.addEventListener('focus', onFocus) + return () => { + document.removeEventListener('visibilitychange', clearUnseen) + window.removeEventListener('focus', onFocus) + } + }, []) + + const loadQueue = useCallback(async () => { setIsLoading(true) setError(null) try { const data = await aiSessionsApi.getEscalationQueue() - // Sort oldest-first — longest waiting = most urgent - const sorted = [...data].sort( - (a, b) => new Date(a.created_at).getTime() - new Date(b.created_at).getTime() - ) + const sorted = [...data].sort(sortOldestFirst) setSessions(sorted) + setNewIds(new Set()) onCountChange?.(sorted.length) } catch { setError('Failed to load escalation queue') } finally { setIsLoading(false) } - } + }, [onCountChange]) useEffect(() => { loadQueue() - // eslint-disable-next-line react-hooks/exhaustive-deps -- load once on mount - }, []) + }, [loadQueue]) + + // ── SSE subscription ── + // Refetch the queue on each `handoff_created` event (the event payload is + // intentionally thin — it's a trigger, not the full card data). Diff + // against the previous list to identify newly-arrived sessions; prepend + // them at the top with the slide-in animation, then keep the rest of the + // queue in oldest-first order below. + const handleHandoffCreated = useCallback(async () => { + let fresh: AISessionSummary[] + try { + fresh = await aiSessionsApi.getEscalationQueue() + } catch { + return + } + + const prevIds = new Set(sessionsRef.current.map((s) => s.id)) + const arrived = fresh.filter((s) => !prevIds.has(s.id)).sort(sortNewestFirst) + const established = fresh.filter((s) => prevIds.has(s.id)).sort(sortOldestFirst) + const next = [...arrived, ...established] + setSessions(next) + onCountChange?.(next.length) + + if (arrived.length === 0) return + + const arrivedIds = arrived.map((s) => s.id) + setNewIds((prev) => { + const merged = new Set(prev) + arrivedIds.forEach((id) => merged.add(id)) + return merged + }) + if (document.hidden) { + setUnseenCount((c) => c + arrived.length) + } + window.setTimeout(() => { + setNewIds((prev) => { + const remaining = new Set(prev) + arrivedIds.forEach((id) => remaining.delete(id)) + return remaining + }) + }, NEW_CARD_HIGHLIGHT_MS) + }, [onCountChange]) + + useEffect(() => { + const abort = new AbortController() + let reconnectTimer: number | null = null + let attempt = 0 + let cancelled = false + + const connect = async () => { + if (cancelled) return + try { + await aiSessionsApi.streamEscalations( + { + onReady: () => { + attempt = 0 + }, + onHandoffCreated: () => { + void handleHandoffCreated() + }, + }, + abort.signal, + ) + // Stream ended cleanly (server hung up). Reconnect quickly. + if (!cancelled) { + reconnectTimer = window.setTimeout(connect, 1000) + } + } catch (err) { + if (cancelled || abort.signal.aborted) return + if (err instanceof DOMException && err.name === 'AbortError') return + // Exponential backoff: 1s, 2s, 4s, 8s, 16s, capped at 30s. + const delay = Math.min(30_000, 1000 * 2 ** attempt) + attempt += 1 + reconnectTimer = window.setTimeout(connect, delay) + } + } + + void connect() + + return () => { + cancelled = true + abort.abort() + if (reconnectTimer !== null) window.clearTimeout(reconnectTimer) + } + }, [handleHandoffCreated]) const handlePickup = (sessionId: string) => { if (onPickup) { @@ -95,7 +238,10 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp return (
-

+

Awaiting pickup ({sessions.length})

- {sessions.map((session) => ( -
-
-

- {session.problem_summary || 'Untitled session'} -

- {session.escalation_reason && ( -

- Reason: {session.escalation_reason} -

- )} -
- -
- {session.problem_domain && ( - - {session.problem_domain} - - )} - - - {session.step_count} steps - - + {sessions.map((session) => { + const isNew = newIds.has(session.id) + return ( +
- - {timeAgo(session.created_at)} - - {session.psa_ticket_id && ( - - - #{session.psa_ticket_id} - - )} -
+
+

+ {session.problem_summary || 'Untitled session'} +

+ {session.escalation_reason && ( +

+ Reason: {session.escalation_reason} +

+ )} +
-
- -
-
- ))} +
+ {session.problem_domain && ( + + {session.problem_domain} + + )} + + + {session.step_count} steps + + + + {timeAgo(session.created_at)} + + {session.psa_ticket_id && ( + + + #{session.psa_ticket_id} + + )} +
+ +
+ +
+
+ ) + })} +
) } diff --git a/frontend/src/types/ai-session.ts b/frontend/src/types/ai-session.ts index c8f90886..281ef543 100644 --- a/frontend/src/types/ai-session.ts +++ b/frontend/src/types/ai-session.ts @@ -258,3 +258,23 @@ export interface SimilarSession { created_at: string | null similarity: number } + +// ── Escalation SSE bus ── +// +// Mirrors the `event_generator` payload in +// backend/app/api/endpoints/session_handoffs.py — keep this in sync with the +// dict published by `HandoffManager.dispatch_escalation_notifications`. + +export interface HandoffCreatedEvent { + type: 'handoff_created' + handoff_id: string + session_id: string + priority: string + engineer_notes: string + created_at: string | null +} + +export interface EscalationStreamHandlers { + onReady?: () => void + onHandoffCreated?: (event: HandoffCreatedEvent) => void +} -- 2.49.1 From f65b65790cfecc7a7b6cac8891f2862a3be06c1a Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 20:57:20 -0400 Subject: [PATCH 14/34] docs(ai): handoff state after frontend SSE slice lands Marks the SSE subscription as shipped, points the next-session resume target at the magic-moment handoff-context screen, and logs the live end-to-end verification. Co-Authored-By: Claude Opus 4.7 --- .ai/CURRENT_TASK.md | 12 ++++++------ .ai/HANDOFF.md | 43 +++++++++++++++++++++---------------------- .ai/SESSION_LOG.md | 11 +++++++++++ 3 files changed, 38 insertions(+), 28 deletions(-) diff --git a/.ai/CURRENT_TASK.md b/.ai/CURRENT_TASK.md index 6dd60035..c38bae1a 100644 --- a/.ai/CURRENT_TASK.md +++ b/.ai/CURRENT_TASK.md @@ -2,7 +2,7 @@ **Task:** Build **Escalation Mode** — the wedge for ResolutionFlow's GTM (first paying-customer push). When a junior tech escalates a FlowPilot session, the senior tech sees structured handoff context in seconds instead of running a 5-minute verbal "tell me what you tried" call. -**Status:** in-flight on `feat/escalation-metric-endpoint`. Backend is **feature-complete and test-stabilized**. **Next:** frontend SSE subscription in `EscalationQueue.tsx`, then the magic-moment handoff-context screen, then push + draft PR. +**Status:** in-flight on `feat/escalation-metric-endpoint`. Backend is **feature-complete and test-stabilized**. **Frontend SSE subscription is shipped** (`EscalationQueue.tsx` now subscribes via fetch-based ReadableStream, prepends new arrivals with the locked 200ms slide-in, flashes tab title when backgrounded, respects `prefers-reduced-motion`, exponential-backoff reconnect). **Next:** magic-moment handoff-context screen, then push + draft PR. **Plan:** [`docs/plans/2026-04-27-escalation-mode-wedge-design.md`](../docs/plans/2026-04-27-escalation-mode-wedge-design.md). Reviewed by `/office-hours`, `/plan-eng-review`, `/plan-design-review`, `/codex review`. Eng + Design CLEARED. Codex's two-metric correction + claim role gate + per-channel notification model + SSE bus diagnostics all applied. @@ -22,15 +22,15 @@ | `bc15952` | Codex: stabilize SSE backend tests — `Depends(..., scope="function")` releases auth DB deps before the long-lived stream body; SSE handshake test calls the generator directly; AI-assessment stub fixture; bus normalizes string vs UUID account_id | | `fff8338` | Doc-only: track escalation assessment latency follow-up | | `9bdd995` | Bound escalation assessment latency to `ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS` (default 5s); handoff still creates if assessment times out | +| _pending_ | Frontend SSE subscription in `EscalationQueue.tsx` — fetch-based `ReadableStream` reader, `handoff_created` triggers refetch + prepend with locked 200ms slide-in, exponential-backoff reconnect, tab-title flash when backgrounded, `prefers-reduced-motion` honored, ARIA live-region | -**Test status:** focused subset (`test_escalation_bus`, `test_handoff_manager`, `test_session_handoffs_api`, `test_flowpilot_analytics_escalations`) → `32 passed in 17.77s` with `-n auto`. Frontend `tsc -b` clean. Branch not pushed. +**Test status:** focused subset (`test_escalation_bus`, `test_handoff_manager`, `test_session_handoffs_api`, `test_flowpilot_analytics_escalations`) → `32 passed in 18.91s` with `-n auto`. Frontend `tsc -b` clean. Live-arrival smoke test against the running dev stack confirmed the SSE handshake delivers the `ready` frame on connect and a `handoff_created` frame with the expected payload after posting a handoff. Branch not pushed. ## Remaining work on this branch -1. **Frontend SSE subscription** in `EscalationQueue.tsx`. Use a fetch-based `ReadableStream` reader (matching [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts) `streamDocumentation` — native `EventSource` can't send auth headers). Prepend new cards with the locked 200ms slide-in. Reconnect with backoff. Tab-title flash when backgrounded. Respect `prefers-reduced-motion`. -2. **Magic-moment handoff-context screen** — 4-section view (problem summary / what's been tried / AI assessment / Start here CTA) that loads on Pick Up before dissolving into the regular FlowPilot session view. ~1.5-2 days. Must render gracefully when `ai_assessment` is `None` (assessment timed out — see `9bdd995`). -3. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. Optional for v1 demo. -4. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. +1. **Magic-moment handoff-context screen** — 4-section view (problem summary / what's been tried / AI assessment / Start here CTA) that loads on Pick Up before dissolving into the regular FlowPilot session view. ~1.5-2 days. Must render gracefully when `ai_assessment` is `None` (assessment timed out — see `9bdd995`). +2. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. Optional for v1 demo. +3. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. ## Two-metric framing — read this before quoting numbers to anyone diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index e7654915..2ab995de 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -2,47 +2,45 @@ # HANDOFF.md -**Last updated:** 2026-04-27 EDT +**Last updated:** 2026-04-27 21:00 EDT **Active task:** **Escalation Mode** wedge build. See [`CURRENT_TASK.md`](CURRENT_TASK.md) for the full status; this file holds the resume point only. -**Branch:** `feat/escalation-metric-endpoint` — SSE backend WIP is now test-stabilized locally. Working tree should be clean after the handoff commit. +**Branch:** `feat/escalation-metric-endpoint` — frontend SSE live-arrival slice is shipped on top of the test-stabilized backend. ## Status -Previous session diagnosed the slow-test issue and fixed the backend test loop. +Previous session shipped the frontend SSE subscription that the next session was set up to do. -Root causes: -- Multiple stale pytest processes were still alive inside `resolutionflow_backend`, despite the prior handoff saying they were dead. They held `resolutionflow_test` transactions open and caused later tests to block on `DROP SCHEMA public CASCADE`. -- `test_escalations_stream_returns_sse_content_type` used HTTPX `ASGITransport` against an infinite SSE stream. That transport buffers the entire response body before returning, so the test waited forever and held the auth DB dependency transaction open. -- Escalation handoff tests created `intent="escalate"` handoffs without stubbing `_generate_ai_assessment()`, so they waited on the real AI path instead of testing handoff behavior. -- The bus keyed subscribers by raw `account_id`; string UUIDs and `UUID` objects for the same account did not match. +What landed: -Fixes made: -- `stream_escalations` now uses `Depends(require_engineer_or_admin, scope="function")` so auth DB dependencies are released before the long-lived stream body. -- The SSE handshake test now calls `stream_escalations()` directly and consumes only the first generator yield, avoiding HTTPX's infinite-stream buffering behavior. -- Handoff manager/API tests stub `_generate_ai_assessment()` with an `AsyncMock`. -- `EscalationBus` normalizes string/UUID account IDs at subscribe/publish/unsubscribe/subscriber_count boundaries, with a regression test. -- Follow-up fix: escalation AI assessment is now latency-bounded by `ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS` (default 5s). If it times out, handoff creation proceeds with no assessment instead of blocking on the model/network path. +- `frontend/src/api/aiSessions.ts` — added `streamEscalations(handlers, signal)`. Fetch-based `ReadableStream` parser (native `EventSource` can't send auth headers). Handles SSE frames including `: keepalive` heartbeats. Dispatches `ready` and `handoff_created` events. +- `frontend/src/types/ai-session.ts` — added `HandoffCreatedEvent` and `EscalationStreamHandlers` types mirroring the backend bus payload. +- `frontend/src/components/flowpilot/EscalationQueue.tsx` — full data-layer rewrite. SSE subscription with `AbortController`, exponential-backoff reconnect (1s → 30s cap, attempt counter resets on `ready`). On `handoff_created` the component refetches the queue, diffs against the previous IDs via a `sessionsRef`, prepends new arrivals (newest-first) above established cards (oldest-first preserved). New IDs tagged for 800ms so the locked 200ms slide-in animation plays before cleanup. Tab-title flash captures `document.title` at mount, prefixes `(N)` while `document.hidden`, clears on `focus` / `visibilitychange`, restores on unmount. `prefers-reduced-motion: reduce` swaps `animate-slide-in-bottom` for `animate-fade-in`. ARIA: `role="region"` + `aria-live="polite"` on the list, `aria-label="N escalations awaiting pickup"` on the heading. Pick Up button bumped to `py-2.5` to clear the 44px touch floor. Verified: -- `pytest tests/test_escalation_bus.py tests/test_handoff_manager.py tests/test_session_handoffs_api.py tests/test_flowpilot_analytics_escalations.py --override-ini=addopts= -q --durations=20` → `31 passed in 46.95s` -- Same subset with `-n auto` → `31 passed in 17.80s` -- After the assessment-timeout fix: same subset with `-n auto` → `32 passed in 17.77s` -- No remaining pytest processes or `resolutionflow%test%` Postgres sessions after the run. + +- Frontend `tsc -b` exit 0. Vite HMR'd the new file with no compile errors. +- Backend regression: focused subset (`test_escalation_bus`, `test_handoff_manager`, `test_session_handoffs_api`, `test_flowpilot_analytics_escalations`) → `32 passed in 18.91s` with `-n auto`. +- Live SSE handshake against the running dev stack returns 200 with `text/event-stream; charset=utf-8` and the locked headers (`cache-control: no-cache`, `x-accel-buffering: no`). Subscriber received the `ready` frame on connect; after posting a handoff via the API, the subscriber received the `handoff_created` frame with the full payload — wire format matches the new parser exactly. + +Not yet verified (would need a real browser session): the slide-in animation visually plays, the tab title actually updates, the reduced-motion media-query path, AbortController cancellation on unmount, backoff after a real network blip. Wire contract is confirmed; these are visual/timing-dependent and follow from correct parser + state machine. + +Smoke-test artifact: a single test handoff (`0f6149db…` on session `50ea20d4…`) is sitting in the engineer's queue from the verification step. Harmless; useful as visual demo data. ## Resume point -1. Continue the **Frontend SSE subscription** in `EscalationQueue.tsx`: fetch-based reader, prepend new cards with the locked 200ms slide-in, reconnect with backoff, tab-title flash when backgrounded, respect `prefers-reduced-motion`. -2. Then ship the **magic-moment handoff-context screen**: 4 sections (problem summary / what's been tried / AI assessment / Start here CTA), loads on Pick Up, then dissolves into regular FlowPilot session view. -3. Push the branch and open a draft PR when the frontend/live-arrival slice is ready. +1. Build the **magic-moment handoff-context screen**: 4 sections (problem summary / what's been tried / AI assessment / Start here CTA), loads on Pick Up, then dissolves into the regular FlowPilot session view. Must render gracefully when `ai_assessment` is `None` (assessment timed out — see commit `9bdd995`). Surface `ai_assessment_data.suggested_steps[]` as chips below the chat input that prefill it on click — do NOT invent a "jump to most-likely-next-step" capability that doesn't exist in the session model. +2. Push the branch and open a draft PR once the magic-moment screen is in. +3. Optional v1: owner-facing `/analytics/escalations` page (period selector + conversion rate + trend chart). ## Useful breadcrumbs - SSE endpoint: [`backend/app/api/endpoints/session_handoffs.py`](../backend/app/api/endpoints/session_handoffs.py) — `stream_escalations`. - Pub/sub bus: [`backend/app/core/escalation_bus.py`](../backend/app/core/escalation_bus.py). In-memory, account-scoped, non-durable, 64-event per-subscriber queue, drop-on-full. +- Frontend SSE consumer: [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts) → `streamEscalations`. +- Live-arrival queue UI: [`frontend/src/components/flowpilot/EscalationQueue.tsx`](../frontend/src/components/flowpilot/EscalationQueue.tsx). - Notification dispatch: [`backend/app/services/handoff_manager.py`](../backend/app/services/handoff_manager.py) — `dispatch_escalation_notifications`, called after `db.commit()` in the handoff endpoint. -- Frontend streaming reference: [`frontend/src/api/aiSessions.ts`](../frontend/src/api/aiSessions.ts) — `streamDocumentation` uses fetch + `ReadableStream`, which remains the right pattern because native `EventSource` cannot send auth headers. - Metric endpoint: [`backend/app/api/endpoints/flowpilot_analytics.py`](../backend/app/api/endpoints/flowpilot_analytics.py) — `get_escalation_metrics`. ## Watch-outs @@ -51,3 +49,4 @@ Verified: - `DROP SCHEMA public CASCADE` per test is still the dominant cost: DB-backed tests spend ~1.7-2.8s in setup. Use `-n auto` for focused backend loops. - The bus is acceptable for v1 pilot scale only because Railway is single-replica. Redis pub/sub is the obvious swap when horizontal scaling appears. - Escalation assessment can be missing when the 5s timeout fires. The handoff-context UI must render a graceful "assessment unavailable/in progress" state rather than treating it as required. +- `streamEscalations` doesn't drive token refresh on a mid-stream 401 — the Axios interceptor only covers axios calls. Acceptable for v1 (queue page lifetime ≤ access-token lifetime in practice); revisit if pilots leave the page open for hours. diff --git a/.ai/SESSION_LOG.md b/.ai/SESSION_LOG.md index aff0f49b..483138a0 100644 --- a/.ai/SESSION_LOG.md +++ b/.ai/SESSION_LOG.md @@ -12,6 +12,17 @@ --- +## 2026-04-27 21:00 EDT — Claude Code — Escalation Mode: frontend SSE subscription in EscalationQueue + +- Picked up `feat/escalation-metric-endpoint` after the Codex test-stabilization pass. Confirmed green starting state: focused backend subset `32 passed in 18.78s` with `-n auto`. +- Implemented the live-arrival frontend slice. Added `streamEscalations(handlers, signal)` to `frontend/src/api/aiSessions.ts` — fetch-based `ReadableStream` reader (native `EventSource` can't send auth headers) that parses SSE frames (event/data/comment lines), buffers partial frames across chunks, ignores `: keepalive` heartbeats, dispatches `ready` and `handoff_created` events. Added `HandoffCreatedEvent` and `EscalationStreamHandlers` types in `frontend/src/types/ai-session.ts` mirroring the backend bus payload. +- Rewrote `frontend/src/components/flowpilot/EscalationQueue.tsx`. SSE subscription with `AbortController` + exponential-backoff reconnect (1s → 30s cap, attempt counter resets on `ready`). On `handoff_created` the component refetches the queue, diffs against the previous IDs via a `sessionsRef`, prepends new arrivals (newest-first) above established cards (oldest-first preserved). New IDs are tagged for 800ms so the locked 200ms slide-in animation plays before cleanup. Tab-title flash: captures `document.title` at mount, prefixes `(N)` while `document.hidden`, clears on `focus` / `visibilitychange`, restores on unmount. `prefers-reduced-motion: reduce` swaps `animate-slide-in-bottom` for `animate-fade-in`. ARIA: `role="region"` + `aria-live="polite"` on the list, `aria-label="N escalations awaiting pickup"` on the heading; Pick Up button bumped to `py-2.5` to clear the 44px touch floor. +- Verified end-to-end against the running dev stack. `tsc -b` exit 0. Vite HMR'd the new component without errors. Raw SSE handshake against `/api/v1/ai-sessions/escalations/stream` returned 200 with `text/event-stream; charset=utf-8` plus the locked headers (`cache-control: no-cache`, `x-accel-buffering: no`). Subscriber received the `ready` frame on connect; after posting a handoff via the API, the subscriber received the `handoff_created` frame with the full payload — wire format matches the parser exactly. Backend regression: same focused subset still `32 passed in 18.91s`. +- Not yet verified (would need a real browser session): the slide-in animation visually plays, the tab title actually updates, the reduced-motion media-query path, AbortController cancellation on unmount, backoff after a real network blip. Wire contract is confirmed; these are visual/timing-dependent and follow from correct parser + state machine. +- Smoke-test artifact: a single test handoff (`0f6149db…` on session `50ea20d4…`) is sitting in the engineer's queue from the verification step. Harmless; useful as visual demo data. +- Left for next session: the magic-moment handoff-context screen — 4 sections (problem summary / what's been tried / AI assessment / Start here CTA), loads on Pick Up, dissolves into the regular FlowPilot session view. Must render gracefully when `ai_assessment` is `None` (per the 5s assessment timeout from Codex's earlier fix). +- Files touched: `frontend/src/api/aiSessions.ts`, `frontend/src/types/ai-session.ts`, `frontend/src/components/flowpilot/EscalationQueue.tsx`, `.ai/CURRENT_TASK.md`, `.ai/HANDOFF.md`, `.ai/SESSION_LOG.md`. + ## 2026-04-27 EDT — Claude Code — Escalation Mode wedge: design through SSE backend (8 commits) - One long session that produced the entire planning artifact stack and most of the backend for the Escalation Mode wedge. Output of `/office-hours` (8 founder-signal session, top-tier YC archetype indicators), `/plan-eng-review` (scope reduced from "2-3 weeks greenfield" to "~6-9 days integration + metric + polish" once the existing handoff_manager surface was inventoried), `/plan-design-review` (6/10 → 9/10 with magic-moment screen, hero metric placement, and real-time arrival visual locked), and `/codex review` (12 findings, 6 applied — two-metric framing, notification routing, claim auth gate moved in-scope, unread-state fix, "Start here" CTA reframe, per-channel delivery model; 5 rejected including the full-scope reduction Codex pushed for). -- 2.49.1 From 8e9d22e0e0095b2e78d1dbaf39ebaf9b66ba18e6 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Mon, 27 Apr 2026 21:06:14 -0400 Subject: [PATCH 15/34] feat(escalations): magic-moment handoff-context screen on pickup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the dedicated 4-section handoff-context view that renders BEFORE the FlowPilot session for senior techs picking up an escalated session, then dissolves on "Start here". This is the wedge's demonstrable magic moment — what the GTM Loom records. - HandoffContextScreen.tsx: pure presentational, takes a HandoffResponse plus onStartHere / onDismiss callbacks. Sections: header (problem summary, domain, step count, escalated-time, priority badge), "What's been tried" (engineer notes + step-count affordance), "AI assessment" (likely_cause / suggested_steps / confidence badge), Start here CTA. Confidence badge accepts both numeric (0..1) and string ("low"/"medium"/"high") shapes — backend currently emits the latter. Renders an explicit "assessment unavailable" branch when ai_assessment_data is null (the 5s timeout from 9bdd995 fired). Honors prefers-reduced-motion (animate-fade-in vs animate-slide-up). ARIA dialog + focus on the primary CTA. Esc dismisses when used as a re-openable overlay; pre-claim, Start here is the only exit. - FlowPilotSessionPage.tsx: on /pilot/:id?pickup=true, fetch the handoff list via handoffsApi.listHandoffs (account-scoped via RLS, no claim required) and find the latest unclaimed escalate handoff. If found, render the magic-moment screen and skip the regular loadSession (the senior isn't yet escalated_to_id, so GET would 404). Start here calls claimHandoff, drops the pickup query param, dismisses the screen — the existing loadSession effect then fires because the senior is now escalated_to_id. A "Context" toolbar button on active sessions re-opens the screen as a dismissible overlay (visible only when the senior arrived via the magic-moment flow this session — handoff lookup on demand). Verified end-to-end against the running dev stack: listHandoffs returns the unclaimed handoff with full payload; claim flips session status from escalated → active; subsequent GET succeeds. tsc -b clean. Defers (TODO followups): suggested-step chips below the chat input that prefill on click (requires threading through to FlowPilotMessageBar); snapshot expansion to include the recent diagnostic steps pre-claim; toolbar Context button on sessions where the senior didn't arrive via magic-moment. Co-Authored-By: Claude Opus 4.7 --- .../flowpilot/HandoffContextScreen.tsx | 308 ++++++++++++++++++ frontend/src/components/flowpilot/index.ts | 1 + frontend/src/pages/FlowPilotSessionPage.tsx | 142 +++++++- 3 files changed, 447 insertions(+), 4 deletions(-) create mode 100644 frontend/src/components/flowpilot/HandoffContextScreen.tsx diff --git a/frontend/src/components/flowpilot/HandoffContextScreen.tsx b/frontend/src/components/flowpilot/HandoffContextScreen.tsx new file mode 100644 index 00000000..8c055cc7 --- /dev/null +++ b/frontend/src/components/flowpilot/HandoffContextScreen.tsx @@ -0,0 +1,308 @@ +import { useEffect, useMemo, useRef } from 'react' +import { + AlertTriangle, + ArrowRight, + Brain, + Clock, + FileText, + Hash, + Sparkles, + Target, + X, +} from 'lucide-react' +import type { HandoffResponse } from '@/types/branching' +import { cn } from '@/lib/utils' +import { timeAgo } from '@/lib/timeAgo' + +// Magic-moment handoff-context screen. Renders BEFORE the FlowPilot session +// view when a senior tech picks up an escalated session, then dissolves on +// "Start here". Re-openable via toolbar in FlowPilotSessionPage. +// +// Four sections per the design plan: +// 1. Problem summary (top, Bricolage h2) +// 2. What's been tried (left column) — engineer notes + step count. +// Full step detail isn't in the handoff snapshot today (snapshot = +// problem_summary, problem_domain, status, step_count, confidence_tier +// per HandoffManager._generate_snapshot); we surface what's there and +// promise the timeline post-pickup. Snapshot expansion is a follow-up. +// 3. AI assessment (right column) — likely_cause / suggested_steps / +// confidence. Renders gracefully when ai_assessment is null (the 5s +// timeout from commit 9bdd995 fired). +// 4. Start here (primary CTA, electric-blue, ≥44px) — claims the handoff +// and dissolves the screen. + +type ConfidenceTier = 'low' | 'medium' | 'high' | string + +interface HandoffContextScreenProps { + handoff: HandoffResponse + onStartHere: () => Promise | void + onDismiss?: () => void + // When true, renders an "X" close affordance in the corner. Used when the + // screen is re-opened from the FlowPilot toolbar (post-claim re-read). + dismissible?: boolean + isProcessing?: boolean +} + +function ConfidenceBadge({ value }: { value: number | string | null | undefined }) { + if (value === null || value === undefined || value === '') return null + // Numeric (0..1) or string tier + let tier: ConfidenceTier = 'medium' + let label = String(value) + if (typeof value === 'number') { + tier = value >= 0.7 ? 'high' : value >= 0.4 ? 'medium' : 'low' + label = `${Math.round(value * 100)}%` + } else { + const s = String(value).toLowerCase() + if (s === 'low' || s === 'medium' || s === 'high') tier = s + label = s.charAt(0).toUpperCase() + s.slice(1) + } + const tone = + tier === 'high' + ? 'bg-success-dim text-success border border-success/20' + : tier === 'low' + ? 'bg-warning-dim text-warning border border-warning/20' + : 'bg-accent-dim text-accent-text border border-accent/20' + return ( + + {label} + + ) +} + +export function HandoffContextScreen({ + handoff, + onStartHere, + onDismiss, + dismissible = false, + isProcessing = false, +}: HandoffContextScreenProps) { + const startBtnRef = useRef(null) + + const prefersReducedMotion = useMemo(() => { + if (typeof window === 'undefined' || !window.matchMedia) return false + return window.matchMedia('(prefers-reduced-motion: reduce)').matches + }, []) + + // Esc dismisses when the screen is re-opened post-claim (dismissible mode). + // Pre-claim, Esc has no escape hatch — they must Start here or back out via + // browser nav. + useEffect(() => { + if (!dismissible || !onDismiss) return + const onKey = (e: KeyboardEvent) => { + if (e.key === 'Escape') onDismiss() + } + window.addEventListener('keydown', onKey) + return () => window.removeEventListener('keydown', onKey) + }, [dismissible, onDismiss]) + + // Focus the primary CTA on mount so keyboard users can hit Enter. + useEffect(() => { + startBtnRef.current?.focus() + }, []) + + const snapshot = handoff.snapshot as Record + const problemSummary = + (snapshot.problem_summary as string | undefined) || 'Untitled session' + const problemDomain = snapshot.problem_domain as string | undefined + const stepCount = (snapshot.step_count as number | undefined) ?? 0 + const confidenceTier = snapshot.confidence_tier as string | undefined + + const assessment = handoff.ai_assessment_data + const likelyCause = assessment?.likely_cause + const suggestedSteps = assessment?.suggested_steps ?? [] + const assessmentConfidence = assessment?.confidence + const assessmentText = handoff.ai_assessment + + const enterClass = prefersReducedMotion ? 'animate-fade-in' : 'animate-slide-up' + + return ( +
+ {/* Header */} +
+ + + +
+

+ Escalation handoff +

+

+ {problemSummary} +

+
+ {problemDomain && ( + + {problemDomain} + + )} + + + {stepCount} {stepCount === 1 ? 'step' : 'steps'} + + {confidenceTier && ( + + Session confidence: {confidenceTier} + + )} + + + Escalated {timeAgo(handoff.created_at)} + + {handoff.priority === 'elevated' && ( + + Elevated + + )} +
+
+ {dismissible && onDismiss && ( + + )} +
+ + {/* Two-column body */} +
+ {/* What's been tried */} +
+
+ +

+ What's been tried +

+
+ {handoff.engineer_notes ? ( +
+

+ Why they escalated +

+

+ {handoff.engineer_notes} +

+
+ ) : ( +

+ No notes from the original engineer. +

+ )} +
+ {stepCount}{' '} + diagnostic {stepCount === 1 ? 'step' : 'steps'} on record. Full + timeline opens when you start the session. +
+
+ + {/* AI assessment */} +
+
+
+ +

+ AI assessment +

+
+ +
+ + {!assessmentText && !likelyCause && suggestedSteps.length === 0 ? ( +
+ + + Assessment unavailable — model didn't respond in time. Pick up + the session to investigate directly. + +
+ ) : ( + <> + {likelyCause && ( +
+

+ Likely cause +

+

{likelyCause}

+
+ )} + {assessmentText && !likelyCause && ( +

+ {assessmentText} +

+ )} + {suggestedSteps.length > 0 && ( +
+

+ Suggested next steps +

+
    + {suggestedSteps.map((step, i) => ( +
  • + + {step} +
    • + ))} +
+
+ )} + + )} +
+
+ + {/* Start here CTA */} + {!dismissible && ( +
+

+ Picking up assigns this session to you and reactivates it. +

+ +
+ )} +
+ ) +} diff --git a/frontend/src/components/flowpilot/index.ts b/frontend/src/components/flowpilot/index.ts index 0cdb9db0..5556008d 100644 --- a/frontend/src/components/flowpilot/index.ts +++ b/frontend/src/components/flowpilot/index.ts @@ -11,6 +11,7 @@ export { EscalateModal } from './EscalateModal' export { EscalationQueue } from './EscalationQueue' export { EscalationMetricCard } from './EscalationMetricCard' export { SessionBriefing } from './SessionBriefing' +export { HandoffContextScreen } from './HandoffContextScreen' export { ProposalCard } from './ProposalCard' export { ProposalDetail } from './ProposalDetail' export { InSessionScriptGenerator } from './InSessionScriptGenerator' diff --git a/frontend/src/pages/FlowPilotSessionPage.tsx b/frontend/src/pages/FlowPilotSessionPage.tsx index e1ecd22e..c4fdec3b 100644 --- a/frontend/src/pages/FlowPilotSessionPage.tsx +++ b/frontend/src/pages/FlowPilotSessionPage.tsx @@ -3,7 +3,7 @@ import { useParams, useSearchParams, useLocation, useBlocker, useNavigate } from import { Sparkles, Loader2, AlertTriangle, CheckCircle2, ArrowUpRight, FileText, MoreHorizontal, Pause, X } from 'lucide-react' import { useFlowPilotSession } from '@/hooks/useFlowPilotSession' import { useBranching } from '@/hooks/useBranching' -import { FlowPilotIntake, FlowPilotSession, SessionBriefing } from '@/components/flowpilot' +import { FlowPilotIntake, FlowPilotSession, SessionBriefing, HandoffContextScreen } from '@/components/flowpilot' import { EscalateModal } from '@/components/flowpilot/EscalateModal' import { StatusUpdateModal } from '@/components/flowpilot/StatusUpdateModal' import { HandoffModal } from '@/components/session/HandoffModal' @@ -11,6 +11,7 @@ import { handoffsApi } from '@/api/handoffs' import { aiSessionsApi } from '@/api' import { integrationsApi } from '@/api/integrations' import type { PSATicketInfo } from '@/types/integrations' +import type { HandoffResponse } from '@/types/branching' import { toast } from '@/lib/toast' export default function FlowPilotSessionPage() { @@ -76,12 +77,95 @@ export default function FlowPilotSessionPage() { const [pickingUp, setPickingUp] = useState(false) - // Load existing session if ID in URL + // ── Magic-moment handoff-context screen ── + // When the senior arrives via /pilot/:id?pickup=true, the regular session + // GET 404s pre-claim (the senior isn't yet escalated_to_id). So we fetch + // the handoff list first (account-scoped via RLS, no claim required), find + // the most recent unclaimed escalate handoff, and render the magic-moment + // screen. "Start here" claims the handoff, then loadSession fires. + const [magicState, setMagicState] = useState<'inactive' | 'loading' | 'visible' | 'dismissed'>( + isPickup ? 'loading' : 'inactive', + ) + const [magicHandoff, setMagicHandoff] = useState(null) + const [overlayHandoff, setOverlayHandoff] = useState(null) + const [overlayLoading, setOverlayLoading] = useState(false) + const [claiming, setClaiming] = useState(false) + useEffect(() => { - if (sessionId && !fp.session) { + if (!isPickup || !sessionId || magicState !== 'loading') return + let cancelled = false + ;(async () => { + try { + const handoffs = await handoffsApi.listHandoffs(sessionId) + if (cancelled) return + // Newest unclaimed escalate handoff. listHandoffs orders desc by + // created_at on the backend, so .find() picks the latest. + const target = handoffs.find((h) => h.intent === 'escalate' && !h.claimed_by) + if (target) { + setMagicHandoff(target) + setMagicState('visible') + } else { + setMagicState('dismissed') + } + } catch { + if (cancelled) return + // Fall through to the legacy SessionBriefing path on failure. + setMagicState('dismissed') + } + })() + return () => { + cancelled = true + } + }, [isPickup, sessionId, magicState]) + + // Load existing session if ID in URL. Skip while the magic-moment screen is + // up — we don't have access to the session detail until claim. + useEffect(() => { + if (sessionId && !fp.session && magicState !== 'loading' && magicState !== 'visible') { fp.loadSession(sessionId) } - }, [sessionId]) // eslint-disable-line react-hooks/exhaustive-deps + }, [sessionId, magicState]) // eslint-disable-line react-hooks/exhaustive-deps + + const handleStartHere = async () => { + if (!sessionId || !magicHandoff) return + setClaiming(true) + try { + await handoffsApi.claimHandoff(sessionId, magicHandoff.id) + // Drop the pickup query param and dismiss the screen — the loadSession + // effect above will fire because magicState is no longer 'visible'. + setSearchParams({}) + setMagicState('dismissed') + } catch (e: unknown) { + const message = e instanceof Error ? e.message : 'Failed to pick up session' + toast.error(message) + } finally { + setClaiming(false) + } + } + + const openHandoffContextOverlay = async () => { + if (!sessionId) return + // Reuse the in-memory copy when we already loaded the handoff during + // pickup, otherwise fetch on demand. + if (magicHandoff) { + setOverlayHandoff(magicHandoff) + return + } + setOverlayLoading(true) + try { + const handoffs = await handoffsApi.listHandoffs(sessionId) + const target = handoffs.find((h) => h.intent === 'escalate') + if (target) { + setOverlayHandoff(target) + } else { + toast.info('No handoff context available for this session.') + } + } catch { + toast.error('Could not load handoff context') + } finally { + setOverlayLoading(false) + } + } // Load branches when session is branching useEffect(() => { @@ -133,6 +217,28 @@ export default function FlowPilotSessionPage() { } } + // Magic-moment handoff-context screen — shown before the senior tech claims + // an escalated session. Takes priority over session loading because the + // senior can't load the session detail until claim succeeds. + if (magicState === 'loading') { + return ( +
+ +
+ ) + } + if (magicState === 'visible' && magicHandoff) { + return ( +
+ +
+ ) + } + // Error state if (fp.error && !fp.session) { return ( @@ -273,6 +379,17 @@ export default function FlowPilotSessionPage() { <> {/* Desktop actions */}
+ {magicHandoff && ( + + )} + )} {activePsaTicketId && (
) -- 2.49.1 From aca915b047c91ddabe7d340f0f777a6f8ad7b6dd Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Tue, 28 Apr 2026 00:04:08 -0400 Subject: [PATCH 22/34] fix(escalations): bump assessment timeout, surface picked-up sessions in sidebar MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two field-reported issues from live wedge testing. ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS bumped 5s → 15s. The 5s bound fired too aggressively against the Sonnet diagnostic assessment prompt; ~4-8s is typical but tail latency hits 12-14s. The fallback "Assessment unavailable — model didn't respond in time" placeholder was showing on the magic-moment screen for two consecutive escalations, which kills the demo. 15s keeps the click-path bounded but lets the typical case return real content. Real fix is async generation (kick off, persist when done, surface "still computing" with refresh) — captured as a follow-up; bumping the bound is the right call for the wedge demo. list_sessions now matches escalated_to_id == current_user.id alongside the existing user_id and escalation_package.picked_up_by clauses. The unified HandoffManager.claim_session sets escalated_to_id but doesn't write the legacy picked_up_by JSONB key, so picked-up sessions never showed in the senior's chat list — the senior would land on the session detail (active chat) but the sidebar showed only their other unrelated sessions. User reported this as "4 different versions of the session in the chat history section" — they were actually 4 unrelated empty sessions the senior owned, plus the picked-up session was just invisible. Backend tests still 94/94. Co-Authored-By: Claude Opus 4.7 --- backend/app/api/endpoints/ai_sessions.py | 14 +++++++++++++- backend/app/core/config.py | 9 ++++++++- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/backend/app/api/endpoints/ai_sessions.py b/backend/app/api/endpoints/ai_sessions.py index 31a1ec5e..f8ceb9fd 100644 --- a/backend/app/api/endpoints/ai_sessions.py +++ b/backend/app/api/endpoints/ai_sessions.py @@ -873,13 +873,25 @@ async def list_sessions( date_to: Optional[datetime] = Query(None), q: Optional[str] = Query(None, min_length=2, max_length=200), ): - """List the current user's AI sessions (owned or picked up).""" + """List the current user's AI sessions (owned or picked up). + + "Picked up" includes both the legacy escalation_package.picked_up_by + marker (set by flowpilot_engine.pickup_session) AND the new + escalated_to_id field (set by HandoffManager.claim_session for the + unified handoff/escalate path). Without the escalated_to_id branch + the senior tech wouldn't see a session they just claimed in their + chat sidebar — the picked-up session lands as the active chat with + no entry in the list, which is what the user reported as "4 versions + of the session" (their unrelated owned sessions show up while the + claimed one is invisible). + """ user_id_str = str(current_user.id) query = ( select(AISession) .where( or_( AISession.user_id == current_user.id, + AISession.escalated_to_id == current_user.id, AISession.escalation_package["picked_up_by"].as_string() == user_id_str, ) ) diff --git a/backend/app/core/config.py b/backend/app/core/config.py index 985bca98..b3135131 100644 --- a/backend/app/core/config.py +++ b/backend/app/core/config.py @@ -111,7 +111,14 @@ class Settings(BaseSettings): GOOGLE_AI_API_KEY: Optional[str] = None AI_MODEL_GEMINI: str = "gemini-2.5-flash" AI_MODEL_ANTHROPIC: str = "claude-sonnet-4-6" - ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS: int = 5 + # 15s is generous for the click-path; Claude usually returns a 500-token + # diagnostic in 4-8s but tail latency on the assessment prompt has hit + # 12-14s in the field. Going below this leaves too many escalations with + # the "Assessment unavailable — model didn't respond in time" placeholder + # the senior sees on the magic-moment screen. Real fix is async generation + # (kick off, persist when done, surface "still computing" with refresh) — + # that's a follow-up; bumping the bound keeps the wedge demo coherent. + ESCALATION_AI_ASSESSMENT_TIMEOUT_SECONDS: int = 15 # Model tier routing — maps action types to model tiers AI_MODEL_TIERS: dict[str, str] = { -- 2.49.1 From e8ba74ed6dabc77f329341f9576cacce0f044ab9 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Tue, 28 Apr 2026 00:34:32 -0400 Subject: [PATCH 23/34] feat(escalations): distinguishable notifications, async AI, richer sidebar MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three improvements driven by live wedge testing. 1) Notification title now includes a problem snippet and PSA ticket suffix when present: "Escalation from Jane · #12345: Outlook is failing to sync email…" Replaces the prior "Session escalated by Jane" copy that made every escalation from the same junior look identical in the bell panel. Snippet is trimmed to 70 chars with ellipsis. handoff_manager now passes psa_ticket_id through in the notify() payload so this works for both /escalate and /handoff entry points. 2) AI enrichment (assessment + enhanced escalation_package) moved to a FastAPI BackgroundTask. The escalating engineer no longer waits on 15-25s of Sonnet latency — handoff creation returns as soon as snapshot, status flip, dual-write, documentation, PSA push, and notify() are committed. enrich_escalation_async opens its own DB session, runs both AI calls, updates handoff.ai_assessment + session.escalation_package, commits, and publishes a new `handoff_assessment_ready` event on the escalation bus. Frontend doesn't yet listen for that event — the magic-moment screen still shows a placeholder ("AI assessment is still generating. Reopen this view in a few seconds…") which is honest about the state. Live polling / auto-refresh on the bus event is the natural next step. 3) ChatSidebar entries now surface the problem summary as a secondary line and tag PSA-linked sessions with a monospace #ticket badge plus an "Escalated" pill on in-transit sessions. ChatListItem grew problem_summary, psa_ticket_id, and status fields; loadChats populates them from listSessions. The user couldn't tell their own sessions apart in the sidebar because they all rendered as "New Chat" with no distinguishing detail — this fixes that for any session, escalated or not. Test plan - Backend full suite: 1103 passed in 255.85s with -n auto. - Frontend tsc -b clean. Co-Authored-By: Claude Opus 4.7 --- backend/app/api/endpoints/ai_sessions.py | 13 +- backend/app/api/endpoints/session_handoffs.py | 11 +- backend/app/services/handoff_manager.py | 168 ++++++++++++++---- backend/app/services/notification_service.py | 24 ++- .../src/components/assistant/ChatSidebar.tsx | 27 ++- .../flowpilot/HandoffContextScreen.tsx | 5 +- frontend/src/pages/AssistantChatPage.tsx | 3 + frontend/src/types/assistant-chat.ts | 8 + 8 files changed, 218 insertions(+), 41 deletions(-) diff --git a/backend/app/api/endpoints/ai_sessions.py b/backend/app/api/endpoints/ai_sessions.py index f8ceb9fd..4fe4ab28 100644 --- a/backend/app/api/endpoints/ai_sessions.py +++ b/backend/app/api/endpoints/ai_sessions.py @@ -15,7 +15,7 @@ from datetime import datetime from typing import Annotated, Optional from uuid import UUID -from fastapi import APIRouter, Depends, HTTPException, Query, Request, status +from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query, Request, status from sqlalchemy import or_, select, func, text from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.orm import selectinload @@ -466,12 +466,13 @@ async def escalate_session( request: Request, session_id: UUID, data: EscalateSessionRequest, + background_tasks: BackgroundTasks, current_user: Annotated[User, Depends(get_current_active_user)], db: Annotated[AsyncSession, Depends(get_db)], _: None = Depends(require_engineer_or_admin), ): """Escalate a FlowPilot session — unified through HandoffManager.""" - from app.services.handoff_manager import HandoffManager + from app.services.handoff_manager import HandoffManager, enrich_escalation_async # Owner-only — matches the original constraint on flowpilot_engine.escalate_session. session_result = await db.execute( @@ -507,6 +508,14 @@ async def escalate_session( await manager.dispatch_escalation_notifications(handoff) + # AI enrichment (Sonnet assessment + enhanced escalation_package) runs + # in the background so the escalating engineer doesn't wait on + # 15-25s of model latency. Result lands on the handoff row when ready; + # the senior's magic-moment screen reads it at pickup time. + background_tasks.add_task( + enrich_escalation_async, handoff.id, current_user.id + ) + return SessionCloseResponse( session_id=session.id, status=session.status, diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 5c70c1e2..48ec3168 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -12,7 +12,7 @@ import logging from typing import Annotated, AsyncGenerator from uuid import UUID -from fastapi import APIRouter, Depends, HTTPException, Request, status +from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request, status from fastapi.responses import StreamingResponse from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession @@ -41,6 +41,7 @@ router = APIRouter(prefix="/ai-sessions/{session_id}", tags=["session-handoffs"] async def create_handoff( session_id: UUID, body: HandoffCreateRequest, + background_tasks: BackgroundTasks, current_user: Annotated[User, Depends(get_current_active_user)], db: Annotated[AsyncSession, Depends(get_db)], ) -> HandoffResponse: @@ -79,7 +80,15 @@ async def create_handoff( # a rolled-back handoff. Failures are swallowed inside the manager — # handoff creation is authoritative; notifications are advisory. if handoff.intent == "escalate": + from app.services.handoff_manager import enrich_escalation_async + await manager.dispatch_escalation_notifications(handoff) + # AI enrichment (Sonnet assessment + enhanced escalation_package) + # runs in the background after the response is sent so the + # escalating engineer doesn't wait on 15-25s of model latency. + background_tasks.add_task( + enrich_escalation_async, handoff.id, current_user.id + ) return HandoffResponse.model_validate(handoff) diff --git a/backend/app/services/handoff_manager.py b/backend/app/services/handoff_manager.py index 3684ce20..cfefafd3 100644 --- a/backend/app/services/handoff_manager.py +++ b/backend/app/services/handoff_manager.py @@ -89,17 +89,16 @@ class HandoffManager: f"Cannot escalate session in status: {session.status}" ) - # Generate snapshot + # Generate snapshot — fast, no AI calls. snapshot = await self._generate_snapshot(session) - # Generate AI assessment for escalations - ai_assessment = None - ai_assessment_data = None - if intent == "escalate": - ai_assessment, ai_assessment_data = ( - await self._generate_ai_assessment_with_timeout(session) - ) - + # AI enrichment (assessment + enhanced escalation_package) is now + # deferred to a background task scheduled by the endpoint after + # commit — both calls hit Sonnet and together can take 15-25s, + # which is too long to block the click path. The handoff row lands + # immediately with `ai_assessment=None`; the magic-moment screen + # shows "Assessment still computing" until enrich_async finishes + # and the senior refreshes (or, eventually, polls). handoff = SessionHandoff( session_id=session_id, account_id=session.account_id, @@ -107,8 +106,8 @@ class HandoffManager: intent=intent, source_branch_id=session.active_branch_id, snapshot=snapshot, - ai_assessment=ai_assessment, - ai_assessment_data=ai_assessment_data, + ai_assessment=None, + ai_assessment_data=None, engineer_notes=engineer_notes, priority=priority, ) @@ -125,27 +124,17 @@ class HandoffManager: session.handoff_count = (session.handoff_count or 0) + 1 - # Dual-write to escalation_package. For escalate, build the - # AI-enhanced package (preserves the legacy rich shape that - # SessionBriefing/PSA writeback consume), then layer in the new - # handoff metadata. For park, the lightweight shape is fine — - # there's no legacy enhanced package for parking. - if intent == "escalate": - enhanced_pkg = await self._build_enhanced_escalation_package( - session, user_id - ) - enhanced_pkg["intent"] = intent - enhanced_pkg["engineer_notes"] = engineer_notes - enhanced_pkg["handoff_id"] = str(handoff.id) - enhanced_pkg["snapshot"] = snapshot - session.escalation_package = enhanced_pkg - else: - session.escalation_package = { - "snapshot": snapshot, - "intent": intent, - "engineer_notes": engineer_notes, - "handoff_id": str(handoff.id), - } + # Dual-write the minimal escalation_package shape now. The async + # enrichment task overwrites this with the AI-enhanced shape + # (`steps_tried`, `remaining_hypotheses`, etc.) when it completes — + # consumers that read these fields (PSA writeback, legacy + # SessionBriefing) tolerate either shape. + session.escalation_package = { + "snapshot": snapshot, + "intent": intent, + "engineer_notes": engineer_notes, + "handoff_id": str(handoff.id), + } await self.db.flush() return handoff @@ -211,6 +200,10 @@ class HandoffManager: "engineer_name": engineer_name, "escalation_reason": handoff.engineer_notes or "", "problem_summary": session.problem_summary or "N/A", + # Surface the PSA ticket id in the bell-icon title so two + # similarly-worded escalations are still distinguishable + # at a glance. + "psa_ticket_id": session.psa_ticket_id, }, self.db, target_user_ids=target_user_ids, @@ -247,6 +240,7 @@ class HandoffManager: ) return {} + async def dispatch_escalation_notifications( self, handoff: SessionHandoff ) -> int: @@ -585,3 +579,113 @@ class HandoffManager: }) return queue_items + + +async def enrich_escalation_async(handoff_id: UUID, user_id: UUID) -> None: + """Run the AI enrichment for an escalation handoff in the background. + + Scheduled by `/escalate` and `/handoff` (intent=escalate) endpoints via + FastAPI BackgroundTasks. Opens its own DB session because the request + session is closed by the time this runs. Generates: + + 1. The legacy AI-enhanced escalation_package (Sonnet, ~5-10s) — saved + to `session.escalation_package`, preserving the `intent` / + `engineer_notes` / `handoff_id` keys the dual-write set so legacy + consumers keep working. + 2. The diagnostic AI assessment (Sonnet, ~4-15s) — saved to + `handoff.ai_assessment` and `handoff.ai_assessment_data`. + + On completion publishes a `handoff_assessment_ready` event on the + escalation bus so any connected magic-moment screen can refresh + without a manual reload. Failures are logged but never propagated — + the click-path-side handoff creation already committed, so worst case + the senior sees the "Assessment still computing" placeholder until + they refresh manually. + """ + from app.core.database import async_session_maker + from app.core.escalation_bus import bus as escalation_bus + + async with async_session_maker() as db: + try: + result = await db.execute( + select(SessionHandoff).where(SessionHandoff.id == handoff_id) + ) + handoff = result.scalar_one_or_none() + if not handoff or handoff.intent != "escalate": + return + + session_result = await db.execute( + select(AISession) + .options(selectinload(AISession.steps), selectinload(AISession.user)) + .where(AISession.id == handoff.session_id) + ) + session = session_result.scalar_one_or_none() + if not session: + logger.warning( + "enrich_escalation_async: session %s gone for handoff %s", + handoff.session_id, + handoff_id, + ) + return + + manager = HandoffManager(db) + + # Build the enhanced package (Sonnet). Don't fail the whole + # task if it errors — the assessment is independently useful. + try: + enhanced_pkg = await manager._build_enhanced_escalation_package( + session, user_id + ) + if enhanced_pkg: + enhanced_pkg["intent"] = "escalate" + enhanced_pkg["engineer_notes"] = handoff.engineer_notes + enhanced_pkg["handoff_id"] = str(handoff.id) + if isinstance(session.escalation_package, dict): + enhanced_pkg.setdefault( + "snapshot", session.escalation_package.get("snapshot") + ) + session.escalation_package = enhanced_pkg + except Exception: + logger.exception( + "enrich_escalation_async: enhanced package build failed for handoff %s", + handoff_id, + ) + + # Generate the diagnostic AI assessment. + try: + ai_assessment, ai_assessment_data = ( + await manager._generate_ai_assessment_with_timeout(session) + ) + handoff.ai_assessment = ai_assessment + handoff.ai_assessment_data = ai_assessment_data + except Exception: + logger.exception( + "enrich_escalation_async: assessment generation failed for handoff %s", + handoff_id, + ) + + await db.commit() + + try: + await escalation_bus.publish( + handoff.account_id, + { + "type": "handoff_assessment_ready", + "handoff_id": str(handoff.id), + "session_id": str(handoff.session_id), + "has_assessment": handoff.ai_assessment is not None, + }, + ) + except Exception: + logger.exception( + "enrich_escalation_async: bus publish failed for handoff %s", + handoff_id, + ) + except Exception: + logger.exception( + "enrich_escalation_async failed for handoff %s", handoff_id + ) + try: + await db.rollback() + except Exception: + pass diff --git a/backend/app/services/notification_service.py b/backend/app/services/notification_service.py index a817b9b6..edf1bf7d 100644 --- a/backend/app/services/notification_service.py +++ b/backend/app/services/notification_service.py @@ -371,13 +371,35 @@ async def _send_teams_message( def _build_notification_title(event: str, payload: dict[str, Any]) -> str: """Human-readable title per event type.""" titles = { - "session.escalated": "Session escalated by {engineer_name}", + # Distinguishability matters in the bell panel: with a generic title + # ("Session escalated by Jane") two different escalations from the + # same junior look like a duplicate notification. Including a short + # problem snippet (and ticket number if present) lets the senior + # tell them apart at a glance. + "session.escalated": "Escalation from {engineer_name}{ticket_suffix}: {problem_snippet}", "session.high_priority": "High-priority session started: {ticket_number}", "proposal.pending": "New flow proposal: {title}", "proposal.approved": "Flow proposal approved: {title}", "knowledge_gap.detected": "Knowledge gap detected: {gap_type}", "test": "Test Notification from ResolutionFlow", } + + # Build the escalation-specific derived fields. Done here rather than at + # the call site so every dispatch path (legacy /escalate shim, /handoff, + # any future entry point) gets consistent formatting without each one + # having to repeat the snippet logic. + if event == "session.escalated": + problem = (payload.get("problem_summary") or "").strip() + if not problem or problem.upper() == "N/A": + problem_snippet = "(no summary provided)" + elif len(problem) > 70: + problem_snippet = problem[:67].rstrip() + "…" + else: + problem_snippet = problem + ticket = payload.get("psa_ticket_id") or payload.get("ticket_number") + ticket_suffix = f" · #{ticket}" if ticket else "" + payload = {**payload, "problem_snippet": problem_snippet, "ticket_suffix": ticket_suffix} + template = titles.get(event, f"Notification: {event}") try: return template.format(**payload) diff --git a/frontend/src/components/assistant/ChatSidebar.tsx b/frontend/src/components/assistant/ChatSidebar.tsx index 28c19239..4a0d2e75 100644 --- a/frontend/src/components/assistant/ChatSidebar.tsx +++ b/frontend/src/components/assistant/ChatSidebar.tsx @@ -219,10 +219,31 @@ function ChatItem({ ) : ( <> -
{chat.title}
-
- {chat.message_count} messages +
+
{chat.title}
+ {chat.psa_ticket_id && ( + + #{chat.psa_ticket_id} + + )} + {(chat.status === 'escalated' || chat.status === 'requesting_escalation') && ( + + Escalated + + )}
+ {/* Secondary line: problem snippet when the title doesn't already + carry it, otherwise the message count. Keeps untitled + sessions from collapsing into identical-looking rows. */} + {chat.problem_summary && chat.problem_summary !== chat.title ? ( +
+ {chat.problem_summary} +
+ ) : ( +
+ {chat.message_count} messages +
+ )} )}
diff --git a/frontend/src/components/flowpilot/HandoffContextScreen.tsx b/frontend/src/components/flowpilot/HandoffContextScreen.tsx index 8c055cc7..5f3e8aa7 100644 --- a/frontend/src/components/flowpilot/HandoffContextScreen.tsx +++ b/frontend/src/components/flowpilot/HandoffContextScreen.tsx @@ -241,8 +241,9 @@ export function HandoffContextScreen({
- Assessment unavailable — model didn't respond in time. Pick up - the session to investigate directly. + AI assessment is still generating. Reopen this view in a few + seconds to see it, or pick up the session to investigate + directly.
) : ( diff --git a/frontend/src/pages/AssistantChatPage.tsx b/frontend/src/pages/AssistantChatPage.tsx index f6c7e5ba..ed39d848 100644 --- a/frontend/src/pages/AssistantChatPage.tsx +++ b/frontend/src/pages/AssistantChatPage.tsx @@ -440,6 +440,9 @@ export default function AssistantChatPage() { pinned: false, created_at: s.created_at, updated_at: s.created_at, + problem_summary: s.problem_summary, + psa_ticket_id: s.psa_ticket_id, + status: s.status, }))) } catch { // silently handle diff --git a/frontend/src/types/assistant-chat.ts b/frontend/src/types/assistant-chat.ts index 8b0d25f4..dbc5cf36 100644 --- a/frontend/src/types/assistant-chat.ts +++ b/frontend/src/types/assistant-chat.ts @@ -5,6 +5,14 @@ export interface ChatListItem { pinned: boolean created_at: string updated_at: string + // Optional secondary fields used by the sidebar to make untitled / generic + // sessions distinguishable. `problem_summary` powers the secondary line + // when the title doesn't already carry it; `psa_ticket_id` shows as a + // monospace badge so PSA-linked sessions are obvious; `status` lets us + // tag escalated / picked-up sessions with a color cue. + problem_summary?: string | null + psa_ticket_id?: string | null + status?: string | null } export interface RetentionSettings { -- 2.49.1 From 891439133688cd0adece528fbe5bab567540d576 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Tue, 28 Apr 2026 01:26:29 -0400 Subject: [PATCH 24/34] fix(assistant-chat): kill stale task-lane flash on new-session entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two compounding bugs caused the previous session's questions/actions to render briefly when entering a new chat — visible as "the new session instantly pops with old session task-lane data" the user reported. The race - AssistantChatPage's activeQuestions / activeActions / showTaskLane useState initializers synchronously read sessionStorage's rf-tasklane-meta. They restore the persisted task-lane state if its saved chatId matches the freshly-resolved activeChatId. - On dashboard prefill flow, the page mounts on /pilot with location.state.prefill set; activeChatId initializes from sessionStorage's rf-active-chat-id (the previous session). The previous session's task-lane meta matches that chatId — so the initializer restores it. First paint shows old questions/actions. sendPrefill's resetSessionDerivedState fires later from a useEffect, but only after the flash. - Same pattern hits the senior-pickup flow: ?pickup=true means we're about to render the magic-moment screen and discard whatever chat the senior was previously on, but the underlying chat surface still initializes with their old task-lane meta. The amplifier - resetSessionDerivedState wiped the in-memory state but never removed sessionStorage's rf-tasklane-meta. Any remount or reload before the next persistence-effect write could re-hydrate the cleared state from the still-stale sessionStorage entry. Fixes - Initializer guard: when location.state.prefill is set OR ?pickup=true is in the URL, skip the sessionStorage restore entirely. Kills the first-paint flash for both entry paths. - Eager wipe: resetSessionDerivedState now also calls sessionStorage.removeItem('rf-tasklane-meta'). The persistence effect re-saves on the next state change anyway, so the only window where sessionStorage is empty is the exact window where stale-tag leakage was happening. tsc -b clean. No backend changes. Co-Authored-By: Claude Opus 4.7 --- frontend/src/pages/AssistantChatPage.tsx | 26 ++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/frontend/src/pages/AssistantChatPage.tsx b/frontend/src/pages/AssistantChatPage.tsx index ed39d848..58d76522 100644 --- a/frontend/src/pages/AssistantChatPage.tsx +++ b/frontend/src/pages/AssistantChatPage.tsx @@ -97,7 +97,21 @@ export default function AssistantChatPage() { const [logContent, setLogContent] = useState('') const [pendingUploads, setPendingUploads] = useState([]) const [isDragOver, setIsDragOver] = useState(false) + // Task-lane mount restoration is gated on (a) the persisted chatId + // matching whatever activeChatId resolved to, AND (b) the page not being + // entered with a prefill in location.state. The prefill case means we're + // about to create a brand-new session and discard the previous one's + // task lane anyway — restoring it just causes the previous chat's + // questions/actions to flash on the first paint before sendPrefill's + // resetSessionDerivedState clears them. Same logic for the bell-icon + // pickup flow (?pickup=true): the senior is entering an unrelated + // session and any leftover task-lane meta from their own prior chat is + // noise. Both gates collapse to "are we about to leave the previous + // chat behind?" — if yes, start clean. + const incomingPrefill = !!(location.state as { prefill?: string } | null)?.prefill + const skipTaskLaneRestore = incomingPrefill || isPickup const [activeQuestions, setActiveQuestions] = useState(() => { + if (skipTaskLaneRestore) return [] try { const saved = sessionStorage.getItem('rf-tasklane-meta') if (saved) { const d = JSON.parse(saved); if (d.chatId === activeChatId) return d.questions || [] } @@ -105,6 +119,7 @@ export default function AssistantChatPage() { return [] }) const [activeActions, setActiveActions] = useState(() => { + if (skipTaskLaneRestore) return [] try { const saved = sessionStorage.getItem('rf-tasklane-meta') if (saved) { const d = JSON.parse(saved); if (d.chatId === activeChatId) return d.actions || [] } @@ -112,6 +127,7 @@ export default function AssistantChatPage() { return [] }) const [showTaskLane, setShowTaskLane] = useState(() => { + if (skipTaskLaneRestore) return false try { const saved = sessionStorage.getItem('rf-tasklane-meta') if (saved) { const d = JSON.parse(saved); return d.show === true && d.chatId === activeChatId } @@ -479,6 +495,16 @@ export default function AssistantChatPage() { // Phase 9: tab strip reset setChatTab('chat') setScriptBuilderHasProgress(false) + // Belt-and-braces: also wipe the persisted task-lane meta. Without this, + // a remount or page reload before the next AI response can re-hydrate + // the previous session's questions/actions from sessionStorage even + // though the in-memory state has been cleared. The persistence effect + // re-saves on the next state change anyway, so the only window where + // sessionStorage is empty is between this reset and the next response — + // which is exactly the window where stale-tag leakage was happening. + try { + sessionStorage.removeItem('rf-tasklane-meta') + } catch { /* ignore */ } }, []) // Phase 2 facts — fetch + handlers. `refreshFacts` is called from selectChat -- 2.49.1 From 0f00ee5e01f32afe4410415391348dbc3e7452dc Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Tue, 28 Apr 2026 01:59:28 -0400 Subject: [PATCH 25/34] feat(escalations): close out plan-locked wedge polish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four items from the design-plan audit, all flagged as locked-design or Codex corrections, shipped together so the GTM demo path covers them end-to-end before bug bash. 1. Live AI assessment refresh on the magic-moment screen. Backend already publishes handoff_assessment_ready when enrich_escalation_async commits; wire the frontend listener so the senior sees the assessment populate without a manual reopen. New event type + onAssessmentReady handler on streamEscalations; AssistantChatPage opens a scoped SSE subscription whenever it tracks a handoff missing its assessment, refetches on match, and replaces magicHandoff / overlayHandoff in place. Closes the loop on the async-assessment commit e8ba74e. 2. Suggested-step chips below the chat input. Locked design from the plan (Codex correction). Chip strip renders above the composer post-claim when ai_assessment_data.suggested_steps[] is non-empty. Click prefills the input and focuses; first send or explicit X hides for the session. 3. Unread 6px dot on EscalationQueue cards. localStorage-persisted seen set (rf-escalation-seen, capped 200). Dot top-right when not seen. Cleared on open (card click) or claim (Pick Up) — NOT on hover, per Codex correction. Pick Up stops propagation so it doesn't double-fire. 4. Race-condition toast on claim conflict. The /claim endpoint previously silently overwrote claimed_by — both seniors thought they owned the session. New HandoffAlreadyClaimedError carries the winner's id/name/ timestamp; claim_session rejects different-user re-claims (same-user is idempotent for double-click safety); endpoint returns 409 with structured detail. AssistantChatPage.handleStartHere extracts and surfaces "Already claimed by {name} {time_ago}." via toast, drops ?pickup=true, dismisses magic-moment so the loser flows back to queue. Tests: 2 new unit tests in test_handoff_manager.py (conflict raises, same-user idempotent). Full handoff + escalation suite (34 tests) green. Frontend tsc -b clean. Co-Authored-By: Claude Opus 4.7 --- .ai/CURRENT_TASK.md | 18 ++- backend/app/api/endpoints/session_handoffs.py | 15 +- backend/app/services/handoff_manager.py | 45 +++++- backend/tests/test_handoff_manager.py | 93 ++++++++++++ frontend/src/api/aiSessions.ts | 8 + .../components/flowpilot/EscalationQueue.tsx | 69 ++++++++- frontend/src/pages/AssistantChatPage.tsx | 137 ++++++++++++++++++ frontend/src/types/ai-session.ts | 11 ++ 8 files changed, 385 insertions(+), 11 deletions(-) diff --git a/.ai/CURRENT_TASK.md b/.ai/CURRENT_TASK.md index d4205ad0..444f1ca8 100644 --- a/.ai/CURRENT_TASK.md +++ b/.ai/CURRENT_TASK.md @@ -34,12 +34,18 @@ ## Remaining work on this branch -1. **Visual QA in a real browser** via `/qa` — slide-in animation, tab-title flash, magic-moment layout, dissolve, full junior-escalates → senior-receives → senior-claims demo path. -2. **Suggested-step chips below the chat input** (Codex correction, design plan locks this) — surfaces `ai_assessment_data.suggested_steps[]` as clickable chips in `FlowPilotMessageBar` that prefill the input. Threading through `FlowPilotSession` → message bar. -3. **Snapshot expansion in `HandoffManager._generate_snapshot`** — include the recent diagnostic steps / conversation tail so the magic-moment screen's "What's been tried" section can render the actual timeline pre-claim instead of "full timeline available after pickup". -4. **Toolbar Context button on legacy-arrival sessions** — currently the button only appears when the senior arrived via the magic-moment flow this session. Lazy-fetching the handoff list on session-load (when status was-escalated) would make it work on revisits. -5. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. Optional for v1 demo. -6. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives via SSE → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. +1. **Visual QA + bug bash** in a real browser — full pickup demo path with the four new pieces below; this is the next active step. +2. **Snapshot expansion in `HandoffManager._generate_snapshot`** — include the recent diagnostic steps / conversation tail so the magic-moment screen's "What's been tried" section can render the actual timeline pre-claim instead of "full timeline available after pickup". +3. **Toolbar Context button on legacy-arrival sessions** — currently the button only appears when the senior arrived via the magic-moment flow this session. Lazy-fetching the handoff list on session-load (when status was-escalated) would make it work on revisits. +4. **Owner-facing analytics page** at `/analytics/escalations` — period selector, conversion-rate, trend chart. ~0.5d. Optional for v1 demo. +5. **Playwright e2e** for the magic-moment demo flow (junior escalates → senior receives via SSE → senior claims → opens session). Critical for the GTM Loom not to crash mid-recording. + +## Just shipped (4 plan-locked items, this session) + +- **Live AI assessment refresh on the magic-moment screen.** New `HandoffAssessmentReadyEvent` type + `onAssessmentReady` handler on `streamEscalations`. `AssistantChatPage` opens a scoped SSE subscription whenever it has a tracked handoff with no AI assessment yet; on a matching event it refetches and replaces both `magicHandoff` and `overlayHandoff` in place. Closes the loop on the async-assessment commit `e8ba74e`. +- **Suggested-step chips below the chat input.** New `chipsHidden` state in `AssistantChatPage` defaulting to false; a chip strip renders above the composer when `magicHandoff?.ai_assessment_data?.suggested_steps[]` is non-empty and the magic-moment has dissolved. Click prefills input + focus; first send hides the strip; explicit X also hides. Per-session lifetime (Codex correction locked design). +- **Unread 6px dot on `EscalationQueue` cards.** localStorage-persisted seen set (`rf-escalation-seen`, capped 200). Dot renders top-right of any card not yet seen. Cleared on **open (card click) or claim (Pick Up)** — NOT on hover (Codex correction). Pick Up onClick now stops propagation so the wrapper's open handler isn't double-fired. +- **Race-condition toast on claim conflict.** New `HandoffAlreadyClaimedError` exception class in `handoff_manager.py`. `claim_session` now eager-loads `claimed_by_user`, rejects different-user re-claims (idempotent for same-user), and raises with the winner's id/name/timestamp. Endpoint translates to 409 with structured detail. `AssistantChatPage.handleStartHere` extracts the detail, formats `"Already claimed by {name} {time_ago}."` via `timeAgo()`, drops `?pickup=true`, and dismisses the magic-moment so the loser flows back to the queue. Backed by 2 new unit tests in `test_handoff_manager.py`. ## Two-metric framing — read this before quoting numbers to anyone diff --git a/backend/app/api/endpoints/session_handoffs.py b/backend/app/api/endpoints/session_handoffs.py index 48ec3168..995419b9 100644 --- a/backend/app/api/endpoints/session_handoffs.py +++ b/backend/app/api/endpoints/session_handoffs.py @@ -22,7 +22,7 @@ from app.core.escalation_bus import bus as escalation_bus from app.models.user import User from app.models.ai_session import AISession from app.models.session_handoff import SessionHandoff -from app.services.handoff_manager import HandoffManager +from app.services.handoff_manager import HandoffAlreadyClaimedError, HandoffManager from app.schemas.session_handoff import ( HandoffCreateRequest, HandoffResponse, @@ -129,6 +129,19 @@ async def claim_handoff( handoff_id=handoff_id, claiming_user_id=current_user.id, ) + except HandoffAlreadyClaimedError as e: + # Loser of the race — the API surfaces structured detail so the + # client can render "Already claimed by {name} {time_ago}" without + # a follow-up fetch. + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail={ + "error": "already_claimed", + "claimed_by_id": str(e.claimed_by_id), + "claimed_by_name": e.claimed_by_name, + "claimed_at": e.claimed_at.isoformat(), + }, + ) except ValueError as e: raise HTTPException(status_code=404, detail=str(e)) diff --git a/backend/app/services/handoff_manager.py b/backend/app/services/handoff_manager.py index cfefafd3..8f0624cb 100644 --- a/backend/app/services/handoff_manager.py +++ b/backend/app/services/handoff_manager.py @@ -36,6 +36,30 @@ from app.services.notification_service import notify logger = logging.getLogger(__name__) +class HandoffAlreadyClaimedError(Exception): + """Raised when a senior tries to claim a handoff another senior already won. + + Carries the winning claimer's id, display name, and claim timestamp so the + API layer can surface a "Already claimed by {name} {time_ago}" toast on + the losing client. The race story is the locked design — without this + exception the endpoint would silently overwrite `claimed_by` and both + seniors would think they own the session. + """ + + def __init__( + self, + claimed_by_id: UUID, + claimed_by_name: str, + claimed_at: datetime, + ) -> None: + super().__init__( + f"Handoff already claimed by {claimed_by_name} at {claimed_at.isoformat()}" + ) + self.claimed_by_id = claimed_by_id + self.claimed_by_name = claimed_by_name + self.claimed_at = claimed_at + + class HandoffManager: """Unified park/escalate handoff management.""" @@ -398,14 +422,31 @@ class HandoffManager: handoff_id: UUID, claiming_user_id: UUID, ) -> SessionHandoff: - """Claim a handed-off session.""" + """Claim a handed-off session. + + If the handoff was already claimed by a *different* user (the race + story: two seniors clicking Pick Up simultaneously), raise + `HandoffAlreadyClaimedError` with the winning claimer's details so + the API can return 409 with the data the loser's toast needs. A + re-claim by the same user is idempotent. + """ result = await self.db.execute( - select(SessionHandoff).where(SessionHandoff.id == handoff_id) + select(SessionHandoff) + .options(selectinload(SessionHandoff.claimed_by_user)) + .where(SessionHandoff.id == handoff_id) ) handoff = result.scalar_one_or_none() if not handoff: raise ValueError(f"Handoff {handoff_id} not found") + if handoff.claimed_by is not None and handoff.claimed_by != claiming_user_id: + claimer = handoff.claimed_by_user + raise HandoffAlreadyClaimedError( + claimed_by_id=handoff.claimed_by, + claimed_by_name=claimer.name if claimer else "another engineer", + claimed_at=handoff.claimed_at or datetime.now(timezone.utc), + ) + handoff.claimed_by = claiming_user_id handoff.claimed_at = datetime.now(timezone.utc) diff --git a/backend/tests/test_handoff_manager.py b/backend/tests/test_handoff_manager.py index a2e75c05..15c76020 100644 --- a/backend/tests/test_handoff_manager.py +++ b/backend/tests/test_handoff_manager.py @@ -189,6 +189,99 @@ async def test_claim_session(client: AsyncClient, test_user, test_admin, auth_he assert session.status == "active" +@pytest.mark.asyncio +async def test_claim_session_conflict_raises_already_claimed( + client: AsyncClient, test_user, test_admin, auth_headers, test_db +): + """Two seniors claiming simultaneously: the second raises the typed + HandoffAlreadyClaimedError carrying the winner's identity. Without this + guard both calls would silently overwrite claimed_by — the locked + race-condition story depends on a real conflict response.""" + from app.services.handoff_manager import ( + HandoffAlreadyClaimedError, + HandoffManager, + ) + + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.flush() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="Need help", + user_id=test_user["user_data"]["id"], + ) + + # First claim — admin wins. + await manager.claim_session( + handoff_id=handoff.id, + claiming_user_id=test_admin["user_data"]["id"], + ) + + # Second claim by a different user — owner of the original session, + # standing in for "the other senior who lost the race." + with pytest.raises(HandoffAlreadyClaimedError) as exc_info: + await manager.claim_session( + handoff_id=handoff.id, + claiming_user_id=test_user["user_data"]["id"], + ) + + err = exc_info.value + assert err.claimed_by_id == test_admin["user_data"]["id"] + assert err.claimed_by_name # populated from User.name + assert err.claimed_at is not None + + +@pytest.mark.asyncio +async def test_claim_session_idempotent_for_same_user( + client: AsyncClient, test_user, test_admin, auth_headers, test_db +): + """A re-claim by the user who already won is a no-op, not a conflict. + Defends against double-clicks / network retries on the loser-side toast.""" + session = AISession( + user_id=test_user["user_data"]["id"], + account_id=test_user["user_data"]["account_id"], + session_type="guided", + intake_type="free_text", + intake_content={"text": "test"}, + status="active", + confidence_tier="discovery", + conversation_messages=[], + ) + test_db.add(session) + await test_db.flush() + + manager = HandoffManager(test_db) + handoff = await manager.create_handoff( + session_id=session.id, + intent="escalate", + engineer_notes="Need help", + user_id=test_user["user_data"]["id"], + ) + + first = await manager.claim_session( + handoff_id=handoff.id, + claiming_user_id=test_admin["user_data"]["id"], + ) + second = await manager.claim_session( + handoff_id=handoff.id, + claiming_user_id=test_admin["user_data"]["id"], + ) + + assert first.claimed_by == second.claimed_by == test_admin["user_data"]["id"] + + # ─── Notification dispatch ──────────────────────────────────────────────────── diff --git a/frontend/src/api/aiSessions.ts b/frontend/src/api/aiSessions.ts index 90531a8d..d59d43dd 100644 --- a/frontend/src/api/aiSessions.ts +++ b/frontend/src/api/aiSessions.ts @@ -19,6 +19,7 @@ import type { ChatMessageRequest, ChatMessageResponse, HandoffCreatedEvent, + HandoffAssessmentReadyEvent, EscalationStreamHandlers, } from '@/types/ai-session' @@ -279,6 +280,13 @@ export const aiSessionsApi = { const parsed = JSON.parse(data) as Record if (eventType === 'handoff_created' && parsed.type === 'handoff_created') { handlers.onHandoffCreated?.(parsed as unknown as HandoffCreatedEvent) + } else if ( + eventType === 'handoff_assessment_ready' && + parsed.type === 'handoff_assessment_ready' + ) { + handlers.onAssessmentReady?.( + parsed as unknown as HandoffAssessmentReadyEvent, + ) } else if (eventType === 'ready') { handlers.onReady?.() } diff --git a/frontend/src/components/flowpilot/EscalationQueue.tsx b/frontend/src/components/flowpilot/EscalationQueue.tsx index dbce00aa..98e73bdb 100644 --- a/frontend/src/components/flowpilot/EscalationQueue.tsx +++ b/frontend/src/components/flowpilot/EscalationQueue.tsx @@ -26,6 +26,34 @@ const sortNewestFirst = (a: AISessionSummary, b: AISessionSummary) => // state transition. const NEW_CARD_HIGHLIGHT_MS = 800 +// localStorage key for the per-user "seen" set. Tracks session IDs the user +// has acknowledged so the unread dot doesn't reappear on refresh. Bounded to +// the last `SEEN_CAP` entries to avoid unbounded growth on long-lived +// accounts. +const SEEN_STORAGE_KEY = 'rf-escalation-seen' +const SEEN_CAP = 200 + +function loadSeenIds(): Set { + try { + const raw = localStorage.getItem(SEEN_STORAGE_KEY) + if (!raw) return new Set() + const parsed = JSON.parse(raw) as unknown + if (!Array.isArray(parsed)) return new Set() + return new Set(parsed.filter((v): v is string => typeof v === 'string')) + } catch { + return new Set() + } +} + +function saveSeenIds(ids: Set): void { + try { + const arr = Array.from(ids).slice(-SEEN_CAP) + localStorage.setItem(SEEN_STORAGE_KEY, JSON.stringify(arr)) + } catch { + // localStorage unavailable / quota — silent. The dot just won't persist. + } +} + function waitTimeColor(createdAt: string): string { const hours = (Date.now() - new Date(createdAt).getTime()) / 3_600_000 if (hours >= 4) return '#f87171' // danger @@ -42,6 +70,20 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp const [newIds, setNewIds] = useState>(new Set()) // Track count of unseen arrivals while the tab is backgrounded. const [unseenCount, setUnseenCount] = useState(0) + // Per-user seen set persisted in localStorage. Cleared on open, claim, or + // explicit dismiss (NOT on hover — Codex correction). The unread dot is + // shown for any session id NOT in this set. + const [seenIds, setSeenIds] = useState>(() => loadSeenIds()) + + const markSeen = useCallback((sessionId: string) => { + setSeenIds(prev => { + if (prev.has(sessionId)) return prev + const next = new Set(prev) + next.add(sessionId) + saveSeenIds(next) + return next + }) + }, []) // Ref mirrors the latest sessions so the SSE handler can diff without // re-binding on every state change. @@ -190,6 +232,7 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp }, [handleHandoffCreated]) const handlePickup = (sessionId: string) => { + markSeen(sessionId) if (onPickup) { onPickup(sessionId) } else { @@ -197,6 +240,14 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp } } + // Click on the card body (anywhere outside Pick Up) marks the session as + // seen — the "open" affordance from the unread-dot spec. Pick Up handles + // its own marking via handlePickup. Hover deliberately does NOT clear + // (Codex correction). + const handleCardOpen = (sessionId: string) => { + markSeen(sessionId) + } + if (isLoading) { return (
@@ -256,15 +307,26 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp
{sessions.map((session) => { const isNew = newIds.has(session.id) + const isUnread = !seenIds.has(session.id) return (
handleCardOpen(session.id)} className={cn( - 'card-flat p-3 sm:p-4 space-y-3', + 'relative card-flat p-3 sm:p-4 space-y-3 cursor-pointer', isNew && !prefersReducedMotion && 'animate-slide-in-bottom', isNew && prefersReducedMotion && 'animate-fade-in', )} > + {/* Unread indicator: 6px dot, top-right corner. Cleared on + open (card click) or claim (Pick Up). Persists across + refresh via localStorage. */} + {isUnread && ( + + )}

{session.problem_summary || 'Untitled session'} @@ -303,7 +365,10 @@ export function EscalationQueue({ onPickup, onCountChange }: EscalationQueueProp

+ ))} +
+ +
+
+ )} + {/* Rich Input */}
void onHandoffCreated?: (event: HandoffCreatedEvent) => void + onAssessmentReady?: (event: HandoffAssessmentReadyEvent) => void } -- 2.49.1 From 665530f812cb8c0f56024bf8959744b4e009ad64 Mon Sep 17 00:00:00 2001 From: Michael Chihlas Date: Tue, 28 Apr 2026 02:42:31 -0400 Subject: [PATCH 26/34] fix(assistant-chat): tag task-lane state with owner chatId to kill stale flash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous fix (8914391) only blocked the mount-time sessionStorage restore when the page entered with prefill or ?pickup=true. It didn't cover any path where the page was already mounted and activeChatId flipped without the in-memory task-lane state going through reset+ repopulate cleanly — in-place URL navigation, mid-flight pickup, HMR re-runs, the gap between setActiveChatId(B) and the AI response that finally populates B's questions/actions. Root cause: activeQuestions / activeActions / showTaskLane were never intrinsically tied to a chatId. They were treated as "the active chat's data" by convention, with no structural enforcement. Any window where they survived past their owning chat leaked previous-session data into the new view. The persistence effect made it worse: it stamped the sessionStorage chatId field with activeChatId at write time, so a mid-transition snapshot {chatId: B, questions: [A's]} would happily restore A's data for B on the next mount. Fix: introduce taskLaneOwnerChatId state that records the chatId those in-memory questions/actions/show values BELONG to. Set at every site that populates them (sendPrefill, selectChat, handleSend, handleTaskSubmit, handleResumeNew, refreshFacts, handleApplyFix). Cleared in resetSessionDerivedState. The persistence effect now writes ownerChatId as the chatId tag, not activeChatId — so the snapshot is always self-consistent. Render gate: taskLaneIsForActiveChat = ownerChatId === activeChatId. ANDed into all three render conditions (toolbar Tasks button, narrow- viewport floating drawer, main side panel). The lane is structurally unable to display data tagged with a different chat. The mount-time skipTaskLaneRestore guard stays — it kills the flash between component mount and the first sendPrefill effect run, which the owner-gate alone doesn't cover. Co-Authored-By: Claude Opus 4.7 --- frontend/src/pages/AssistantChatPage.tsx | 60 +++++++++++++++++++++--- 1 file changed, 54 insertions(+), 6 deletions(-) diff --git a/frontend/src/pages/AssistantChatPage.tsx b/frontend/src/pages/AssistantChatPage.tsx index bda1f3a7..c366a8f4 100644 --- a/frontend/src/pages/AssistantChatPage.tsx +++ b/frontend/src/pages/AssistantChatPage.tsx @@ -142,6 +142,24 @@ export default function AssistantChatPage() { } catch { /* ignore */ } return false }) + // Task-lane owner: the chatId these in-memory questions/actions/show + // values BELONG to, set every time we populate the lane. Render is gated + // on `taskLaneOwnerChatId === activeChatId` so any path that flips the + // active chat without clearing the lane state (in-place URL change, + // mid-flight pickup, etc.) cannot leak the previous chat's task data + // into the new view. The mount-time flash protection still lives in + // `skipTaskLaneRestore`; this guard handles every other transition. + const [taskLaneOwnerChatId, setTaskLaneOwnerChatId] = useState(() => { + if (skipTaskLaneRestore) return null + try { + const saved = sessionStorage.getItem('rf-tasklane-meta') + if (saved) { + const d = JSON.parse(saved) + if (typeof d.chatId === 'string' && d.chatId === activeChatId) return d.chatId + } + } catch { /* ignore */ } + return null + }) const [sidebarCollapsed, setSidebarCollapsed] = useState(() => localStorage.getItem('rf-chat-sidebar-collapsed') === 'true' ) @@ -495,6 +513,7 @@ export default function AssistantChatPage() { setActiveQuestions(response.questions || []) setActiveActions(response.actions || []) setShowTaskLane(true) + setTaskLaneOwnerChatId(session.session_id) } // Refetch facts + active fix — the AI may have emitted markers. refreshSessionDerived(session.session_id) @@ -509,17 +528,31 @@ export default function AssistantChatPage() { // eslint-disable-next-line react-hooks/exhaustive-deps }, []) - // Persist task lane metadata to sessionStorage + // Render gate: the in-memory task-lane data is shown only when the chatId + // it belongs to (taskLaneOwnerChatId) matches activeChatId. Any path that + // flips activeChatId without clearing the lane state — in-place URL + // navigation, mid-flight pickup, HMR — produces a window where ownerChatId + // still tags the previous chat. The render gate keeps the lane hidden + // through that window until reset+repopulate runs for the new chat. + const taskLaneIsForActiveChat = + taskLaneOwnerChatId !== null && taskLaneOwnerChatId === activeChatId + + // Persist task lane metadata to sessionStorage. The chatId field tags + // ownership — the chatId these questions/actions belong to, NOT the + // currently-active chat. Writing activeChatId here was the original bug: + // when activeChatId flipped to B but activeQuestions still had A's data, + // the snapshot stamped {chatId: B, questions: [A's]} and a subsequent + // restore would happily render A's data for B. useEffect(() => { try { sessionStorage.setItem('rf-tasklane-meta', JSON.stringify({ show: showTaskLane, - chatId: activeChatId, + chatId: taskLaneOwnerChatId, questions: activeQuestions, actions: activeActions, })) } catch { /* ignore */ } - }, [showTaskLane, activeChatId, activeQuestions, activeActions]) + }, [showTaskLane, taskLaneOwnerChatId, activeQuestions, activeActions]) // Auto-scroll useEffect(() => { @@ -575,6 +608,7 @@ export default function AssistantChatPage() { setShowTaskLane(false) setActiveQuestions([]) setActiveActions([]) + setTaskLaneOwnerChatId(null) setFacts([]) setActiveFix(null) setPreviewKind(null) @@ -615,7 +649,12 @@ export default function AssistantChatPage() { // Auto-open the task lane when the session has facts so the engineer // can see them — without this, a session with only facts (no open // questions) would hide the lane and the facts would be invisible. - if (list.length > 0) setShowTaskLane(true) + // Tag ownership too so the lane render gate accepts it as belonging + // to the active chat (the gate is `taskLaneOwnerChatId === activeChatId`). + if (list.length > 0) { + setShowTaskLane(true) + setTaskLaneOwnerChatId(chatId) + } } catch { // Best-effort — facts are accessory state. Surfacing a toast on every // refetch failure would be noisy; the empty state explains the absence. @@ -788,7 +827,10 @@ export default function AssistantChatPage() { // TemplateMatchPanel is mounted inside TaskLane.bottomSlot, so the // lane must be visible for the panel to render. On fresh sessions // (no questions/facts) the lane defaults closed, so we open it here. + // Tag ownership to the current active chat so the lane render gate + // (taskLaneOwnerChatId === activeChatId) accepts it. setShowTaskLane(true) + if (activeChatId) setTaskLaneOwnerChatId(activeChatId) setScriptPanelOpen(true) return } @@ -1055,6 +1097,7 @@ export default function AssistantChatPage() { setActiveQuestions(q) setActiveActions(a) setShowTaskLane(true) + setTaskLaneOwnerChatId(chatId) } } } catch { @@ -1158,6 +1201,7 @@ export default function AssistantChatPage() { setActiveQuestions(response.questions || []) setActiveActions(response.actions || []) setShowTaskLane(true) + setTaskLaneOwnerChatId(sentForChatId) } // Phase 8: increment post-apply message counter for nudge logic. // Only increments when fix is still in 'proposed' (verifying) state — @@ -1238,11 +1282,13 @@ export default function AssistantChatPage() { setActiveQuestions(response.questions || []) setActiveActions(response.actions || []) setShowTaskLane(true) + setTaskLaneOwnerChatId(sentForChatId) } else { // AI sent no new tasks — clear the lane setShowTaskLane(false) setActiveQuestions([]) setActiveActions([]) + setTaskLaneOwnerChatId(null) } // Phase 8: increment post-apply message counter for nudge logic (mirrors handleSend). // Only increments in 'proposed' (verifying) state — same rationale as handleSend. @@ -1337,6 +1383,7 @@ export default function AssistantChatPage() { setActiveQuestions(response.questions || []) setActiveActions(response.actions || []) setShowTaskLane(true) + setTaskLaneOwnerChatId(session.session_id) } // Refetch facts + active fix — resume turn may emit markers. refreshSessionDerived(session.session_id) @@ -1960,7 +2007,7 @@ export default function AssistantChatPage() { Paste Logs )} - {!showTaskLane && (activeQuestions.length > 0 || activeActions.length > 0) && ( + {!showTaskLane && taskLaneIsForActiveChat && (activeQuestions.length > 0 || activeActions.length > 0) && (