From 9395e5ecf70febbc2163041beed16241ceb0dd6a Mon Sep 17 00:00:00 2001 From: chihlasm Date: Fri, 20 Mar 2026 02:45:52 +0000 Subject: [PATCH] docs: add Search & Recall + Evidence-Rich Sessions design document Evidence: Railway Object Storage, file_uploads model, clipboard paste UX. Search: structured filters, PostgreSQL FTS on sessions, semantic similar matching. Co-Authored-By: Claude Opus 4.6 (1M context) --- ...026-03-20-search-recall-evidence-design.md | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/plans/2026-03-20-search-recall-evidence-design.md diff --git a/docs/plans/2026-03-20-search-recall-evidence-design.md b/docs/plans/2026-03-20-search-recall-evidence-design.md new file mode 100644 index 00000000..db36d885 --- /dev/null +++ b/docs/plans/2026-03-20-search-recall-evidence-design.md @@ -0,0 +1,151 @@ +# Search & Recall + Evidence-Rich Sessions — Design Document + +> **Date:** 2026-03-20 +> **Status:** Approved +> **Source:** Stack priorities plan items #1 (Search and recall improvements) and #2 (Evidence-rich sessions) + +--- + +## Overview + +Two complementary features that make ResolutionFlow a team memory system, not just a flow runner. Engineers can capture proof (screenshots, logs, command output) during troubleshooting via clipboard paste, and search across all past sessions by content, domain, ticket, or semantic similarity. + +--- + +## Feature 1: Evidence-Rich Sessions + +### Storage + +- **Railway Object Storage** — S3-compatible bucket provisioned via Railway dashboard/CLI +- Backend uses `boto3` with S3-compatible endpoint configured via env vars: + - `STORAGE_ENDPOINT` — Railway bucket endpoint + - `STORAGE_ACCESS_KEY` — bucket access key + - `STORAGE_SECRET_KEY` — bucket secret key + - `STORAGE_BUCKET_NAME` — bucket name + - `STORAGE_REGION` — region (default: us-east-1) + +### Data Model + +New `file_uploads` table: + +| Column | Type | Notes | +|--------|------|-------| +| id | UUID PK | | +| account_id | UUID FK → accounts | Tenant scope | +| uploaded_by | UUID FK → users | Who uploaded | +| session_id | UUID FK → ai_sessions (nullable) | Linked session | +| filename | String(255) | Original filename | +| content_type | String(100) | MIME type | +| size_bytes | Integer | File size | +| storage_key | String(500) | S3 object key | +| created_at | DateTime(tz) | | + +### API + +``` +POST /uploads — Multipart file upload, returns upload record with presigned URL +GET /uploads/{id}/url — Presigned download URL (time-limited, 1 hour) +GET /uploads?session_id={id} — List uploads for a session +DELETE /uploads/{id} — Delete upload + S3 object +``` + +### Limits + +- Image types: PNG, JPEG, GIF, WebP — max 5MB each +- Text types: .txt, .log, .csv — max 1MB each +- Per-session: 20 files, 50MB total +- Rate limit: 10 uploads/minute per user + +### Clipboard Paste UX + +A `RichTextInput` component that wraps textareas with clipboard paste support: + +- Listens for `paste` event on the textarea +- Detects image blobs in `clipboardData.items` +- On image paste: uploads in background via `POST /uploads`, shows inline thumbnail with progress +- Thumbnail states: uploading (spinner overlay) → success (image preview) → error (retry button) +- Text content and image references stored together in JSONB +- Images rendered as small thumbnails below the textarea, removable with X button + +**Where it's used:** +- FlowPilot intake textarea +- Free-text response input (escape hatch) +- Escalation reason textarea +- Session scratchpad + +### Evidence in Exports + +Extend the existing export service: +- Markdown: images as `![filename](presigned_url)` links +- HTML/PDF: images embedded as `` with presigned URLs +- PSA: image URLs listed as references (ConnectWise notes don't support inline images) + +--- + +## Feature 2: Search & Recall + +### Layer 1: Structured Filters + +Extend `GET /ai-sessions` with query parameters: + +| Param | Type | Filter | +|-------|------|--------| +| problem_domain | string | Exact match on domain | +| matched_flow_id | UUID | Sessions that matched a specific flow | +| confidence_tier | string | guided / exploring / discovery | +| ticket_id | string | PSA ticket ID | +| date_from | datetime | Sessions created after | +| date_to | datetime | Sessions created before | +| q | string | Full-text search (Layer 2) | + +Frontend: add filter bar to AI Sessions tab on SessionHistoryPage — domain dropdown, confidence pills, date range, search input. Match the existing Flow Sessions filter pattern. + +### Layer 2: Content Search (PostgreSQL FTS) + +Add generated `tsvector` column to `ai_sessions`: + +```sql +ALTER TABLE ai_sessions ADD COLUMN search_vector tsvector + GENERATED ALWAYS AS ( + to_tsvector('english', + coalesce(intake_summary, '') || ' ' || + coalesce(resolution_summary, '') || ' ' || + coalesce(escalation_reason, '') || ' ' || + coalesce(problem_domain, '')) + ) STORED; + +CREATE INDEX idx_ai_sessions_search ON ai_sessions USING gin(search_vector); +``` + +The `q` parameter uses `plainto_tsquery('english', q)` against this vector. Same pattern as existing tree FTS search. + +Extend Command Palette to search AI sessions alongside flows. + +### Layer 3: Similar Session Matching + +New endpoint: `GET /ai-sessions/{id}/similar?limit=5` + +- Generates embedding for the session's intake_summary using existing Voyage AI integration +- New `ai_session_embeddings` table (same pattern as `tree_embeddings`): `session_id`, `embedding` (pgvector) +- Embeddings generated on session creation (after intake) and updated on resolution +- Cosine similarity query against all session embeddings for the account +- Returns top N matches with similarity score and session summary + +**Where similar sessions appear:** +- FlowPilot session sidebar: "Similar Past Sessions" section (3-5 matches) +- Session detail page: "Related Sessions" at bottom +- Both show: session name/summary, resolution status, date, similarity % + +--- + +## Implementation Order + +1. **File upload infrastructure** — S3 service, file_uploads model, upload/download endpoints +2. **RichTextInput component** — clipboard paste handler, thumbnail rendering, upload integration +3. **Wire into FlowPilot** — intake, free-text, escalation, scratchpad +4. **Evidence in exports** — extend export service with image references +5. **Structured filters** — extend AI session list endpoint + frontend filter bar +6. **Content search (FTS)** — migration for tsvector + GIN index, wire into list endpoint +7. **Command palette session search** — extend CommandPalette with AI session results +8. **Similar session matching** — embeddings table, generation service, similar endpoint +9. **Similar sessions UI** — sidebar section + session detail section