# Solutions Library + Smart RAG + Community Knowledge — Design Spec

> **Status:** SPEC ONLY — not implementing yet. Build after colleague pilot (Week 3-4).
> **Date:** 2026-03-23 (updated 2026-03-23 — added Community tier)
> **Context:** [GTM validation plan](resolutionflow-gtm-design.md) — copilot-first, team knowledge flywheel

---

## Problem

Engineers solve the same problems repeatedly across an MSP. Today that knowledge lives in engineers' heads, scattered PSA ticket notes, or nowhere. When an engineer resolves a tricky issue through the FlowPilot copilot, that knowledge dies with the session. The next engineer who hits the same issue starts from scratch.

## Solution

**Solutions Library** — a team knowledge base that builds itself from resolved copilot sessions and feeds back into future sessions via RAG.

Two halves:
1. **Capture & Dedup** — save resolutions from copilot sessions, prevent duplicates
2. **Smart RAG** — FlowPilot pulls from the Solutions Library during live sessions and surfaces relevant prior resolutions

## How It Works

### 1. Resolution Capture (post-session)

When an engineer resolves a copilot session, FlowPilot auto-generates a structured resolution:

```
{
  "title": "Exchange Online mailbox not receiving email",
  "problem": "User reports not receiving emails in Outlook. OWA also shows no new mail.",
  "root_cause": "Mail flow rule blocking external senders due to tenant-wide transport rule misconfiguration",
  "resolution_steps": [
    "Checked MX records — correct",
    "Ran message trace in Exchange Admin Center — messages queued",
    "Found transport rule 'Block External' was enabled tenant-wide instead of per-group",
    "Disabled rule, emails delivered within 5 minutes"
  ],
  "environment_tags": ["exchange-online", "mail-flow", "transport-rules"],
  "auto_detected_category": "Microsoft 365"
}
```

Engineer gets prompted: **"Save this as a reusable solution?"**
- One-click save with auto-generated content
- Can edit title, tags, steps before saving
- Can skip (not every session produces reusable knowledge)

### 2. Dedup Check (on save)

Before saving, system does a similarity search (embedding cosine similarity) against existing solutions.

**If similarity > 0.85 (strong match):**
- Show existing solution side-by-side with new one
- Three options:
  - **Merge** — update existing solution with new context/steps (keeps the better version, increments usage count)
  - **Keep Both** — they look similar but are actually different problems
  - **Discard** — it's the same thing, don't save

**If similarity 0.6-0.85 (partial match):**
- Show as "Related solutions" but save as new by default
- Engineer can choose to merge if they recognize it's the same

**If similarity < 0.6:**
- Save directly, no prompt

### 3. RAG During Live Sessions

When an engineer starts or progresses through a copilot conversation:

1. After the first 2-3 message exchanges (enough context to understand the problem), FlowPilot searches the Solutions Library
2. Uses the conversation context as the query (not just the initial message)
3. If a solution scores above threshold, FlowPilot surfaces it naturally:

> *"I found a similar issue. Sarah resolved an Exchange mail flow problem 3 days ago — she found a transport rule was blocking external senders. Want me to walk you through her resolution?"*

**If engineer says yes:**
- FlowPilot presents the resolution steps one at a time
- Engineer confirms each step worked or skips
- At the end, the solution's usage count increments

**If engineer says no:**
- FlowPilot continues open-ended troubleshooting
- The suggestion is noted (helps tune future relevance)

**Retrieval rules:**
- Only surface solutions from the same team
- Max 1 suggestion per session (don't nag)
- Don't suggest solutions the same engineer saved (they already know)
- Prefer recent solutions over old ones (tie-breaker)

### 4. Confidence Scoring

Each solution gets a confidence score (0-100):

| Event | Score change |
|-------|-------------|
| Saved from resolved session | +50 (base) |
| Another engineer uses it successfully | +15 |
| Engineer accepts RAG suggestion | +10 |
| Engineer rejects RAG suggestion | -5 |
| Multiple engineers save similar (merged) | +20 |
| Not suggested in 90 days | -10 (decay) |

High-confidence solutions are suggested more aggressively. Low-confidence solutions still appear in search but aren't proactively surfaced.

### 5. Solutions Library UI

Replaces the current Step Library page. Card-based grid with:

**Each solution card shows:**
- Title (e.g., "Exchange Online mailbox not receiving email")
- Problem summary (2 lines, truncated)
- Root cause (1 line)
- Tags (environment, category)
- Saved by [engineer name] · [date]
- Used [N] times · Confidence [high/medium/low]

**Page features:**
- Search (full-text + semantic)
- Filter by tag, engineer, confidence, recency
- Sort by most used, most recent, highest confidence
- Manual "Add Solution" button (not just from sessions)
- Edit/delete for solutions you created (team admins can edit any)

---

## Data Model

### `solutions` table

| Column | Type | Notes |
|--------|------|-------|
| id | UUID | PK |
| team_id | UUID | FK to teams |
| created_by | UUID | FK to users |
| title | VARCHAR(255) | |
| problem_description | TEXT | What the user reported |
| root_cause | TEXT | What was actually wrong |
| resolution_steps | JSONB | Array of step strings |
| environment_tags | JSONB | Array of tag strings |
| category | VARCHAR(100) | Auto-detected or manual |
| source_session_id | UUID | FK to ai_sessions (nullable — manual entries have no source) |
| embedding | VECTOR(1536) | For similarity search (pgvector) |
| confidence_score | INTEGER | 0-100, default 50 |
| use_count | INTEGER | Times used via RAG suggestion |
| last_used_at | TIMESTAMPTZ | |
| created_at | TIMESTAMPTZ | |
| updated_at | TIMESTAMPTZ | |

### `solution_events` table (for confidence scoring)

| Column | Type | Notes |
|--------|------|-------|
| id | UUID | PK |
| solution_id | UUID | FK to solutions |
| event_type | VARCHAR(30) | 'used', 'accepted', 'rejected', 'merged', 'decayed' |
| user_id | UUID | FK to users (nullable for decay events) |
| session_id | UUID | FK to ai_sessions (nullable) |
| created_at | TIMESTAMPTZ | |

---

## Existing Infrastructure to Reuse

| What exists | Where | How it maps |
|-------------|-------|-------------|
| Knowledge Flywheel | `services/knowledge_flywheel.py` | Session analysis → can generate solution drafts |
| Knowledge Gap Service | `services/knowledge_gap_service.py` | Detects weak options → can flag sessions worth saving |
| RAG in assistant chat | `services/ai_chat_service.py` | Already does retrieval — extend to Solutions Library |
| Step Library UI | `components/step-library/` | Restyle as Solutions Library |
| pgvector | Already in Docker image (`pgvector/pgvector:pg16`) | Embedding storage + similarity search |
| FlowPilot session conclusion | `components/flowpilot/` | Add "Save as Solution" prompt |

---

## Implementation Phases (future)

### Phase 1: Capture & Library
- Solutions table + migrations
- Post-session "Save as Solution" prompt in FlowPilot
- Auto-generate resolution summary from session transcript
- Solutions Library page (replaces Step Library)
- Manual add/edit/delete

### Phase 2: Dedup
- Embedding generation on save (Anthropic or OpenAI embeddings)
- Similarity search on save
- Merge/Keep Both/Discard UI

### Phase 3: Smart RAG
- Mid-session similarity search
- Natural language suggestion in FlowPilot conversation
- Accept/reject tracking
- Confidence scoring + decay job

### Phase 4: Team Intelligence
- "Trending solutions" on dashboard
- "Your team resolved 12 Exchange issues this week" insights
- Solution suggestions in the copilot intake ("Common issues today: VPN, Exchange, AD lockouts")

---

## Open Questions (to answer during pilot)

1. Do engineers actually want to save resolutions, or is it friction?
2. How similar do problems need to be before a suggestion is helpful vs. annoying?
3. Should solutions be editable by the whole team, or only the creator + admins?
4. What's the right moment to prompt "Save as Solution" — right after resolution, or in a follow-up?
5. Do engineers trust AI-generated resolution summaries, or do they want to write their own?

These questions should be answerable after 2-4 weeks of pilot usage.

---

## Community Solutions Library

### Overview

The Solutions Library has three tiers of knowledge:

| Tier | Source | Who sees it |
| ---- | ------ | ----------- |
| **Private** | Your team's resolutions | Your team only |
| **Community** | Anonymized resolutions from all paid users | All paid users |
| **None** | — | Free users (upgrade CTA) |

This creates a network effect moat — every paid user who resolves a ticket makes the product better for all paid users. Solo MSP engineers (who don't have a team to build a knowledge base) get collective wisdom from day one.

### Sharing Permissions

- Any engineer can share their own resolutions to Community (opt-in per resolution)
- Team admins can **retract** any community solution shared by their team
- Team admins can **edit** any community solution from their team (in case something slips through AI sanitization)
- Team admins can toggle community sharing on/off at the **account level** (enabled by default)
- When sharing is disabled, the "Share to Community" button disappears for all team members; existing community solutions from that team remain live but no new ones can be added

### AI Sanitization Pipeline

When an engineer clicks "Share to Community," the resolution goes through an AI sanitization pass before publishing.

**What gets stripped/replaced:**

| Data type | Example | Becomes |
| --------- | ------- | ------- |
| Company names | "Contoso Corp" | `[Company]` |
| Server/host names | "DC01.contoso.local" | `[Domain Controller]` |
| IP addresses | "192.168.1.50" | `[Internal IP]` |
| Usernames | "jsmith@contoso.com" | `[User]` |
| Passwords/secrets | "P@ssw0rd123" | `[REDACTED]` |
| Ticket IDs | "TKT-2847" | `[Ticket]` |
| Client names | "Acme Medical Group" | `[Client]` |
| File paths with org info | `\\contoso-fs01\shares` | `[File Server]\shares` |

**Sanitization pipeline:**

1. **LLM pass** — sanitization prompt: "Replace all identifying information with descriptive placeholders while preserving technical accuracy"
2. **Regex pass** — safety net to catch IP patterns, email formats, UNC paths the LLM might miss
3. **Preview** — show the engineer the sanitized version before publishing; they can edit or cancel
4. **Low-confidence highlights** — if the LLM can't tell whether something is a product name or company name (e.g., "Apollo"), it highlights those sections in yellow for the engineer to confirm

The engineer always approves the sanitized preview before anything goes live.

### AI Confidence Assessment

When a sanitized solution is submitted, AI runs a quality assessment before publishing.

**AI Confidence Score (0-100):**

| Signal | Score impact |
| ------ | ----------- |
| Session ended in "resolved" status | Required (gate — only resolved sessions can be shared) |
| Clear root cause identified | +25 |
| Specific, actionable resolution steps | +20 |
| Problem is generalizable (not hyper-specific to one environment) | +15 |
| Resolution steps are reproducible by another engineer | +15 |
| Session had multiple troubleshooting steps (not a one-liner) | +10 |
| Similar solution already exists in community | -15 |
| Resolution is vague ("restarted and it worked") | -20 |

**Publishing thresholds (admin-configurable):**

- **Score 70+** — Auto-published, visible immediately
- **Score 40-69** — Published with "Unverified" badge, needs community votes to graduate
- **Score < 40** — Rejected with feedback ("This resolution is too vague to be useful. Try adding the specific steps you took.")

### Community Voting

- Any paid user can upvote or downvote a community solution
- Upvote = "This helped me" / Downvote = "This didn't work for me"
- **5 net upvotes** → "Unverified" badge removed, becomes "Community Verified"
- **3 net downvotes** → Solution hidden, flagged for staff review
- Users can leave a comment with their vote — comments go to a ResolutionFlow staff moderation queue

**Ongoing confidence adjustment:**

| Event | Score change |
| ----- | ----------- |
| Community upvote | +5 |
| Community downvote | -8 |
| Used successfully via RAG (community) | +10 |
| Rejected via RAG (community) | -3 |
| Staff review confirms quality | +20 |
| Not used in 120 days | -5 (decay) |

### Admin Controls (Super Admin Panel)

A new "Community" tab in the `/admin` panel with configurable thresholds:

| Setting | Default | Description |
| ------- | ------- | ----------- |
| Auto-publish threshold | 70 | Min AI confidence score to publish immediately |
| Review threshold | 40 | Min score to publish with "Unverified" badge |
| Upvotes to verify | 5 | Net upvotes needed to remove "Unverified" |
| Downvotes to hide | 3 | Net downvotes to auto-hide for review |
| Confidence decay days | 120 | Days of inactivity before score decays |
| Confidence decay amount | 5 | Points lost per decay cycle |
| Community sharing enabled | true | Global kill switch for all community features |

These live in the existing `platform_settings` table (JSONB). The admin panel also shows the **moderation queue** — flagged solutions and user comments requiring staff review.

### RAG Priority (Community Extension)

During live FlowPilot sessions, retrieval works in two passes:

1. **Team pass** — search team's private Solutions Library (existing behavior)
2. **Community pass** — search community solutions (only if team has `community_sharing_enabled` and user is on a paid plan)

**How FlowPilot presents community solutions differently:**

- Team solutions: *"Your colleague Sarah resolved a similar Exchange issue 3 days ago..."*
- Community solutions: *"A community solution with 12 upvotes matches your issue — an engineer found that a transport rule was blocking external senders. Want me to walk you through it?"*

**Community RAG rules:**

- Only surface community solutions with `community_status = 'published'` (not `'unverified'` or `'hidden'`)
- Require AI confidence score 60+ for RAG surfacing
- Team solutions always rank above community solutions at equal similarity
- Still max 1 suggestion per session (team OR community, whichever scores higher)
- If team solution exists at similarity > 0.7, don't bother checking community (team context is better)

### Solutions Library UI (Community Tab)

Add a tab switcher at the top of the existing Solutions Library page:

| Tab | What it shows |
| --- | ------------- |
| **My Team** | Private team solutions (current behavior) |
| **Community** | All published community solutions, searchable/filterable |

Community tab adds:

- Upvote/downvote buttons on each card
- "Community Verified" or "Unverified" badge
- "Used by X engineers" count instead of team member name
- Vote comment modal (optional, sends to staff queue)
- Filter by: confidence, votes, category, recency

### Pricing Gate

| Feature | Free | Paid |
| ------- | ---- | ---- |
| Private Solutions Library (team) | Limited (10 solutions) | Unlimited |
| Save from FlowPilot sessions | Yes | Yes |
| Community browsing | Upgrade CTA | Full access |
| Community RAG in sessions | No | Yes |
| Share to Community | No | Yes |
| Vote on community solutions | No | Yes |
| Reputation badges | No | Yes |

Free users see the "Community" tab but with a blurred preview + upgrade CTA: *"Join 500+ MSP engineers sharing solutions. Upgrade to access the community knowledge base."*

### Reputation & Incentives

**Visible reputation:**

- Profile badge showing contribution tier: **Contributor** (1+ shared) → **Expert** (10+) → **Authority** (50+)
- "Your solutions helped X engineers this month" notification (monthly digest)
- Contributor count visible on user avatar tooltip in the app

**Psychological nudges (v1):**

- **Social proof on share prompt:** After resolving a session, the save prompt says: *"47 engineers solved similar Exchange issues this week using community solutions. Share yours?"*
- **First-share celebration:** Confetti moment + "Welcome to the community!" toast
- **Streak tracking:** "You've shared 3 solutions this week" (consistency bias)
- **Loss aversion (gentle):** "You resolved 8 tickets this month but only shared 2 — 6 solutions that could help others"

**Future incentives (post-v1):**

- Leaderboard on the Community page: top contributors this month (opt-in visibility)
- Early access to new features for top contributors
- "Featured Contributor" badge on profile
- Potential: discount on subscription for consistent contributors

### Community Data Model Extensions

**New columns on `solutions` table:**

| Column | Type | Notes |
| ------ | ---- | ----- |
| visibility | VARCHAR(20) | `'private'`, `'community'` — default `'private'` |
| community_status | VARCHAR(20) | `'published'`, `'unverified'`, `'hidden'`, `'rejected'` — null for private |
| ai_confidence_score | INTEGER | 0-100, from sanitization/quality assessment |
| sanitized_content | JSONB | Anonymized version (title, problem, root_cause, steps) |
| source_solution_id | UUID | FK to solutions — links community copy to private original |
| upvote_count | INTEGER | default 0 |
| downvote_count | INTEGER | default 0 |
| shared_by_team_id | UUID | FK to accounts — which team shared it (for retraction) |
| shared_at | TIMESTAMPTZ | When it was published to community |

**New table: `solution_votes`**

| Column | Type | Notes |
| ------ | ---- | ----- |
| id | UUID | PK |
| solution_id | UUID | FK to solutions (community ones) |
| user_id | UUID | FK to users |
| vote | SMALLINT | +1 or -1 |
| comment | TEXT | Optional — goes to staff review queue |
| created_at | TIMESTAMPTZ | |
| **UNIQUE** | (solution_id, user_id) | One vote per user per solution |

**New table: `community_reports`**

| Column | Type | Notes |
| ------ | ---- | ----- |
| id | UUID | PK |
| solution_id | UUID | FK to solutions |
| reporter_id | UUID | FK to users |
| comment | TEXT | From the vote comment |
| status | VARCHAR(20) | `'pending'`, `'reviewed'`, `'actioned'` |
| reviewed_by | UUID | FK to users (staff) — nullable |
| created_at | TIMESTAMPTZ | |
| reviewed_at | TIMESTAMPTZ | |

**New columns on `accounts` (team settings):**

| Column | Type | Notes |
| ------ | ---- | ----- |
| community_sharing_enabled | BOOLEAN | default `true` — team admin toggle |

**New columns on `users` (reputation):**

| Column | Type | Notes |
| ------ | ---- | ----- |
| community_shares_count | INTEGER | default 0 — denormalized for fast badge lookup |
| community_helped_count | INTEGER | default 0 — how many times their solutions were used |

**Platform settings** (admin-tunable, in existing JSONB):

```json
{
  "community_auto_publish_threshold": 70,
  "community_review_threshold": 40,
  "community_upvotes_to_verify": 5,
  "community_downvotes_to_hide": 3,
  "community_decay_days": 120,
  "community_decay_amount": 5,
  "community_enabled": true
}
```

### Community Implementation Phases

Extends the existing implementation phases:

#### Phase 5: Community Sharing

- `visibility` and `community_status` columns on solutions
- AI sanitization pipeline (LLM + regex + preview)
- "Share to Community" button on solution cards
- Team admin toggle for community sharing
- Team admin retract/edit for community solutions

#### Phase 6: Community Quality & Voting

- AI confidence assessment on community submissions
- `solution_votes` table + upvote/downvote UI
- `community_reports` table + staff moderation queue
- Admin-configurable thresholds in platform settings
- "Community" tab in admin panel

#### Phase 7: Community RAG

- Two-pass RAG (team-first, community-second)
- Community solution presentation in FlowPilot
- Paid-plan gating for community features
- Blurred preview + upgrade CTA for free users

#### Phase 8: Reputation & Growth

- Contribution tier badges (Contributor → Expert → Authority)
- Social proof nudges on share prompt
- First-share celebration
- "Helped X engineers" monthly digest
- Streak tracking

---

## Related: FlowPilot Engagement Nudges (Future Design)

Separate from community sharing — how do we nudge engineers to resolve issues through FlowPilot in the first place? This is the top of the funnel. Community sharing nudges are pointless if engineers aren't using FlowPilot to resolve.

**To design:**

- What triggers bring engineers back to FlowPilot?
- How do we make the resolve action feel rewarding (dopamine loop)?
- Post-resolution moments: celebration, streak tracking, team leaderboard
- Push notifications / email digests for stale sessions
- "Your team resolved 12 tickets today" social proof

Design this as a separate spec — it deserves focused thinking.
