1  # Phase 3 Implementation Plan: Search Integration & LLM Agent Tools
  2  
  3  **Target Start:** January 27, 2025  
  4  **Estimated Duration:** 1 week  
  5  **Dependencies:** Phase 2 (Vector Embeddings) - ✅ Complete
  6  
  7  ## Overview
  8  
  9  Phase 3 focuses on making the vector embeddings useful by implementing semantic search capabilities and integrating them with LLM agents. This bridges the gap between having embeddings and actually leveraging them for AI-powered features.
 10  
 11  ## Phase 3 Goals
 12  
 13  ### Primary Objectives
 14  1. **Vector Search API**: Implement similarity search with filtering capabilities
 15  2. **LLM Agent Integration**: Add vector search tools to existing agents
 16  3. **Discord Commands**: User-facing search commands via slash commands
 17  4. **Hybrid Search**: Combine vector similarity with metadata filtering
 18  
 19  ### Success Criteria
 20  - Sub-100ms search response times for typical queries
 21  - Relevant results (subjective evaluation during testing)
 22  - Seamless integration with existing agents (no breaking changes)
 23  - Discord users can search message history semantically
 24  
 25  ## Implementation Breakdown
 26  
 27  ### 3.1 Vector Search Engine (2-3 days)
 28  
 29  #### Core Search Functions
 30  ```typescript
 31  // services/vector-search-engine.ts
 32  export class VectorSearchEngine {
 33    // Basic similarity search
 34    async searchSimilar(
 35      query: string, 
 36      options: SearchOptions
 37    ): Promise<SearchResult[]>
 38    
 39    // Hybrid search (vector + metadata)
 40    async hybridSearch(
 41      query: string,
 42      filters: MetadataFilters,
 43      options: SearchOptions
 44    ): Promise<SearchResult[]>
 45    
 46    // Multi-query search
 47    async searchMultiple(
 48      queries: string[],
 49      options: SearchOptions
 50    ): Promise<SearchResultSet[]>
 51  }
 52  ```
 53  
 54  #### Search Options & Filtering
 55  ```typescript
 56  interface SearchOptions {
 57    limit?: number;              // Max results (default: 10)
 58    threshold?: number;          // Similarity threshold (0.0-1.0)
 59    guildIds?: string[];         // Guild filtering (from agent config)
 60    channelId?: string;          // Specific channel
 61    authorId?: string;           // Specific author
 62    timeRange?: TimeRange;       // Time-based filtering
 63    hasAttachments?: boolean;    // Filter by attachment presence
 64    messageType?: MessageType;   // Filter by message type
 65  }
 66  
 67  interface TimeRange {
 68    after?: string;   // ISO timestamp
 69    before?: string;  // ISO timestamp
 70    preset?: 'last_hour' | 'last_day' | 'last_week' | 'last_month';
 71  }
 72  ```
 73  
 74  #### Result Ranking & Scoring
 75  ```typescript
 76  interface SearchResult {
 77    // Vector similarity
 78    similarity: number;          // 0.0-1.0 similarity score
 79    
 80    // Source data
 81    messageId: string;
 82    content: string;
 83    
 84    // Metadata
 85    guildId: string;
 86    channelId: string;
 87    authorId: string;
 88    timestamp: string;
 89    
 90    // Enhanced scoring
 91    relevanceScore: number;      // Combined similarity + metadata score
 92    contextSnippet: string;      // Highlighted/truncated content
 93  }
 94  ```
 95  
 96  ### 3.2 LLM Agent Tool Integration (1-2 days)
 97  
 98  #### Vector Search Tool
 99  ```typescript
100  // tools/vector-search-tool.ts
101  export const VectorSearchTool = {
102    name: "vector_search",
103    description: "Search for similar messages using semantic similarity",
104    parameters: {
105      query: { 
106        type: "string", 
107        required: true,
108        description: "Search query or topic to find similar messages about"
109      },
110      limit: { 
111        type: "number", 
112        default: 10,
113        description: "Maximum number of results to return"
114      },
115      channel_id: { 
116        type: "string", 
117        required: false,
118        description: "Limit search to specific channel"
119      },
120      time_range: { 
121        type: "string", 
122        required: false,
123        description: "Time range: 'last_hour', 'last_day', 'last_week', 'last_month'"
124      },
125      author_id: {
126        type: "string",
127        required: false,
128        description: "Limit search to messages by specific user"
129      }
130    },
131    
132    async execute(params: ToolParams, agentConfig: AgentConfig): Promise<ToolResult> {
133      const searchEngine = getVectorSearchEngine();
134      const guildIds = agentConfig.guildsToIndex || [];
135      
136      const results = await searchEngine.hybridSearch(params.query, {
137        guildIds,
138        channelId: params.channel_id,
139        authorId: params.author_id,
140        timeRange: params.time_range ? { preset: params.time_range } : undefined,
141        limit: params.limit
142      });
143      
144      return {
145        results: results.map(r => ({
146          content: r.contextSnippet,
147          similarity: r.similarity,
148          channel: `<#${r.channelId}>`,
149          author: `<@${r.authorId}>`,
150          timestamp: r.timestamp,
151          messageLink: `https://discord.com/channels/${r.guildId}/${r.channelId}/${r.messageId}`
152        })),
153        summary: `Found ${results.length} similar messages`
154      };
155    }
156  };
157  ```
158  
159  #### Related Messages Tool
160  ```typescript
161  export const RelatedMessagesTool = {
162    name: "find_related_messages",
163    description: "Find messages related to a specific message or topic",
164    parameters: {
165      message_id: { 
166        type: "string", 
167        required: false,
168        description: "Find messages similar to this specific message"
169      },
170      topic: {
171        type: "string",
172        required: false, 
173        description: "Find messages related to this topic"
174      },
175      exclude_author: {
176        type: "boolean",
177        default: false,
178        description: "Exclude messages from the same author"
179      }
180    },
181    
182    async execute(params: ToolParams, agentConfig: AgentConfig): Promise<ToolResult> {
183      // Implementation for finding related messages
184    }
185  };
186  ```
187  
188  ### 3.3 Discord Slash Commands Integration (1 day)
189  
190  #### New Search Commands
191  ```typescript
192  // In discord-slash-commands.ts, add:
193  
194  // /search-similar command
195  {
196    name: 'search-similar',
197    description: 'Search for messages similar to your query using AI',
198    options: [
199      {
200        name: 'query',
201        description: 'What to search for (topic, concept, or question)',
202        type: ApplicationCommandOptionType.String,
203        required: true,
204      },
205      {
206        name: 'limit',
207        description: 'Number of results (1-20)',
208        type: ApplicationCommandOptionType.Integer,
209        required: false,
210        min_value: 1,
211        max_value: 20,
212      },
213      {
214        name: 'channel',
215        description: 'Limit search to specific channel',
216        type: ApplicationCommandOptionType.Channel,
217        required: false,
218      },
219      {
220        name: 'timeframe',
221        description: 'Limit search to timeframe',
222        type: ApplicationCommandOptionType.String,
223        required: false,
224        choices: [
225          { name: 'Last hour', value: 'last_hour' },
226          { name: 'Last day', value: 'last_day' },
227          { name: 'Last week', value: 'last_week' },
228          { name: 'Last month', value: 'last_month' },
229        ],
230      },
231    ],
232  },
233  
234  // /find-related command  
235  {
236    name: 'find-related',
237    description: 'Find messages related to a specific message',
238    options: [
239      {
240        name: 'message_link',
241        description: 'Discord message link to find related messages for',
242        type: ApplicationCommandOptionType.String,
243        required: true,
244      },
245      {
246        name: 'limit',
247        description: 'Number of results (1-20)', 
248        type: ApplicationCommandOptionType.Integer,
249        required: false,
250      },
251    ],
252  },
253  ```
254  
255  #### Command Handlers
256  ```typescript
257  async function handleSearchSimilar(interaction: ChatInputCommandInteraction) {
258    const query = interaction.options.getString('query', true);
259    const limit = interaction.options.getInteger('limit') ?? 10;
260    const channel = interaction.options.getChannel('channel');
261    const timeframe = interaction.options.getString('timeframe');
262    
263    await interaction.deferReply({ ephemeral: true });
264    
265    try {
266      const searchEngine = getVectorSearchEngine();
267      const results = await searchEngine.hybridSearch(query, {
268        guildIds: [interaction.guildId!],
269        channelId: channel?.id,
270        timeRange: timeframe ? { preset: timeframe } : undefined,
271        limit
272      });
273      
274      if (results.length === 0) {
275        await interaction.editReply({
276          content: `No similar messages found for: "${query}"`
277        });
278        return;
279      }
280      
281      const embed = new EmbedBuilder()
282        .setTitle(`🔍 Similar Messages: "${query}"`)
283        .setColor(0x00AE86)
284        .setDescription(`Found ${results.length} similar messages`)
285        .addFields(
286          results.slice(0, 5).map((result, i) => ({
287            name: `${i + 1}. ${result.similarity.toFixed(2)} similarity`,
288            value: `${result.contextSnippet}\n[Jump to message](${result.messageLink})`,
289            inline: false
290          }))
291        );
292        
293      await interaction.editReply({ embeds: [embed] });
294      
295    } catch (error) {
296      console.error('Error in search-similar command:', error);
297      await interaction.editReply({
298        content: 'An error occurred while searching. Please try again.'
299      });
300    }
301  }
302  ```
303  
304  ### 3.4 Enhanced LLM Agent Integration (1-2 days)
305  
306  #### LLM Response Agent Enhancement
307  ```typescript
308  // In llm-response-agent.ts, add context search:
309  
310  async function generateResponseWithContext(prompt: string, agentConfig: AgentConfig) {
311    // Search for relevant context
312    const searchEngine = getVectorSearchEngine();
313    const contextResults = await searchEngine.hybridSearch(prompt, {
314      guildIds: agentConfig.guildsToIndex,
315      limit: 5,
316      timeRange: { preset: 'last_month' }
317    });
318    
319    // Build context for LLM
320    const context = contextResults.map(r => 
321      `[${r.timestamp}] ${r.content}`
322    ).join('\n');
323    
324    const enhancedPrompt = `
325  Context from recent messages:
326  ${context}
327  
328  User query: ${prompt}
329  
330  Please provide a helpful response based on the context and your knowledge.
331  `;
332    
333    return await generateLLMResponse(enhancedPrompt, agentConfig);
334  }
335  ```
336  
337  #### LM Studio Insight Agent Enhancement  
338  ```typescript
339  // In lm-studio-insight-agent.ts, add pattern discovery:
340  
341  async function findInsightPatterns(messages: DiscordMessageData[], agentConfig: AgentConfig) {
342    const searchEngine = getVectorSearchEngine();
343    
344    // For each message, find similar historical messages
345    const patterns = [];
346    for (const message of messages) {
347      const similar = await searchEngine.searchSimilar(message.content, {
348        guildIds: agentConfig.guildsToIndex,
349        limit: 3,
350        timeRange: { preset: 'last_month' }
351      });
352      
353      if (similar.length > 1) {
354        patterns.push({
355          theme: extractTheme(message.content),
356          examples: similar,
357          frequency: similar.length
358        });
359      }
360    }
361    
362    return patterns;
363  }
364  ```
365  
366  ## Implementation Timeline
367  
368  ### Week 1: Core Search Implementation
369  - **Day 1-2**: Vector search engine core functionality
370  - **Day 3**: Search result ranking and filtering
371  - **Day 4**: LLM agent tool development  
372  - **Day 5**: Discord slash commands integration
373  - **Day 6-7**: Testing, optimization, and bug fixes
374  
375  ### Milestone Checkpoints
376  - **Day 2**: Basic vector search working with manual testing
377  - **Day 4**: LLM agents can use vector search tools
378  - **Day 6**: Discord users can use `/search-similar` command
379  - **Day 7**: Performance optimized, error handling complete
380  
381  ## Testing Strategy
382  
383  ### Unit Testing
384  - Vector search accuracy with known datasets
385  - Filter combinations and edge cases
386  - Performance benchmarks (search speed)
387  - Error handling (malformed queries, empty results)
388  
389  ### Integration Testing  
390  - End-to-end search from Discord commands
391  - LLM agent tool integration
392  - Cross-guild search filtering
393  - Time-based filtering accuracy
394  
395  ### User Testing
396  - Subjective relevance evaluation
397  - Search experience usability
398  - Command interface intuitiveness
399  - Performance perception
400  
401  ## Risk Mitigation
402  
403  ### Performance Risks
404  - **Risk**: Slow search responses (>1s)
405  - **Mitigation**: Implement query optimization, result caching, pagination
406  
407  ### Relevance Risks  
408  - **Risk**: Poor search results quality
409  - **Mitigation**: Implement result ranking, similarity thresholds, user feedback
410  
411  ### Integration Risks
412  - **Risk**: Breaking existing agent functionality
413  - **Mitigation**: Backward compatibility, optional tool usage, comprehensive testing
414  
415  ## Success Metrics
416  
417  ### Performance Targets
418  - **Search Speed**: <100ms average response time
419  - **Relevance**: >80% user satisfaction in manual testing
420  - **Availability**: >99% uptime for search functionality
421  
422  ### Feature Completeness
423  - All planned search tools implemented and tested
424  - Discord commands working reliably
425  - LLM agents enhanced with search capabilities
426  - Documentation complete for Phase 4 handoff
427  
428  ## Phase 4 Preparation
429  
430  ### Data Needed for Phase 4
431  - Search usage analytics and patterns
432  - Performance bottlenecks identified
433  - User feedback on search relevance
434  - Technical debt and optimization opportunities
435  
436  ### Architecture Extensions
437  - Search result caching infrastructure
438  - Cross-entity search foundations
439  - Advanced filtering capability frameworks
440  - Vector store monitoring and health checks
441  
442  **Phase 3 sets the foundation for advanced AI capabilities in Phase 4 by making vector embeddings accessible and useful to both agents and users.**