nixos/shared/linked-dotfiles/opencode/llmemory/README.md
2025-10-29 18:46:16 -06:00

306 lines
8.0 KiB
Markdown

# LLMemory - AI Agent Memory System
A persistent memory/journal system for AI agents with grep-like search and fuzzy matching.
## Overview
LLMemory provides AI agents with long-term memory across sessions. Think of it as a personal knowledge base with powerful search capabilities, designed specifically for agent workflows.
**Key Features:**
- 🔍 **Grep-like search** - Familiar query syntax for AI agents
- 🎯 **Fuzzy matching** - Handles typos automatically
- 🏷️ **Tag-based organization** - Easy categorization and filtering
-**Expiration support** - Auto-cleanup of time-sensitive info
- 📊 **Relevance ranking** - Best results first, token-efficient
- 🔌 **OpenCode integration** - Plugin API for seamless workflows
## Status
**Current Phase:** Planning Complete (Phase 0)
**Next Phase:** MVP Implementation (Phase 1)
This project is in the initial planning stage. The architecture and implementation plan are complete, ready for development.
## Quick Start (Future)
```bash
# Installation (when available)
npm install -g llmemory
# Store a memory
memory store "Docker Compose uses bridge networks by default" \
--tags docker,networking
# Search memories
memory search "docker networking"
# List recent memories
memory list --limit 10
# Show agent documentation
memory --agent-context
```
## Documentation
- **[SPECIFICATION.md](./SPECIFICATION.md)** - Complete technical specification
- **[IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md)** - Phased development plan
- **[ARCHITECTURE.md](./docs/ARCHITECTURE.md)** - System design (to be created)
- **[AGENT_GUIDE.md](./docs/AGENT_GUIDE.md)** - Guide for AI agents (to be created)
## Architecture
### Three-Phase Implementation
**Phase 1: MVP (2-3 days)**
- Basic CLI with store/search/list/prune commands
- Simple LIKE-based search
- Tag filtering and expiration handling
- Target: <500 memories, <50ms search
**Phase 2: FTS5 (3-5 days)**
- Migrate to SQLite FTS5 for production search
- BM25 relevance ranking
- Boolean operators (AND/OR/NOT)
- Target: 10K+ memories, <100ms search
**Phase 3: Fuzzy Layer (3-4 days)**
- Trigram indexing for typo tolerance
- Levenshtein distance matching
- Intelligent cascade (exact fuzzy)
- Target: 100K+ memories, <200ms search
### Technology Stack
- **Language:** Node.js (JavaScript/TypeScript)
- **Database:** SQLite with better-sqlite3
- **CLI:** Commander.js
- **Search:** FTS5 + trigram fuzzy matching
- **Testing:** Vitest
## Project Structure
```
llmemory/
├── src/
│ ├── cli.js # CLI entry point
│ ├── commands/ # Command implementations
│ ├── db/ # Database layer
│ ├── search/ # Search strategies (LIKE, FTS5, fuzzy)
│ ├── utils/ # Utilities (validation, formatting)
│ └── extractors/ # Auto-extraction (*Remember* pattern)
├── test/ # Test suite
├── docs/ # Documentation
├── bin/ # Executable wrapper
├── SPECIFICATION.md # Technical spec
├── IMPLEMENTATION_PLAN.md # Development roadmap
└── README.md # This file
```
## Development
### Setup
```bash
cd llmemory
npm install
npm test
```
### Implementation Status
See [IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md) for detailed progress tracking.
**Current Progress:**
- [x] Phase 0: Planning and documentation
- [ ] Phase 1: MVP (Simple LIKE search)
- [ ] Project setup
- [ ] Database layer
- [ ] Store command
- [ ] Search command
- [ ] List command
- [ ] Prune command
- [ ] CLI integration
- [ ] Testing
- [ ] Phase 2: FTS5 migration
- [ ] Phase 3: Fuzzy layer
### Contributing
1. Review [SPECIFICATION.md](./SPECIFICATION.md) for architecture
2. Check [IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md) for next steps
3. Pick an uncompleted task from the current phase
4. Write tests first (TDD approach)
5. Implement feature
6. Update checkboxes in IMPLEMENTATION_PLAN.md
7. Commit with clear message
### Testing
```bash
# Run all tests
npm test
# Run tests in watch mode
npm run test:watch
# Run specific test file
npm test search.test.js
# Coverage report
npm run test:coverage
```
## Usage Examples (Future)
### Storing Memories
```bash
# Basic storage
memory store "PostgreSQL VACUUM FULL locks tables, use VACUUM ANALYZE instead"
# With tags
memory store "Docker healthchecks need curl --fail for proper exit codes" \
--tags docker,best-practices
# With expiration
memory store "Staging server at https://staging.example.com" \
--tags infrastructure,staging \
--expires "2025-12-31"
# From agent
memory store "NixOS flake.lock must be committed for reproducible builds" \
--tags nixos,build-system \
--entered-by investigate-agent
```
### Searching Memories
```bash
# Basic search
memory search "docker"
# Multiple terms (implicit AND)
memory search "docker networking"
# Boolean operators
memory search "docker AND compose"
memory search "docker OR podman"
memory search "database NOT postgresql"
# Phrase search
memory search '"exact phrase"'
# With filters
memory search "kubernetes" --tags production,k8s
memory search "error" --after "2025-10-01"
memory search "config" --entered-by optimize-agent --limit 5
```
### Managing Memories
```bash
# List recent
memory list --limit 20
# List by tag
memory list --tags docker --sort created --order desc
# Show statistics
memory stats
memory stats --tags # Tag frequency
memory stats --agents # Memories per agent
# Prune expired
memory prune --dry-run # Preview
memory prune --force # Execute
# Export/import
memory export backup.json
memory import backup.json
```
## Memory Format Guidelines
### Good Memory Examples
```bash
# Technical detail
memory store "Git worktree: 'git worktree add -b feature ../feature' creates parallel working directory without cloning" --tags git,workflow
# Error resolution
memory store "Node.js ENOSPC: Increase inotify watches with 'echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p'" --tags nodejs,linux,troubleshooting
# Configuration pattern
memory store "Nginx reverse proxy: Set 'proxy_set_header X-Real-IP \$remote_addr' to preserve client IP through proxy chain" --tags nginx,networking
```
### Anti-Patterns
```bash
# Too vague ❌
memory store "Fixed the bug"
# Better ✅
memory store "Fixed React infinite render loop by adding missing dependencies to useEffect array"
# Widely known ❌
memory store "Docker is a containerization platform"
# Specific insight ✅
memory store "Docker container networking requires explicit subnet config when using multiple custom networks"
```
## OpenCode Integration (Future)
### Plugin API
```javascript
import llmemory from '@opencode/llmemory';
// Store from agent
await llmemory.api.store(
'Discovered performance bottleneck in database query',
{ tags: ['performance', 'database'], entered_by: 'optimize-agent' }
);
// Search
const results = await llmemory.api.search('performance', {
tags: ['database'],
limit: 5
});
// Auto-extract *Remember* patterns
const memories = await llmemory.api.extractRemember(agentOutput, {
agentName: 'investigate-agent',
currentTask: 'debugging'
});
```
## Performance Targets
| Phase | Dataset Size | Search Latency | Storage Overhead |
|-------|-------------|----------------|------------------|
| 1 (MVP) | <500 memories | <50ms | Base |
| 2 (FTS5) | 10K memories | <100ms | +30% (FTS5 index) |
| 3 (Fuzzy) | 100K+ memories | <200ms | +200% (trigrams) |
## License
MIT
## Credits
**Planning & Design:**
- Agent A: Pragmatic iteration strategy, OpenCode integration patterns
- Agent B: Technical depth, comprehensive implementation specifications
- Combined approach: Hybrid FTS5 + fuzzy matching architecture
**Implementation:** To be determined
---
**Status:** Phase 0 Complete - Ready for Phase 1 implementation
**Next Step:** Project setup and database layer (see IMPLEMENTATION_PLAN.md)
**Estimated Time to MVP:** 12-15 hours of focused development