306 lines
8.0 KiB
Markdown
306 lines
8.0 KiB
Markdown
# LLMemory - AI Agent Memory System
|
|
|
|
A persistent memory/journal system for AI agents with grep-like search and fuzzy matching.
|
|
|
|
## Overview
|
|
|
|
LLMemory provides AI agents with long-term memory across sessions. Think of it as a personal knowledge base with powerful search capabilities, designed specifically for agent workflows.
|
|
|
|
**Key Features:**
|
|
- 🔍 **Grep-like search** - Familiar query syntax for AI agents
|
|
- 🎯 **Fuzzy matching** - Handles typos automatically
|
|
- 🏷️ **Tag-based organization** - Easy categorization and filtering
|
|
- ⏰ **Expiration support** - Auto-cleanup of time-sensitive info
|
|
- 📊 **Relevance ranking** - Best results first, token-efficient
|
|
- 🔌 **OpenCode integration** - Plugin API for seamless workflows
|
|
|
|
## Status
|
|
|
|
**Current Phase:** Planning Complete (Phase 0)
|
|
**Next Phase:** MVP Implementation (Phase 1)
|
|
|
|
This project is in the initial planning stage. The architecture and implementation plan are complete, ready for development.
|
|
|
|
## Quick Start (Future)
|
|
|
|
```bash
|
|
# Installation (when available)
|
|
npm install -g llmemory
|
|
|
|
# Store a memory
|
|
memory store "Docker Compose uses bridge networks by default" \
|
|
--tags docker,networking
|
|
|
|
# Search memories
|
|
memory search "docker networking"
|
|
|
|
# List recent memories
|
|
memory list --limit 10
|
|
|
|
# Show agent documentation
|
|
memory --agent-context
|
|
```
|
|
|
|
## Documentation
|
|
|
|
- **[SPECIFICATION.md](./SPECIFICATION.md)** - Complete technical specification
|
|
- **[IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md)** - Phased development plan
|
|
- **[ARCHITECTURE.md](./docs/ARCHITECTURE.md)** - System design (to be created)
|
|
- **[AGENT_GUIDE.md](./docs/AGENT_GUIDE.md)** - Guide for AI agents (to be created)
|
|
|
|
## Architecture
|
|
|
|
### Three-Phase Implementation
|
|
|
|
**Phase 1: MVP (2-3 days)**
|
|
- Basic CLI with store/search/list/prune commands
|
|
- Simple LIKE-based search
|
|
- Tag filtering and expiration handling
|
|
- Target: <500 memories, <50ms search
|
|
|
|
**Phase 2: FTS5 (3-5 days)**
|
|
- Migrate to SQLite FTS5 for production search
|
|
- BM25 relevance ranking
|
|
- Boolean operators (AND/OR/NOT)
|
|
- Target: 10K+ memories, <100ms search
|
|
|
|
**Phase 3: Fuzzy Layer (3-4 days)**
|
|
- Trigram indexing for typo tolerance
|
|
- Levenshtein distance matching
|
|
- Intelligent cascade (exact → fuzzy)
|
|
- Target: 100K+ memories, <200ms search
|
|
|
|
### Technology Stack
|
|
|
|
- **Language:** Node.js (JavaScript/TypeScript)
|
|
- **Database:** SQLite with better-sqlite3
|
|
- **CLI:** Commander.js
|
|
- **Search:** FTS5 + trigram fuzzy matching
|
|
- **Testing:** Vitest
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
llmemory/
|
|
├── src/
|
|
│ ├── cli.js # CLI entry point
|
|
│ ├── commands/ # Command implementations
|
|
│ ├── db/ # Database layer
|
|
│ ├── search/ # Search strategies (LIKE, FTS5, fuzzy)
|
|
│ ├── utils/ # Utilities (validation, formatting)
|
|
│ └── extractors/ # Auto-extraction (*Remember* pattern)
|
|
├── test/ # Test suite
|
|
├── docs/ # Documentation
|
|
├── bin/ # Executable wrapper
|
|
├── SPECIFICATION.md # Technical spec
|
|
├── IMPLEMENTATION_PLAN.md # Development roadmap
|
|
└── README.md # This file
|
|
```
|
|
|
|
## Development
|
|
|
|
### Setup
|
|
|
|
```bash
|
|
cd llmemory
|
|
npm install
|
|
npm test
|
|
```
|
|
|
|
### Implementation Status
|
|
|
|
See [IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md) for detailed progress tracking.
|
|
|
|
**Current Progress:**
|
|
- [x] Phase 0: Planning and documentation
|
|
- [ ] Phase 1: MVP (Simple LIKE search)
|
|
- [ ] Project setup
|
|
- [ ] Database layer
|
|
- [ ] Store command
|
|
- [ ] Search command
|
|
- [ ] List command
|
|
- [ ] Prune command
|
|
- [ ] CLI integration
|
|
- [ ] Testing
|
|
- [ ] Phase 2: FTS5 migration
|
|
- [ ] Phase 3: Fuzzy layer
|
|
|
|
### Contributing
|
|
|
|
1. Review [SPECIFICATION.md](./SPECIFICATION.md) for architecture
|
|
2. Check [IMPLEMENTATION_PLAN.md](./IMPLEMENTATION_PLAN.md) for next steps
|
|
3. Pick an uncompleted task from the current phase
|
|
4. Write tests first (TDD approach)
|
|
5. Implement feature
|
|
6. Update checkboxes in IMPLEMENTATION_PLAN.md
|
|
7. Commit with clear message
|
|
|
|
### Testing
|
|
|
|
```bash
|
|
# Run all tests
|
|
npm test
|
|
|
|
# Run tests in watch mode
|
|
npm run test:watch
|
|
|
|
# Run specific test file
|
|
npm test search.test.js
|
|
|
|
# Coverage report
|
|
npm run test:coverage
|
|
```
|
|
|
|
## Usage Examples (Future)
|
|
|
|
### Storing Memories
|
|
|
|
```bash
|
|
# Basic storage
|
|
memory store "PostgreSQL VACUUM FULL locks tables, use VACUUM ANALYZE instead"
|
|
|
|
# With tags
|
|
memory store "Docker healthchecks need curl --fail for proper exit codes" \
|
|
--tags docker,best-practices
|
|
|
|
# With expiration
|
|
memory store "Staging server at https://staging.example.com" \
|
|
--tags infrastructure,staging \
|
|
--expires "2025-12-31"
|
|
|
|
# From agent
|
|
memory store "NixOS flake.lock must be committed for reproducible builds" \
|
|
--tags nixos,build-system \
|
|
--entered-by investigate-agent
|
|
```
|
|
|
|
### Searching Memories
|
|
|
|
```bash
|
|
# Basic search
|
|
memory search "docker"
|
|
|
|
# Multiple terms (implicit AND)
|
|
memory search "docker networking"
|
|
|
|
# Boolean operators
|
|
memory search "docker AND compose"
|
|
memory search "docker OR podman"
|
|
memory search "database NOT postgresql"
|
|
|
|
# Phrase search
|
|
memory search '"exact phrase"'
|
|
|
|
# With filters
|
|
memory search "kubernetes" --tags production,k8s
|
|
memory search "error" --after "2025-10-01"
|
|
memory search "config" --entered-by optimize-agent --limit 5
|
|
```
|
|
|
|
### Managing Memories
|
|
|
|
```bash
|
|
# List recent
|
|
memory list --limit 20
|
|
|
|
# List by tag
|
|
memory list --tags docker --sort created --order desc
|
|
|
|
# Show statistics
|
|
memory stats
|
|
memory stats --tags # Tag frequency
|
|
memory stats --agents # Memories per agent
|
|
|
|
# Prune expired
|
|
memory prune --dry-run # Preview
|
|
memory prune --force # Execute
|
|
|
|
# Export/import
|
|
memory export backup.json
|
|
memory import backup.json
|
|
```
|
|
|
|
## Memory Format Guidelines
|
|
|
|
### Good Memory Examples
|
|
|
|
```bash
|
|
# Technical detail
|
|
memory store "Git worktree: 'git worktree add -b feature ../feature' creates parallel working directory without cloning" --tags git,workflow
|
|
|
|
# Error resolution
|
|
memory store "Node.js ENOSPC: Increase inotify watches with 'echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p'" --tags nodejs,linux,troubleshooting
|
|
|
|
# Configuration pattern
|
|
memory store "Nginx reverse proxy: Set 'proxy_set_header X-Real-IP \$remote_addr' to preserve client IP through proxy chain" --tags nginx,networking
|
|
```
|
|
|
|
### Anti-Patterns
|
|
|
|
```bash
|
|
# Too vague ❌
|
|
memory store "Fixed the bug"
|
|
|
|
# Better ✅
|
|
memory store "Fixed React infinite render loop by adding missing dependencies to useEffect array"
|
|
|
|
# Widely known ❌
|
|
memory store "Docker is a containerization platform"
|
|
|
|
# Specific insight ✅
|
|
memory store "Docker container networking requires explicit subnet config when using multiple custom networks"
|
|
```
|
|
|
|
## OpenCode Integration (Future)
|
|
|
|
### Plugin API
|
|
|
|
```javascript
|
|
import llmemory from '@opencode/llmemory';
|
|
|
|
// Store from agent
|
|
await llmemory.api.store(
|
|
'Discovered performance bottleneck in database query',
|
|
{ tags: ['performance', 'database'], entered_by: 'optimize-agent' }
|
|
);
|
|
|
|
// Search
|
|
const results = await llmemory.api.search('performance', {
|
|
tags: ['database'],
|
|
limit: 5
|
|
});
|
|
|
|
// Auto-extract *Remember* patterns
|
|
const memories = await llmemory.api.extractRemember(agentOutput, {
|
|
agentName: 'investigate-agent',
|
|
currentTask: 'debugging'
|
|
});
|
|
```
|
|
|
|
## Performance Targets
|
|
|
|
| Phase | Dataset Size | Search Latency | Storage Overhead |
|
|
|-------|-------------|----------------|------------------|
|
|
| 1 (MVP) | <500 memories | <50ms | Base |
|
|
| 2 (FTS5) | 10K memories | <100ms | +30% (FTS5 index) |
|
|
| 3 (Fuzzy) | 100K+ memories | <200ms | +200% (trigrams) |
|
|
|
|
## License
|
|
|
|
MIT
|
|
|
|
## Credits
|
|
|
|
**Planning & Design:**
|
|
- Agent A: Pragmatic iteration strategy, OpenCode integration patterns
|
|
- Agent B: Technical depth, comprehensive implementation specifications
|
|
- Combined approach: Hybrid FTS5 + fuzzy matching architecture
|
|
|
|
**Implementation:** To be determined
|
|
|
|
---
|
|
|
|
**Status:** Phase 0 Complete - Ready for Phase 1 implementation
|
|
**Next Step:** Project setup and database layer (see IMPLEMENTATION_PLAN.md)
|
|
**Estimated Time to MVP:** 12-15 hours of focused development
|