# RFD 032: Grizzly Semantic Search - **Status**: Accepted - **Category**: Design - **Authors**: Jean Mertz - **Date**: 2026-03-07 ## Summary Add semantic vector search, FTS5 full-text search, and typo tolerance to grizzly's `note_search` tool. The goal is to find notes by *meaning*, not just literal text. Searching "productivity systems" should surface notes about GTD and Pomodoro even when those exact words don't appear. ## Context Grizzly is a Bear.app MCP server that lives in `crates/contrib/grizzly/`. It exposes `note_get`, `note_search`, and `note_create` tools over the MCP protocol. The current search implementation uses SQL `LIKE '%query%'` with hand-rolled relevance scoring. It works but has three weaknesses: 1. **Substring matching** — "prod" matches "reproduce", no word boundary awareness. 2. **No typo handling** — "productivty" finds nothing. 3. **No semantic understanding** — searching "task management" won't find a note titled "GTD methodology" unless those words appear literally. This RFD covers three layers of improvement, each independent and additive. ## Architecture Overview ```txt ┌─────────────────────┐ │ note_search tool │ └──────────┬──────────┘ │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ ┌────────┐ ┌──────────┐ ┌──────────┐ │ LIKE │ │ FTS5 │ │ Semantic │ │ search │ │ search │ │ search │ └────────┘ └──────────┘ └──────────┘ │ │ │ └──────────────┼──────────────┘ ▼ ┌────────────────┐ │ Score Fusion │ │ (RRF) │ └────────────────┘ ``` The search function dispatches to one or more backends depending on what's available, then merges results using Reciprocal Rank Fusion. ## Existing Code You Need to Know | File | What it does | |-----------------|------------------------------------------| | `src/db.rs` | `BearDb` — opens Bear's read-only SQLite | | | database, discovers schema at init, | | | hands out short-lived connections. All | | | queries go through | | | `with_connection(fn(conn, cte))`. | | `src/schema.rs` | Discovers Bear's Core Data junction | | | table names (`Z_5TAGS`, etc.) at | | | runtime, generates a normalizing CTE | | | that maps raw tables to clean `notes`, | | | `tags`, `note_tags` views. | | `src/search.rs` | `SearchParams` and `execute()` — the | | | current LIKE-based search with SQL-level | | | scoring and Rust-side line extraction. | | | This is the file you'll modify most. | | `src/server.rs` | MCP tool handlers. `note_search` calls | | | `db.search(¶ms)` and formats results | | | as XML. | | `src/main.rs` | CLI entry point. `--jp` flag for JP tool | | | protocol, `--note-create` for write | | | access. You'll add new flags here. | | `src/error.rs` | Error enum. You'll add new variants. | | `Cargo.toml` | Dependencies are gated behind the | | | workspace. New deps go here with feature | | | flags. | Bear's database is read-only to us (we never write to it). The sidecar database for embeddings is ours and we can write to it freely. ## Layer 1: FTS5 Full-Text Search ### What Replace `LIKE '%query%'` with SQLite FTS5 for word-aware search with BM25 ranking. ### Why FTS5 gives us tokenized word matching (no false substring hits), BM25 relevance ranking, prefix queries (`product*`), phrase queries (`"getting things done"`), and boolean operators (`productivity OR gtd`). ### How Our `rusqlite` dependency already compiles SQLite with FTS5 enabled (the `bundled` feature includes it). No new dependencies needed. **On each search call**, create a temporary in-memory FTS5 table, populate it from the notes CTE, query it, then let it drop when the connection closes. Since we use short-lived connections, this happens every search. ```sql CREATE VIRTUAL TABLE temp.fts_notes USING fts5( id UNINDEXED, title, content, tokenize='unicode61' ); INSERT INTO temp.fts_notes(id, title, content) SELECT id, title, content FROM notes WHERE is_trashed = 0; -- Search with BM25 ranking SELECT id, title, content, rank FROM temp.fts_notes WHERE fts_notes MATCH ? ORDER BY rank LIMIT ?; ``` **File changes:** - New file `src/fts.rs` — functions to create the temp FTS table, run FTS5 queries, and map results back to `ScoredNote`. - Modify `src/search.rs` — `execute()` tries FTS5 first. If the FTS5 query fails (malformed syntax, etc.), fall back to LIKE search. Add a `mode` field to `SearchParams`: ```rust pub enum SearchMode { /// Try FTS5, fall back to LIKE on error. Auto, /// Force FTS5 (fail if unavailable). Fts, /// Force LIKE (current behavior). Like, } ``` - Modify `src/error.rs` — add `FtsError` variant. **Performance concern:** For a few hundred notes, building the FTS table per-search is <50ms. For thousands, it could be slow. Measure this. If it's a problem, consider caching the FTS table in a persistent temp file that gets rebuilt when Bear's database modification time changes. **Testing:** Add tests that exercise FTS5 query syntax (prefix, phrase, boolean). Test the LIKE fallback by passing malformed FTS5 queries. ### Typo Tolerance via Trigram Tokenizer FTS5 supports a `trigram` tokenizer that indexes character trigrams instead of words. This naturally handles typos because "productivty" and "productivity" share most trigrams. Add a second FTS5 table using the trigram tokenizer: ```sql CREATE VIRTUAL TABLE temp.fts_trigram USING fts5( id UNINDEXED, title, content, tokenize='trigram' ); ``` When the `unicode61` FTS5 search returns few or no results, retry against the trigram table. The trigram table has worse ranking quality (many false positives) so it's a fallback, not the primary. This is a one-line tokenizer change with no new dependencies. The tradeoff is a larger in-memory index and noisier results. ## Layer 2: Semantic Vector Search ### What Embed notes and queries as vectors using a local ML model, then find semantically similar notes by cosine distance. ### Dependencies Two new dependencies, both behind a `semantic` cargo feature flag: 1. **`fastembed`** (Apache 2.0) — Rust library for generating text embeddings locally using ONNX Runtime. Handles model download, tokenization, and inference. We use it to convert text into dense float vectors. Crate: https://crates.io/crates/fastembed 2. **`sqlite-vector`** (Elastic License 2.0 with OSI open-source exception) — C-based SQLite extension for vector similarity search. SIMD-accelerated distance functions, quantization for memory efficiency, no preindexing required. We compile it from source via `build.rs`. Source: https://github.com/sqliteai/sqlite-vector The license situation: `sqlite-vector` is free for OSI-licensed open-source projects (grizzly is MIT). If grizzly is ever used in a closed-source product, a commercial license would be needed. `fastembed` is Apache 2.0, no restrictions. ```toml # In Cargo.toml [features] default = [] semantic = ["dep:fastembed"] [dependencies] fastembed = { version = "5", optional = true, default-features = false, features = [ "ort-download-binaries-native-tls", "hf-hub-native-tls", ] } [build-dependencies] cc = "1" # for compiling sqlite-vector C source ``` For `sqlite-vector`, vendor the C source files into `crates/contrib/grizzly/vendor/sqlite-vector/` and compile them in `build.rs` using the `cc` crate. Then register the extension at runtime via `rusqlite`'s `load_extension_enable()` + `load_extension()`, or (better) use `Connection::load_extension()` with the compiled static library. The exact integration path needs prototyping — rusqlite's `load_extension` wants a `.dylib`/`.so` path, so the static compilation approach may require using rusqlite's `create_module()` API or calling `sqlite3_vector_init` directly via FFI. ### Sidecar Database Embeddings are stored in a separate SQLite `embeddings.db` database. This database is owned by grizzly and is read-write. The path is set using a CLI flag or environment variable. Ideally the caller uses something like `directories::ProjectDirs::cache_dir()` to get the cache directory. Schema: ```sql CREATE TABLE IF NOT EXISTS meta ( key TEXT PRIMARY KEY, value TEXT ); -- Stores: model_name, dimension, last_full_index_at CREATE TABLE IF NOT EXISTS embeddings ( note_id TEXT PRIMARY KEY, embedding BLOB NOT NULL, -- float32 vector, dimension from meta updated_at TEXT NOT NULL -- ISO 8601, from Bear's ZMODIFICATIONDATE ); ``` The `embedding` column stores raw float32 bytes (via `fastembed`'s output). `sqlite-vector` operates directly on these BLOBs. ### Indexing Embeddings are built via a CLI subcommand: ``` grizzly index [--model MODEL_NAME] ``` This: 1. Opens Bear's database (read-only) and the sidecar (read-write). 2. Loads the embedding model (`BAAI/bge-small-en-v1.5` by default, 512 dimensions, ~127MB download on first run). 3. Queries all non-trashed notes from Bear. 4. For each note whose `updated_at` has changed (or is missing from the sidecar), generates an embedding from `"{title}\n{content}"`. 5. Writes embeddings to the sidecar in batches. 6. Calls `vector_init` and `vector_quantize` on the sidecar for fast search. Incremental: only re-embeds notes whose modification timestamp changed. Full rebuild via `grizzly index --rebuild`. **File changes:** - New file `src/embedding.rs` — sidecar DB management (open/create, schema migration), embedding generation via `fastembed`, incremental update logic. - New file `src/semantic.rs` — vector search against the sidecar, returns `Vec<(note_id, distance)>`. - Modify `src/main.rs` — add `index` subcommand with `--model` and `--rebuild` flags. - Modify `src/error.rs` — add `EmbeddingError`, `SidecarError` variants. ### Search Flow When semantic search is enabled (embeddings exist in the sidecar): 1. Embed the query string using the same model. 2. Query the sidecar: `SELECT note_id, distance FROM vector_quantize_scan('embeddings', 'embedding', ?, 50)`. 3. Run FTS5/LIKE search in parallel (same query string). 4. Merge results using Reciprocal Rank Fusion (RRF). RRF formula: for each note appearing in any result list, `score = sum(1 / (k + rank_i))` where `k = 60` (standard constant) and `rank_i` is the note's position in result list `i`. Notes are then sorted by fused score descending. ```rust fn reciprocal_rank_fusion( results: &[Vec], // each inner vec is note_ids in ranked order k: f64, // typically 60.0 ) -> Vec<(String, f64)> { let mut scores: HashMap = HashMap::new(); for list in results { for (rank, note_id) in list.iter().enumerate() { *scores.entry(note_id.clone()).or_default() += 1.0 / (k + rank as f64 + 1.0); } } let mut ranked: Vec<_> = scores.into_iter().collect(); ranked.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal)); ranked } ``` **File changes:** - New file `src/fusion.rs` — the RRF implementation. - Modify `src/search.rs` — `execute()` checks if sidecar exists and has embeddings, dispatches to semantic + text search, merges via RRF. ### Graceful Degradation The search pipeline degrades gracefully: | Condition | Behavior | |----------------------------|----------------------------------| | Sidecar exists, FTS5 works | Semantic + FTS5, merged with RRF | | Sidecar exists, FTS5 fails | Semantic + LIKE, merged with RRF | | No sidecar, FTS5 works | FTS5 only | | No sidecar, FTS5 fails | LIKE only (current behavior) | The user never sees an error from missing embeddings. They just get less relevant results. ### CLI Flags ``` grizzly [--jp] [--note-create] [--no-semantic] grizzly index [--model MODEL] [--rebuild] ``` - `--no-semantic` — disable semantic search even if embeddings exist (useful for debugging). - `index` subcommand — build or update the embedding index. - `--model` — override the embedding model (default: `BAAI/bge-small-en-v1.5`). - `--rebuild` — force full re-embedding of all notes. ### Model Choice Default: `BAAI/bge-small-en-v1.5` (fastembed's own default) - 512 dimensions, 33M parameters, ~127MB download - Cached in `~/.cache/huggingface/` after first download - Fast inference (~1ms per short text on M-series Mac) - Ranks significantly higher than `all-MiniLM-L6-v2` on MTEB benchmarks (rank 94 vs 123, mean score 43.76 vs 41.39) at the same memory tier Other reasonable choices if users want to trade speed/size for quality: | Model | Memory | Dims | MTEB Rank | |--------------------------------------|--------|------|-----------| | `BAAI/bge-base-en-v1.5` | 390MB | 768 | 82 | | `nomic-ai/nomic-embed-text-v1.5` | 522MB | 768 | 87 | | `mixedbread-ai/mxbai-embed-large-v1` | 639MB | 1024 | 61 | Users can switch via `--model`, but changing models requires a full rebuild since embeddings from different models aren't compatible. Store the model name in the sidecar's `meta` table. On search, verify the model matches. If it doesn't, log a warning and skip semantic search (don't crash). ## Implementation Order 1. **FTS5** — no new dependencies, moderate complexity. Validate that the temp-table-per-search approach is fast enough. 2. **Trigram typo tolerance** — trivial addition on top of FTS5. 3. **sqlite-vector integration** — figure out the build.rs / FFI story. This is the riskiest part. Prototype it in isolation before integrating. 4. **fastembed integration** — straightforward API, but the ONNX Runtime binary is large. Verify it works behind a feature flag without bloating the default build. 5. **Sidecar database** — schema, migration, incremental indexing. 6. **Search fusion** — wire it all together with RRF. Steps 1-2 can ship independently. Steps 3-6 ship together as the `semantic` feature. ## Risks - **sqlite-vector build integration**: Compiling C source via `build.rs` and loading it into rusqlite's bundled SQLite may be tricky. The extension expects to be loaded via `sqlite3_vector_init`, which is normally called by `load_extension()`. With bundled SQLite, we may need to call the init function directly via FFI after getting the `sqlite3*` handle from rusqlite. This needs a prototype. - **ONNX Runtime binary size**: The `ort` crate (used by `fastembed`) downloads prebuilt ONNX Runtime binaries (~50MB). This only affects builds with the `semantic` feature, but it's a large addition. The download happens at build time, not runtime. - **First-run model download latency**: The first `grizzly index` call downloads the embedding model from Hugging Face (~127MB for the default `bge-small-en-v1.5`). Subsequent runs use the cache. This should be clearly communicated to the user (fastembed supports a download progress callback). - **FTS5 temp table rebuild cost**: If Bear has 5,000+ notes, rebuilding the FTS5 index on every search could be slow. Measure this early. Mitigation: cache the FTS table in a persistent temp file, keyed by the Bear database's file modification time. - **License**: `sqlite-vector`'s Elastic License 2.0 with open-source exception is fine for grizzly (MIT), but worth tracking if the project's license situation changes. ## Not In Scope - Multi-language embedding models (English-only for now; users can override with `--model` if needed) - Streaming index updates / watch mode (batch rebuild is fine for personal note collections) - Remote embedding APIs (everything stays local) - Image embeddings (Bear notes are text) - Graph-based note relationships (explored `sqlitegraph`, rejected due to GPL license and overkill)