Tune LanceDB vector search¶

Configure embedding models, retrieval parameters, and maintenance settings for optimal CTI retrieval quality. Adjust similarity threshold, entity boost, and cross-encoder reranking to balance precision and recall for your workload.

Prerequisites¶

ZettelForge installed (pip install zettelforge)
Notes already stored to test retrieval against (see Store threat actor intelligence)

Steps¶

1. Configure the embedding model¶

Edit config.yaml:

embedding:
  provider: fastembed
  model: nomic-ai/nomic-embed-text-v1.5-Q
  dimensions: 768

Or set via environment variables:

export ZETTELFORGE_EMBEDDING_PROVIDER=fastembed
export AMEM_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5-Q

Supported configurations:

Provider	Config value	Model	Dimensions	Notes
fastembed (default)	`fastembed`	`nomic-ai/nomic-embed-text-v1.5-Q`	768	In-process ONNX, ~130 MB, ~7 ms/embed
Ollama (optional)	`ollama`	`nomic-embed-text-v2-moe:latest`	768	Requires Ollama running on `embedding.url`

Warning

Changing the embedding model after data has been indexed requires a full re-index. Existing vectors become incompatible with new model embeddings. Run python scripts/rebuild_index.py after changing models.

2. Verify embedding connectivity¶

from zettelforge.vector_memory import get_embedding

vector = get_embedding("APT28 uses Cobalt Strike for command and control")
print(f"Embedding dimensions: {len(vector)}")
print(f"First 5 values: {vector[:5]}")

Expected output:

Embedding dimensions: 768

3. Configure retrieval parameters¶

Edit the retrieval section of config.yaml:

retrieval:
  default_k: 10
  similarity_threshold: 0.25
  entity_boost: 2.5
  max_graph_depth: 2

Parameter reference:

Parameter	Default	Range	Effect
`default_k`	10	1–100	Maximum results returned per query
`similarity_threshold`	0.25	0.0–1.0	Minimum cosine similarity to include a result
`entity_boost`	2.5	0.0–10.0	Multiplicative boost per overlapping entity between query and note
`max_graph_depth`	2	1–5	Hops to traverse in knowledge graph during blended retrieval

4. Configure cross-encoder reranking¶

ZettelForge applies a cross-encoder reranker after initial vector retrieval to improve ranking quality. Reranking is enabled by default and bounded to control CPU cost.

retrieval:
  rerank_enabled: true
  rerank_max_candidates: 8
  rerank_doc_chars: 256
  rerank_model: Xenova/ms-marco-MiniLM-L-6-v2

Parameter	Default	Effect
`rerank_enabled`	`true`	Enable cross-encoder reranking pass
`rerank_max_candidates`	8	Maximum candidates the reranker scores
`rerank_doc_chars`	256	Characters of each note fed to the reranker
`rerank_model`	`Xenova/ms-marco-MiniLM-L-6-v2`	ONNX cross-encoder model

Raise rerank_max_candidates if relevant results are being ranked below noise. Lower it to reduce CPU time per query.

5. Tune for high precision (fewer, more relevant results)¶

retrieval:
  default_k: 5
  similarity_threshold: 0.50
  entity_boost: 3.0
  max_graph_depth: 1

from zettelforge.memory_manager import MemoryManager

mm = MemoryManager()
notes = mm.recall("APT28 Cobalt Strike C2", domain="cti", k=5)
print(f"High-precision results: {len(notes)}")

6. Tune for high recall (cast a wide net)¶

retrieval:
  default_k: 25
  similarity_threshold: 0.10
  entity_boost: 1.5
  max_graph_depth: 3

notes = mm.recall("APT28 Cobalt Strike C2", domain="cti", k=25)
print(f"High-recall results: {len(notes)}")

Tip

Start with the defaults (similarity_threshold: 0.25, entity_boost: 2.5). Lower the threshold only if relevant notes are being filtered out. Raise entity_boost if entity-specific queries return too much noise from semantically similar but entity-unrelated notes.

7. Configure the data directory¶

storage:
  data_dir: ~/.amem

Or:

export AMEM_DATA_DIR=/data/zettelforge

LanceDB stores its vector index at {data_dir}/vectordb/. The full directory layout:

~/.amem/
  notes.jsonl          # Note metadata
  vectordb/            # LanceDB vector index
  kg_nodes.jsonl       # Knowledge graph nodes
  kg_edges.jsonl       # Knowledge graph edges
  entity_index.json    # Entity index
  entity_aliases.json  # Local alias mappings
  zettelforge.db       # SQLite database
  telemetry/           # Operational telemetry
  logs/                # Log files

8. Configure LanceDB maintenance¶

ZettelForge runs a background cleanup daemon that prunes old LanceDB version chains. On write-heavy instances, unbounded version chains cause tail-latency growth. The daemon collapses the chain on a configurable interval.

lance:
  cleanup_interval_minutes: 60
  cleanup_older_than_seconds: 3600

Parameter	Default	Effect
`cleanup_interval_minutes`	60	Interval between cleanup passes. Set to `0` to disable.
`cleanup_older_than_seconds`	3600	Versions older than this are eligible for pruning.

For one-shot compaction of accumulated fragment chains, use the bundled script:

# Dry run first — inspect without mutating
python -m zettelforge.scripts.compact_lance --data-dir ~/.amem --dry-run

# Compact all shards
python -m zettelforge.scripts.compact_lance --data-dir ~/.amem --all --force

9. Rebuild the index after configuration changes¶

python scripts/rebuild_index.py

Optional flags to override default paths:

python scripts/rebuild_index.py --jsonl /path/to/notes.jsonl --lance /path/to/vectordb

Warning

Rebuilding the index re-embeds all notes. With the default fastembed provider this takes approximately 0.7 seconds per 100 notes.

LLM quick reference

Embedding config: embedding.provider (default fastembed, alternative ollama), embedding.model (default nomic-ai/nomic-embed-text-v1.5-Q), embedding.dimensions (default 768). Env overrides: ZETTELFORGE_EMBEDDING_PROVIDER, AMEM_EMBEDDING_MODEL. The default fastembed provider runs in-process via ONNX with no external service required.

Retrieval config: retrieval.default_k (10), retrieval.similarity_threshold (0.25, range 0.0–1.0), retrieval.entity_boost (2.5, multiplicative per overlapping entity), retrieval.max_graph_depth (2, hops in KG traversal).

Reranking config: retrieval.rerank_enabled (true), retrieval.rerank_max_candidates (8), retrieval.rerank_doc_chars (256), retrieval.rerank_model (Xenova/ms-marco-MiniLM-L-6-v2).

Maintenance config: lance.cleanup_interval_minutes (60), lance.cleanup_older_than_seconds (3600).

Data directory: storage.data_dir (default ~/.amem). LanceDB vector index at {data_dir}/vectordb/. Env override: AMEM_DATA_DIR.

High precision preset: default_k: 5, similarity_threshold: 0.50, entity_boost: 3.0, max_graph_depth: 1. High recall preset: default_k: 25, similarity_threshold: 0.10, entity_boost: 1.5, max_graph_depth: 3.

Rebuild index after model change: python scripts/rebuild_index.py.