Retrieval policies

Use this reference to trace how ZettelForge 2.8.0 ranks notes when you call MemoryManager.recall(). The read path classifies your query, retrieves vector and graph candidates, blends those two scored signals, adds intent-specific candidates, reranks a bounded head, applies result filters, and returns at most k notes.

Prerequisite: Read the Memory Manager API for the public recall() signature. Read the Configuration reference before you change retrieval settings.

Retrieval controls are in progress

ZettelForge loads retrieval.default_k, retrieval.similarity_threshold, retrieval.entity_boost, and retrieval.max_graph_depth, but MemoryManager.recall() does not read those four values in v2.8.0. Pass k to recall() directly. The direct read path uses the VectorRetriever constructor defaults and passes graph depth 2 directly.

The LanceDB branch also does not apply VectorRetriever.similarity_threshold. Treat threshold configuration as in progress. The run-238 verification of the high-fanout entity boundary also fails, so do not treat retrieval.entity_max_fanout as a guaranteed cutoff until that defect is resolved.

Read-path order

MemoryManager.recall() executes these stages in order:

1. Classify the query and load its policy dictionary.
2. Extract and resolve query entities, then apply the high-fanout filter.
3. Retrieve scored vector candidates.
4. For a temporal query, move candidates containing recognized query dates to the head when dateparser is installed.
5. Traverse the active store's knowledge graph to depth 2.
6. Normalize and blend vector and graph scores.
7. Fall back to the vector list when blending returns fewer notes than vector retrieval produced.
8. For a causal query, add notes reached through forward and incoming causal edges.
9. Add up to three entity-index notes per resolved query entity.
10. Rerank a bounded head when reranking is enabled.
11. Filter superseded and expired notes according to the recall() arguments.
12. Truncate the final list to k.

The policy dictionary does not control every stage. Only its vector and graph fields enter normalized score fusion. Entity, temporal, causal, rerank, and final-limit behavior use separate logic.

Intent classification

Module: zettelforge.intent_classifier

Intent values

Value	Meaning	Example
factual	Direct fact or entity lookup	What CVE was used in the SolarWinds attack?
temporal	Time or change query	What changed since May 2024?
relational	Relationship or graph query	Who uses Cobalt Strike?
causal	Cause-and-effect query	Why did the attacker pivot?
exploratory	Broad context query and classifier fallback	Tell me about APT28
unknown	Enum and policy value available to callers	The built-in classifier does not return this value as its no-match fallback.

With the default IntentClassifier(use_llm_fallback=False), a query with no accepted keyword result returns exploratory with confidence 0.3 and method default.

Keywords

The classifier lowercases the query and counts substring matches.

Intent	Keywords
factual	cve-, cve, vulnerability, exploit, threat, what is, what was, who is, name, identify
temporal	when, timeline, since, before, after, changed, history, previously, earlier, latest, recent
relational	who uses, who targets, who conducts, what tools does, what malware does, what technique, what cve does, used by, attributed to, related to, connected to, associated with, linked to, uses tool, between, relationship, connection, link, which actor, which group, which apt, what does
causal	why, because, caused by, enables, leads to, results in, due to, reason for
exploratory	tell me about, explain, describe, overview, information on, details about, context, summarize, what do we know, brief

Decision rules

Condition	Result
Best score is at least 2	Return the best intent; confidence is min(1.0, score / 4); method is keyword.
Best score is 1 and no other intent has a positive score	Return the best intent; confidence is 0.6; method is keyword_unambiguous.
Neither rule accepts the result and LLM fallback is enabled	Call the LLM classifier with max_tokens=20 and temperature=0.1. A recognized response has confidence 0.8 and method llm.
LLM classification fails or returns an unrecognized value	Return exploratory; confidence is 0.5; method is llm_fallback.
Neither keyword rule accepts the result and LLM fallback is disabled	Return exploratory; confidence is 0.3; method is default.

get_intent_classifier() creates its global classifier with LLM fallback disabled.

Policy dictionary

IntentClassifier.get_traversal_policy() returns these values:

Intent	vector	entity_index	graph	temporal	top_k
factual	0.3	0.7	0.2	0.0	3
temporal	0.2	0.1	0.2	0.5	5
relational	0.2	0.2	0.5	0.1	10
causal	0.1	0.1	0.6	0.2	10
exploratory	0.5	0.2	0.2	0.1	10
unknown	0.4	0.2	0.2	0.2	5

The four weights total 1.2 for factual and 1.0 for every other row. ZettelForge does not normalize the whole policy dictionary.

Runtime use

Field	Runtime effect in MemoryManager.recall()
vector	Multiplies the normalized vector score during BlendedRetriever.blend().
graph	Multiplies the normalized graph score during BlendedRetriever.blend().
entity_index	Not consumed. Entity augmentation uses a fixed k=3 per resolved entity.
temporal	Not consumed. Temporal handling reorders date-matching vector tuples without changing their scores.
top_k	Not consumed. The k argument passed to recall() controls retrieval and the final limit.

Vector retrieval

Module: zettelforge.vector_retriever

Constructor defaults

Parameter	Default	Effect
similarity_threshold	0.15	Filters the in-memory branch after boosts. The LanceDB branch does not apply it.
entity_boost	2.5	Multiplies a score once per overlapping resolved entity.
exact_match_boost	1.0	Multiplies once for each resolved query entity whose text appears in the note content. The default leaves the score unchanged.
regenerate_invalid_embeddings	True	Regenerates invalid note vectors in the in-memory branch.
memory_store	None	Creates a MemoryStore when you do not pass one.
note_lookup	None	Uses the retriever's store unless you provide a cross-backend lookup.

retrieve() defaults to domain=None, k=10, include_links=True, use_lancedb=True, and return_scores=False.

Return values

return_scores	Return value
False	list[MemoryNote]
True	list[tuple[MemoryNote, float]]

MemoryManager.recall() requests scores because the blend stage requires them.

In-memory scoring

The in-memory branch computes cosine similarity. A zero-norm input produces 0.0.

For each candidate, the branch applies these multipliers before thresholding:

• Entity overlap: entity_boost ** overlap_count.
• Exact text matches: one exact_match_boost multiplier for each resolved query entity found in the note content.

The branch keeps a note when its boosted score is at least similarity_threshold, sorts retained notes by score, and keeps the first k before link expansion.

The branch treats an embedding as invalid when it is missing, has a length other than 768, contains only zeros, or has variance below 0.001. With regeneration enabled, it calls get_embedding() on the first 1,000 content characters. It skips a note when the resulting vector remains invalid.

The default embedding configuration uses provider fastembed, model nomic-ai/nomic-embed-text-v1.5-Q, and 768 dimensions.

LanceDB scoring

When LanceDB is available and enabled, the retriever searches each selected table, requests up to k * 2 rows, and calculates a score as 1.0 - row["_distance"]. It sorts across tables, keeps the first k, applies entity and exact-match boosts, and then expands links.

This branch does not apply similarity_threshold and does not run the in-memory embedding validation routine. If LanceDB raises an exception or returns no usable result, retrieve() runs the in-memory branch.

Link expansion

With include_links=True, each directly related note receives parent_score * 0.5. The retriever de-duplicates linked note IDs and returns at most k * 2 vector-stage results.

Graph retrieval

Module: zettelforge.graph_retriever

GraphRetriever.retrieve_note_ids() accepts query_entities: dict[str, list[str]] and defaults max_depth to 2. MemoryManager.recall() wraps its active storage backend in StoreGraphSource, which reads the same per-store knowledge graph populated by the write path.

The graph source supplies get_node(), get_node_by_id(), and get_outgoing_edges(). Breadth-first search starts at each resolved entity and scores every reached note as 1 / (1 + hops).

Hops	Score
0	1.000
1	0.500
2	0.333

When more than one path reaches a note, the result keeps the shortest path. retrieve_note_ids() sorts ScoredResult values by descending score. Each result contains note_id, score, hops, and path.

Normalized score fusion

Module: zettelforge.blended_retriever

BlendedRetriever.blend() normalizes vector and graph scores independently with (score - minimum) / (maximum - minimum). When every score in one signal is equal, it assigns 0.5 to every item in that signal.

It then calculates contributions as follows:

• Vector contribution: normalized_vector_score * policy["vector"].
• Graph contribution: normalized_graph_score * policy["graph"].
• A note present in both signals receives the sum of both contributions.

The method sorts by the combined score and returns at most its k argument. It does not read entity_index, temporal, or top_k from the policy dictionary.

Reciprocal Rank Fusion alternative

blend_rrf() is an available alternative, but MemoryManager.recall() calls blend() in v2.8.0. blend_rrf() defaults rrf_k to 60 and adds 1 / (rrf_k + rank) for every signal containing the note.

Post-blend augmentation and reranking

The direct recall path performs additional work outside BlendedRetriever:

• A causal intent traverses outgoing and incoming causal edges with max_depth=3 and max_visited=50, then adds note endpoints and source-note references.
• Entity augmentation calls recall_entity(..., k=3) for every retained resolved query entity and de-duplicates note IDs.
• Reranking processes only the first retrieval.rerank_max_candidates notes and the first retrieval.rerank_doc_chars characters of each note. The tail keeps its pre-rerank order.
• A reranker load or scoring failure logs a warning and preserves the original order.

Configuration status

Key	Default	v2.8.0 direct-recall status
retrieval.default_k	10	Loaded but not read; recall() defaults its own k to 10.
retrieval.similarity_threshold	0.25	Loaded but not passed to VectorRetriever; its constructor defaults to 0.15, and LanceDB does not enforce either value.
retrieval.entity_boost	2.5	Loaded but not passed to VectorRetriever; its constructor independently defaults to 2.5.
retrieval.max_graph_depth	2	Loaded but not read; recall() passes 2 directly.
retrieval.rerank_enabled	true	Read before reranking.
retrieval.rerank_max_candidates	8	Bounds the reranked head.
retrieval.rerank_doc_chars	256	Bounds candidate text sent to the reranker.
retrieval.rerank_model	Xenova/ms-marco-MiniLM-L-6-v2	Selects the TextCrossEncoder model.
retrieval.rerank_threads	8	Sets ONNX threads; 0 delegates to the runtime default.
retrieval.entity_max_fanout	25	Supplies the high-fanout filter threshold; run-238 behavior verification fails and remains in progress.