RFC-003: Read-Path Depth Routing¶

Field	Value
Author	Patrick Roland
Status	Draft — adversarial review completed, changes requested
Created	2026-04-16
Last updated	2026-04-16
ZettelForge version	v2.7.0 (design only; not implemented)
Related RFCs	RFC-001 Conversational Entity Extractor · RFC-002 Universal LLM Provider Interface

Not implemented

RFC-003 is a design proposal. As of v2.7.0, none of the proposed modules (quality_gate.py, query_profile.py, query_markers.py, retrieval/) exist in the codebase and config.default.yaml has no recall.depth_routing section. The current recall() runs the uniform System 1 pipeline on every query. This page documents the proposal and its review status.

Context¶

recall() applies the same retrieval pipeline to every query regardless of complexity. Measured on a DGX Spark ARM64 with a ~4,450-note store:

CTI single-entity lookups: p50 111 ms, p95 ~200 ms, ~75% accuracy
LOCOMO multi-hop long-horizon: p50 1,240 ms, p95 2,282 ms
Intent classification (keyword path): < 1 ms

Two failure modes follow from these numbers.

Low-complexity queries overpay. A query like CVE-2024-1234 triggers the full pipeline — intent classification, entity extraction, vector retrieval over 4,450 notes, graph traversal at depth 2, entity-augmented pull, and cross-encoder reranking — when a single entity-index lookup would answer it. The 111 ms p50 is inflated by phases that contribute no signal for this query class.

High-complexity queries hit a ceiling. Multi-hop CTI attribution (for example, "which threat actor used CVE-2024-3400 in campaigns that targeted healthcare between Q3 2024 and Q1 2025?") requires crossing 3+ entity boundaries with temporal constraints. The current top-k blend (default k=10) with depth-2 graph traversal cannot guarantee coverage of the full causal chain. Notes that are individually low-similarity but collectively form the answer are dropped before reranking. This explains the 25% accuracy gap on CTI and the long-tail p95 on LOCOMO.

The 5-way intent classifier already identifies these query classes but only weights traversal policy, not depth.

Proposal¶

Introduce a Quality Gate on recall() that routes queries to one of two retrieval depths:

System 1 — the current blended-retrieval pipeline (fast, unchanged)
System 2 — a new exhaustive-scan path with pre-filtering and temporal-chunk scoring

The gate is deterministic, LLM-free on the fast path, and uses heuristic signals from the already-computed intent classification and entity extraction plus one additional cheap probe (vector top-1 score). A D-Mem (arXiv:2603.18631) finding grounds the sizing: routing roughly 3% of queries to System 2 recovers 96.7% of full-deliberation accuracy while preserving System 1 latency for the other 97%.

Architecture¶

recall(query, ...)
  │
  ▼
IntentClassifier.classify(query)        # Existing — unchanged, <1ms
  │
  ▼
QueryProfile                            # New — cheap feature bundle
  ├─ intent + confidence
  ├─ query entities (from EntityExtractor)
  ├─ span markers (temporal, causal, multi-hop)
  └─ aggregation markers
  │
  ▼
QualityGate.route(profile) ──────┐      # New — deterministic, <5ms
  │                              │
  │ System 1                     │ System 2
  ▼                              ▼
System1Retriever          System2Retriever
= current recall() body   = candidate pre-filter
  (unchanged)               + temporal chunking
  │                         + chunk scoring
  │                         + within-chunk rerank
  │                              │
  └─── results ── upgrade? ──────┘
  │   (if System 1 score floor breached)
  ▼
List[MemoryNote]

Gate design: additive bounded scoring¶

The gate computes a score ∈ [0.0, 1.0] from cheap features. Queries above system2_threshold (default 0.70) route to System 2; the rest stay on System 1.

Signal	Weight
CAUSAL intent prior	+0.35
RELATIONAL intent prior	+0.20
EXPLORATORY intent prior	+0.20
TEMPORAL intent prior	+0.10
Ambiguous intent (confidence < 0.4)	+0.15
Multi-hop span markers	+0.25
Temporal span markers	+0.20
Causal chain markers	+0.25
Aggregation markers	+0.30
Comparison markers	+0.15
Entity count ≥ 3	+0.10
Entity type count ≥ 2	+0.05

A fast-path shortcut forces System 1 unconditionally when: FACTUAL intent + entity count ≤ 1 + no multi-hop markers + no aggregation markers.

System 2 pipeline¶

Where System 1 scores and ranks individual notes, System 2 scores temporal chunks to catch "collectively true, individually weak" patterns.

QueryProfile
  │
  ▼
1. CandidateFilter (entity-index expansion + 1-hop KG neighbors
                    + time-window expansion + vector top-200 fallback)
  │ target: 200–500 candidates
  ▼
2. TemporalChunker (sort by created_at, bucket by hourly|daily|weekly)
  │
  ▼
3. ChunkScorer (coverage 0.30 + density 0.20 + causal 0.20
               + centrality 0.15 + recency 0.15)
  │ select top-M chunks (default 5)
  ▼
4. Within-chunk rerank (cross-encoder on selected chunks only,
                        same as System 1 but bounded to ~100 pairs)
  │
  ▼
List[MemoryNote] (deduped, capped at k)

Upgrade path¶

After System 1 completes, a second check can escalate to System 2 without re-running the gate:

Top vector score falls below system1_confidence_floor (default 0.40), or
CAUSAL/EXPLORATORY query returns fewer than 3 results

A config knob max_upgrade_rate (default 15%) caps the fraction of queries that upgrade in a rolling 100-query window. Chronic upgrades indicate the threshold is set too conservatively.

Public API: no change¶

recall(query, domain, k, include_links, exclude_superseded, include_expired) keeps its current signature and return type. Gate decisions appear in structured logs as gate_depth, gate_score, and gate_reasons.

Config¶

Ships disabled by default. To opt in after upgrading to the version that delivers this RFC:

# config.yaml
recall:
  depth_routing:
    enabled: true

    quality_gate:
      system2_threshold: 0.70        # fraction [0,1]; lower → more System 2
      system1_confidence_floor: 0.40 # upgrade if top score < this
      max_upgrade_rate: 0.15         # cap upgrade rate in rolling window

    system2:
      max_candidates: 500
      chunk_size: auto               # auto | hourly | daily | weekly
      max_chunks: 5
      chunk_weights:
        coverage: 0.30
        density: 0.20
        causal: 0.20
        centrality: 0.15
        recency: 0.15

New storage primitives¶

System 2 requires three new read methods on StorageBackend:

def get_entity_neighbors(
    self, etype: str, evalue: str, hop: int = 1, max_notes: int = 100,
) -> list[MemoryNote]: ...

def iterate_notes_in_range(
    self, start: str, end: str,
) -> Iterator[MemoryNote]: ...

def get_entity_degree(self, etype: str, evalue: str) -> int: ...

All three are SQLite queries on existing indexes. No schema migrations required.

Adversarial review¶

An adversarial review completed on 2026-04-16 returned REQUEST CHANGES with 4 blockers and 13 warnings.

Blockers (unresolved as of this writing)¶

B1. Motivating example misroutes to System 1. The query "which threat actor used CVE-2024-3400 in campaigns targeting healthcare between Q3 2024 and Q1 2025?" scores 0.40 under the proposed weights — below the 0.70 threshold. The pattern \bused by .+ (?:to|in|for|against)\b does not match "used CVE". The query that RFC-003 was designed to fix routes to the path it already fails on.

B2. meta["top_vector_score"] does not exist. The upgrade path reads this key but VectorRetriever.retrieve() currently returns List[MemoryNote]. Delivering the upgrade check requires a VectorRetriever signature change, which the RFC presents as a "verbatim lift" with no side effects.

B3. System 2 will bust the 800 ms budget if LLM NER is used. The candidate filter runs entity extraction across 200–500 notes. If extract_all() calls the LLM NER path (the current default in recall()), that is 50–200 ms per note — well above the total budget. The RFC does not pin use_llm=False for candidate-note extraction.

B4. Two of three storage primitives require indexes that are not stated to exist. iterate_notes_in_range assumes an index on created_at; get_entity_degree assumes an incidence-count index. Without these indexes, Phase 2 regresses under a full-scan fallback.

Decision¶

Status: Draft — changes requested. Implementation deferred pending a revised RFC that resolves the blockers above. See the adversarial review document for the full list of warnings and recommended fixes.

Migration¶

No migration needed. This RFC has not shipped. recall() behavior in v2.7.0 is unchanged from v2.6.x. When the implementation lands, the gate ships disabled by default (enabled: false); existing deployments require no config changes to preserve current behavior.

Alternatives considered¶

Alternative	Verdict
LLM-based router	Rejected — adds 300–500 ms to every query; non-deterministic; LLM-free fast path is an explicit constraint
Learned classifier router	Rejected for this RFC — no labeled dataset exists; deferred to a follow-up RFC once telemetry accumulates
Always-System-2	Rejected — 7× latency regression on the dominant query class
Depth as explicit `recall()` argument	Rejected as sole mechanism — agents cannot reliably decide depth without gate heuristics
Deeper graph traversal in System 1	Rejected — traversal cost is exponential; does not address "collectively true, individually weak" pattern