RFC-003: Read-Path Depth Routing

Field	Value
Author	Patrick Roland
Status	Draft — adversarial review completed, changes requested
Created	2026-04-16
Last updated	2026-07-27
ZettelForge version	v2.7.0 (design only; not implemented)
Related RFCs	RFC-001 Conversational Entity Extractor · RFC-002 Universal LLM Provider Interface

Not implemented

RFC-003 is a design proposal. Re-verified on 2026-07-27 against ZettelForge v2.7.0: none of the proposed modules (quality_gate.py, query_profile.py, query_markers.py, retrieval/) exist in the codebase and config.default.yaml has no recall.depth_routing section. The current recall() runs the uniform System 1 pipeline on every query. This page documents the proposal and its review status.

Context

recall() applies the same retrieval pipeline to every query regardless of complexity. The v2.7.0 benchmark report records these historical baseline numbers from a DGX Spark ARM64 run:

• CTI single-entity lookups: p50 111 ms, p95 ~200 ms, ~75% accuracy
• LOCOMO multi-hop long-horizon: p50 1,240 ms, p95 2,282 ms
• Intent classification (keyword path): < 1 ms

Two failure modes follow from these numbers.

Low-complexity queries overpay. A query like CVE-2024-1234 triggers the full pipeline — intent classification, entity extraction, vector retrieval, graph traversal at depth 2, entity-augmented pull, and cross-encoder reranking — when a single entity-index lookup would answer it. The historical CTI p50 is inflated by phases that contribute no signal for this query class.

High-complexity queries hit a ceiling. Multi-hop CTI attribution (for example, "which threat actor used CVE-2024-3400 in campaigns that targeted healthcare between Q3 2024 and Q1 2025?") requires crossing 3+ entity boundaries with temporal constraints. The current top-k blend (default k=10) with depth-2 graph traversal cannot guarantee coverage of the full causal chain. Notes that are individually low-similarity but collectively form the answer are dropped before reranking. This explains the 25% accuracy gap on CTI and the long-tail p95 on LOCOMO.

The 5-way intent classifier already identifies these query classes but only weights traversal policy, not depth.

Proposal

Introduce a Quality Gate on recall() that routes queries to one of two retrieval depths:

• System 1 — the current blended-retrieval pipeline (fast, unchanged)
• System 2 — a new exhaustive-scan path with pre-filtering and temporal-chunk scoring

The gate is deterministic, LLM-free on the fast path, and uses heuristic signals from the already-computed intent classification and entity extraction plus one additional cheap probe (vector top-1 score). A D-Mem (arXiv:2603.18631) finding grounds the approach: its multi-dimensional quality gate recovers 96.7% of full-deliberation performance while reducing computational cost.

Architecture

recall(query, ...)
  │
  ▼
IntentClassifier.classify(query)        # Existing — unchanged, <1ms
  │
  ▼
QueryProfile                            # New — cheap feature bundle
  ├─ intent + confidence
  ├─ query entities (from EntityExtractor)
  ├─ span markers (temporal, causal, multi-hop)
  └─ aggregation markers
  │
  ▼
QualityGate.route(profile) ──────┐      # New — deterministic, <5ms
  │                              │
  │ System 1                     │ System 2
  ▼                              ▼
System1Retriever          System2Retriever
= current recall() body   = candidate pre-filter
  (unchanged)               + temporal chunking
  │                         + chunk scoring
  │                         + within-chunk rerank
  │                              │
  └─── results ── upgrade? ──────┘
  │   (if System 1 score floor breached)
  ▼
List[MemoryNote]

Gate design: additive bounded scoring

The gate computes a score ∈ [0.0, 1.0] from cheap features. Queries above system2_threshold (default 0.70) route to System 2; the rest stay on System 1.

Signal	Weight
CAUSAL intent prior	+0.35
RELATIONAL intent prior	+0.20
EXPLORATORY intent prior	+0.20
TEMPORAL intent prior	+0.10
Ambiguous intent (confidence < 0.4)	+0.15
Multi-hop span markers	+0.25
Temporal span markers	+0.20
Causal chain markers	+0.25
Aggregation markers	+0.30
Comparison markers	+0.15
Entity count ≥ 3	+0.10
Entity type count ≥ 2	+0.05

A fast-path shortcut forces System 1 unconditionally when: FACTUAL intent + entity count ≤ 1 + no multi-hop markers + no aggregation markers.

System 2 pipeline

Where System 1 scores and ranks individual notes, System 2 scores temporal chunks to catch "collectively true, individually weak" patterns.

QueryProfile
  │
  ▼
1. CandidateFilter (entity-index expansion + 1-hop KG neighbors
                    + time-window expansion + vector top-200 fallback)
  │ target: 200–500 candidates
  ▼
2. TemporalChunker (sort by created_at, bucket by hourly|daily|weekly)
  │
  ▼
3. ChunkScorer (coverage 0.30 + density 0.20 + causal 0.20
               + centrality 0.15 + recency 0.15)
  │ select top-M chunks (default 5)
  ▼
4. Within-chunk rerank (cross-encoder on selected chunks only,
                        same as System 1 but bounded to ~100 pairs)
  │
  ▼
List[MemoryNote] (deduped, capped at k)

Upgrade path

After System 1 completes, a second check can escalate to System 2 without re-running the gate:

• Top vector score falls below system1_confidence_floor (default 0.40), or
• CAUSAL/EXPLORATORY query returns fewer than 3 results

A config knob max_upgrade_rate (default 15%) caps the fraction of queries that upgrade in a rolling 100-query window. Chronic upgrades indicate the threshold is set too conservatively.

Public API: no change

recall(query, domain, k, include_links, exclude_superseded, include_expired) keeps its current signature and return type. Gate decisions appear in structured logs as gate_depth, gate_score, and gate_reasons.

Config

Ships disabled by default. To opt in after upgrading to the version that delivers this RFC:

# config.yaml
recall:
  depth_routing:
    enabled: true

    quality_gate:
      system2_threshold: 0.70        # fraction [0,1]; lower → more System 2
      system1_confidence_floor: 0.40 # upgrade if top score < this
      max_upgrade_rate: 0.15         # cap upgrade rate in rolling window

    system2:
      max_candidates: 500
      chunk_size: auto               # auto | hourly | daily | weekly
      max_chunks: 5
      chunk_weights:
        coverage: 0.30
        density: 0.20
        causal: 0.20
        centrality: 0.15
        recency: 0.15

New storage primitives

System 2 requires three new read methods on StorageBackend:

def get_entity_neighbors(
    self, etype: str, evalue: str, hop: int = 1, max_notes: int = 100,
) -> list[MemoryNote]: ...

def iterate_notes_in_range(
    self, start: str, end: str,
) -> Iterator[MemoryNote]: ...

def get_entity_degree(self, etype: str, evalue: str) -> int: ...

The proposal intends these to be SQLite read queries over existing note, KG, and entity indexes. The revised RFC still needs to prove the exact query plans before keeping the "no schema migrations required" claim.

Adversarial review

An adversarial review completed on 2026-04-16 returned REQUEST CHANGES with 4 blockers and 13 warnings.

Open review findings after the 2026-07-27 re-audit

B1. Motivating example misroutes to System 1. The query "which threat actor used CVE-2024-3400 in campaigns targeting healthcare between Q3 2024 and Q1 2025?" scores 0.40 under the proposed weights — below the 0.70 threshold. The pattern \bused by .+ (?:to|in|for|against)\b does not match "used CVE". The query that RFC-003 was designed to fix routes to the path it already fails on.

B2. meta["top_vector_score"] does not exist. The upgrade path reads this key, but no current API exposes that metadata object. VectorRetriever.retrieve() can return (MemoryNote, score) tuples when called with return_scores=True; the revised RFC still needs to define where the top score is carried into the post-System-1 upgrade check.

B3. System 2 will bust the 800 ms budget if LLM NER is used. Current recall() query extraction calls extract_all() with its default use_llm=False, so the shipped read path is regex-only. The proposed System 2 candidate filter still needs to pin candidate-note extraction to regex-only behavior so future implementation does not call the LLM NER path across 200–500 notes.

B4. The storage contract is still underspecified. SQLite now has indexes on notes.created_at, KG edge endpoints, and entity lookup columns, but the three proposed StorageBackend methods do not exist. The revised RFC needs to specify exact queries and any incidence-count index required for get_entity_degree().

Decision

Status: Draft — changes requested. Implementation remains deferred pending a revised RFC that resolves the findings above. See the adversarial review document for the full list of warnings and recommended fixes.

Migration

No migration is needed today because this RFC has not shipped. The verified v2.7.0 recall() path remains the uniform blended retrieval path. When the implementation lands, the gate ships disabled by default (enabled: false); existing deployments require no config changes to preserve current behavior.

Alternatives considered

Alternative	Verdict
LLM-based router	Rejected — adds 300–500 ms to every query; non-deterministic; LLM-free fast path is an explicit constraint
Learned classifier router	Rejected for this RFC — no labeled dataset exists; deferred to a follow-up RFC once telemetry accumulates
Always-System-2	Rejected — 7× latency regression on the dominant query class
Depth as explicit recall() argument	Rejected as sole mechanism — agents cannot reliably decide depth without gate heuristics
Deeper graph traversal in System 1	Rejected — traversal cost is exponential; does not address "collectively true, individually weak" pattern