Module inventory¶

Use this page to find the ZettelForge module that owns a behavior before you read, extend, or debug the code. This inventory covers the immutable ZettelForge v2.8.0 release tag; paths are relative to the repository root.

Prerequisites: You have the ZettelForge v2.8.0 source tree. For installation and first use, start with Quickstart.

At a glance¶

Metric	Verified value
Python files under `src/zettelforge/`	107
Package directories with `__init__.py`	17
Public exports in `zettelforge.__all__`	29
Root-level Python files under `web/`	3
JavaScript files under `web/ui/`	17
Most-imported package module	`zettelforge.log` — 48 dependents

The 107 package files divide into 42 top-level modules and 65 modules in the detection, integration, LLM-provider, MCP, OSINT, script, Sigma, and YARA subtrees. The web files are outside that package count.

Top-level package modules¶

Public entry points and configuration¶

Module	Responsibility
`src/zettelforge/__init__.py`	Defines package exports and `__version__`.
`src/zettelforge/__main__.py`	Defines the package CLI `main()` entry point.
`src/zettelforge/demo.py`	Runs the interactive CTI memory demonstration.
`src/zettelforge/config.py`	Loads configuration into the `ZettelForgeConfig` model and its nested config models.
`src/zettelforge/edition.py`	Preserves legacy optional-extension detection helpers. These helpers do not define another product: the products are ZettelForge OSS and ThreatRecall.ai SaaS.
`src/zettelforge/extensions.py`	Loads and queries optional extension packages.

Memory and storage¶

Module	Responsibility
`src/zettelforge/memory_manager.py`	Provides the primary `MemoryManager` interface for remember, recall, synthesis, graph, and enrichment operations.
`src/zettelforge/storage_backend.py`	Defines the `StorageBackend` abstract contract and `BackendClosedError`.
`src/zettelforge/sqlite_backend.py`	Implements the storage contract with SQLite and WAL mode.
`src/zettelforge/backend_factory.py`	Selects and creates the configured storage backend.
`src/zettelforge/memory_store.py`	Provides JSONL note and graph persistence utilities.
`src/zettelforge/vector_memory.py`	Manages LanceDB-backed semantic memory and embedding generation.
`src/zettelforge/enrichment_ledger.py`	Defines immutable records for deferred enrichment jobs.
`src/zettelforge/lance_maintenance.py`	Prunes old LanceDB versions and reports reclaimed storage.

Retrieval and graph¶

Module	Responsibility
`src/zettelforge/vector_retriever.py`	Searches memory notes by vector similarity.
`src/zettelforge/graph_retriever.py`	Traverses graph relationships and returns scored note results.
`src/zettelforge/blended_retriever.py`	Combines vector and graph retrieval results.
`src/zettelforge/intent_classifier.py`	Classifies query intent for adaptive retrieval routing.
`src/zettelforge/knowledge_graph.py`	Stores and traverses JSONL knowledge-graph nodes and edges.
`src/zettelforge/ontology.py`	Defines the typed entity and relationship system plus ontology validation.
`src/zettelforge/entity_indexer.py`	Extracts entities and maintains the entity-to-note index.
`src/zettelforge/alias_resolver.py`	Resolves canonical entities and their aliases.

Note construction and lifecycle¶

Module	Responsibility
`src/zettelforge/note_schema.py`	Defines `MemoryNote` and its nested Pydantic models.
`src/zettelforge/note_constructor.py`	Builds enriched memory notes from source content.
`src/zettelforge/fact_extractor.py`	Extracts discrete facts in phase one of the write pipeline.
`src/zettelforge/memory_updater.py`	Chooses add, update, delete, or no-op in phase two of the write pipeline.
`src/zettelforge/memory_evolver.py`	Refines neighboring memories and tracks contradictions or supersession.
`src/zettelforge/consolidation.py`	Detects semantic shift and consolidates related notes.

Synthesis and LLM dispatch¶

Module	Responsibility
`src/zettelforge/synthesis_generator.py`	Produces retrieval-augmented answers from recalled notes.
`src/zettelforge/synthesis_validator.py`	Validates synthesized output, confidence, and source attribution.
`src/zettelforge/llm_client.py`	Dispatches text and structured generation to the configured LLM provider.
`src/zettelforge/json_parse.py`	Removes reasoning tags and extracts JSON from LLM output.

Governance, resilience, and observability¶

Module	Responsibility
`src/zettelforge/governance_validator.py`	Applies write-time governance validation.
`src/zettelforge/pii_validator.py`	Detects or redacts PII when the optional Presidio dependencies are installed.
`src/zettelforge/prompt_injection_guard.py`	Detects prompt-injection and retrieval-poisoning patterns.
`src/zettelforge/memory_defense.py`	Scores candidate memories for anomalous content at write time.
`src/zettelforge/retry.py`	Provides configurable retry and backoff behavior.
`src/zettelforge/cache.py`	Provides an in-memory cache with TTL and entry limits.
`src/zettelforge/log.py`	Configures structured logging and returns bound loggers.
`src/zettelforge/ocsf.py`	Emits OCSF v1.3 audit events.
`src/zettelforge/telemetry.py`	Records recall and synthesis quality telemetry.
`src/zettelforge/observability.py`	Collects operation metrics through `Observability` and `timed_operation`.

LLM provider package¶

The seven files under src/zettelforge/llm_providers/ isolate provider-specific behavior behind one contract.

File	Responsibility
`__init__.py`	Exports provider package symbols.
`base.py`	Defines the `LLMProvider` protocol.
`local_provider.py`	Implements in-process GGUF generation through llama-cpp-python.
`ollama_provider.py`	Implements generation through the Ollama HTTP API.
`litellm_provider.py`	Routes requests through LiteLLM.
`mock_provider.py`	Supplies deterministic responses for tests.
`registry.py`	Registers providers by name with thread-safe lookup.

Detection and rule-ingestion packages¶

Shared detection package¶

File	Responsibility
`src/zettelforge/detection/__init__.py`	Exports shared detection types.
`src/zettelforge/detection/base.py`	Defines the `DetectionRule` dataclass shared by Sigma and YARA.
`src/zettelforge/detection/explainer.py`	Generates an LLM explanation for a normalized rule.
`src/zettelforge/detection/consumers.py`	Defines the deferred detection-match consumer protocol.

Sigma package¶

The seven Sigma Python modules are sigma/__init__.py, cli.py, entities.py, ingest.py, parser.py, tags.py, and schemas/__init__.py. They expose the CLI and Python ingest paths, parse and validate YAML, map rules to typed entities, and resolve Sigma tags. Vendored JSON schemas and their notice are data files, not Python modules.

YARA package¶

The eight YARA Python modules are yara/__init__.py, cccs_metadata.py, cli.py, entities.py, ingest.py, parser.py, tags.py, and schemas/__init__.py. They expose the CLI and Python ingest paths, parse YARA through plyara, validate CCCS metadata, map rules to typed entities, and resolve tags. Vendored YAML schemas and their notice are data files, not Python modules.

OSINT package¶

Core modules¶

File	Responsibility
`src/zettelforge/osint/__init__.py`	Exports OSINT types and registers the released collectors.
`src/zettelforge/osint/ontology.py`	Defines OSINT entity types, relationships, and canonicalizers.
`src/zettelforge/osint/entity_resolver.py`	Normalizes entity keys and manages aliases.
`src/zettelforge/osint/executor.py`	Executes registered transforms and writes results to the knowledge graph.
`src/zettelforge/osint/investigation.py`	Scopes enrichment data to named investigations.
`src/zettelforge/osint/transform_registry.py`	Stores collector metadata and callable registrations.

Collector modules and feature state¶

The collector tree contains seven package __init__.py files: one at collectors/ and one in each of breach/, financial/, infrastructure/, people/, social/, and tech/. The remaining 16 modules are below.

Collector module	Released source state
`infrastructure/dns_collector.py`	Implements DNS and reverse-DNS collection.
`infrastructure/cert_collector.py`	Implements Certificate Transparency collection.
`infrastructure/whois_collector.py`	Implements domain and IP WHOIS collection.
`infrastructure/bgp_collector.py`	Implements passive BGP enrichment.
`infrastructure/port_scanner.py`	Implements an active nmap path gated by `ZETTELFORGE_OSINT_ACTIVE_SCAN`; it returns no results unless you opt in and install its dependencies.
`breach/hibp_collector.py`	Implements HIBP REST lookup; it fails closed without `HIBP_API_KEY`.
`breach/breach_directory.py`	In progress: the registered collector is a stub.
`financial/wallet_collector.py`	Implements EVM transaction lookup through Etherscan; it fails closed without `ETHERSCAN_API_KEY`.
`people/maigret_collector.py`	Implements optional username lookup through Maigret; it returns no results when the dependency is absent.
`people/holehe_collector.py`	Disabled no-op because its upstream dependency has an incompatible license.
`people/hunter_collector.py`	In progress: the registered collector is a stub.
`people/namechk_collector.py`	In progress: the registered collector is a stub.
`tech/builtwith_collector.py`	In progress: the registered collector is a stub.
`tech/wappalyzer_collector.py`	In progress: the registered collector is a stub.
`social/hashtag_tracker.py`	In progress: the registered collector is a stub.
`social/twitter_collector.py`	In progress: the registered collector is a stub.

This inventory verifies module presence, registration, source branches, and the released focused test suite. It does not claim that external OSINT services returned live data during this audit.

Integration and MCP packages¶

Integrations¶

File	Responsibility
`src/zettelforge/integrations/__init__.py`	Exports integration package symbols.
`src/zettelforge/integrations/langchain_retriever.py`	Adapts recall to the LangChain retriever interface.
`src/zettelforge/integrations/crewai.py`	Adapts ZettelForge operations to CrewAI tools.

MCP¶

File	Responsibility
`src/zettelforge/mcp/__init__.py`	Exports MCP package symbols.
`src/zettelforge/mcp/__main__.py`	Defines the package entry point for the MCP server.
`src/zettelforge/mcp/server.py`	Implements JSON-RPC 2.0 over standard input and output.

Operator scripts¶

The release contains four Python modules under src/zettelforge/scripts/ and no __init__.py in that directory.

File	Responsibility
`compact_lance.py`	Runs one-shot LanceDB compaction.
`human_eval_sampler.py`	Samples items for human evaluation.
`telemetry_aggregator.py`	Aggregates daily telemetry records.
`telemetry_dashboard.py`	Runs the Streamlit telemetry dashboard.

Web layer¶

The web application lives outside src/zettelforge/ and is not included in the 107-module package count.

Root Python file	Responsibility
`web/app.py`	FastAPI application with 21 `/api/*` route decorators and two HTML route decorators.
`web/auth.py`	API-key validation and loopback access rules.
`web/mcp_server.py`	MCP adapter for web deployment.

The 17 JavaScript files under web/ui/ contain one bootstrap file, two library files, six components, and eight views. The UI uses the checked-in stylesheets web/static/css/design_tokens.css and web/ui/colors_and_type.css; it has no generated JavaScript bundle in the release tree.

Dependency map¶

Use these seams to choose where a change belongs:

memory_manager
├── backend_factory → storage_backend → sqlite_backend
├── vector_retriever → vector_memory, entity_indexer
├── graph_retriever → knowledge_graph
├── blended_retriever → vector_retriever, graph_retriever
├── synthesis_generator → llm_client
├── note_constructor → entity_indexer, note_schema
├── fact_extractor → llm_client, json_parse
├── memory_updater → note_schema
├── governance_validator → pii_validator, prompt_injection_guard, memory_defense
├── enrichment_ledger → deferred enrichment job records
└── log → structured events across 48 dependent modules

Start in memory_manager.py when you trace a user-visible memory operation. Move down one seam to change a storage, retrieval, synthesis, validation, or enrichment implementation without widening the change across unrelated layers.