Passive OSINT enrichment¶

Use ZettelForge's OSINT executor to run registered passive collectors for a single seed entity: a domain, IP address, ASN, or IP block. The executor canonicalizes the seed, invokes matching collectors, validates emitted tuples against the ontology, and writes verified nodes and edges into the knowledge graph when persistence is enabled.

Live collector output is in progress

The executor, registry, canonicalization, validation, dry-run mode, and safety gates are verified in ZettelForge 2.7.0. Live network collectors are best-effort: a collector can return no tuples or record tuple validation errors without aborting the run. Inspect error_count and errors before you rely on a result.

Prerequisites¶

ZettelForge 2.7.0 installed and initialized.
For DNS, WHOIS, and certificate transparency collectors, install the OSINT extra with pip install "zettelforge[osint]". The [osint] extra adds dnspython, python-whois, and ipwhois. The base package already includes httpx, so the BGP collector can run without this extra.
Network egress for collectors that contact public services such as DNS resolvers, crt.sh, WHOIS, or BGPView.

Supported seed types¶

The executor accepts five seed types: DomainName, IPv4Address, IPv6Address, ASNumber, and Netblock. Seed type must match exactly — the executor raises ValueError for any other value.

Collectors registered per seed type:

Seed type	Registered collectors	Current state
`DomainName`	`cert_collector`, `dns_collector`, `whois_collector`, `builtwith_collector`, `wappalyzer_collector`	DNS, WHOIS, and cert collectors run passive lookups. BuiltWith and Wappalyzer are registered stubs and return empty results today.
`IPv4Address`	`port_scanner`, `whois_collector`	WHOIS runs passive lookup. `port_scanner` is active-gated and returns empty results unless scanning is explicitly enabled.
`IPv6Address`	`port_scanner`, `whois_collector`	Same as IPv4.
`ASNumber`	`bgp_collector`	Uses BGPView when reachable; returns empty results on HTTP, DNS, or parse failure.
`Netblock`	None	Valid seed type, but no collectors are registered today.

You can inspect the registry from Python:

from zettelforge import osint as _osint  # imports collector registrations
from zettelforge.osint import SUPPORTED_SEED_TYPES
from zettelforge.osint.transform_registry import get_transform_registry

registry = get_transform_registry()
for seed_type in SUPPORTED_SEED_TYPES:
    names = [meta.name for meta, _ in registry.find_by_input(seed_type)]
    print(seed_type, names)

Run a dry-run smoke test¶

Run the executor with an empty collector allow-list when you want to verify canonicalization and result handling without network calls or knowledge-graph writes:

from zettelforge.osint import run_osint_collection

result = run_osint_collection(
    "ASNumber",
    "AS15169",
    persist=False,
    collector_names=[],
)

print(result.canonical_input_value)  # 15169
print(result.seed_node_id)           # None
print(result.collectors_run)         # []
print(result.persisted_count)
print(result.error_count)

Input values are canonicalized before any KG writes: "Example.COM." becomes "example.com", "AS15169" becomes "15169". Alternate spellings of the same entity collapse onto the same KG node.

The module also exposes collect_osint(), a compatibility wrapper that delegates to run_osint_collection().

Run a live passive collector¶

Use persist=False first so you can inspect collector output before writing to the knowledge graph:

from zettelforge import osint as _osint  # imports collector registrations
from zettelforge.osint import run_osint_collection

result = run_osint_collection(
    "DomainName",
    "example.com",
    persist=False,
    collector_names=["dns_collector"],
)

print(result.collectors_run)
print(result.tuples_collected)
print(result.error_count)
for error in result.errors:
    print(error.collector_name, error.message)

Collector output depends on the current public records and on network reachability. Treat nonzero error_count as part of the result, not as an exception path; the executor continues with the remaining collectors.

Result shape¶

run_osint_collection returns an OSINTCollectionResult dataclass:

Field	Type	Notes
`input_entity_type`	`str`	Seed type you passed
`input_value`	`str`	Seed value you passed
`canonical_input_value`	`str`	Canonicalized form
`seed_node_id`	`str \\| None`	KG node ID for the seed (`None` when `persist=False`)
`collectors_run`	`list[str]`	Names of collectors that ran
`tuples_collected`	`int`	Total tuples emitted across all collectors
`persisted`	`list[PersistedOSINTTuple]`	Validated tuples written to the KG
`persisted_count`	`int`	Tuples actually written to the KG
`error_count`	`int`	Collector or tuple errors (non-fatal)
`errors`	`list[OSINTExecutionError]`	Per-collector error detail
`started_at`	`str`	ISO-8601 timestamp
`finished_at`	`str \\| None`	ISO-8601 timestamp

A failing collector does not abort the run. Its error is captured in result.errors and execution continues with the remaining collectors.

Dry-run without KG writes¶

Pass persist=False to run collectors and validate output without writing to the knowledge graph. Use this mode for new seeds and for any collector that depends on public network data.

Run only specific collectors¶

Use collector_names to restrict which collectors run for a given seed. The live collector example above runs only dns_collector.

Any name not in the registry is silently skipped — no error is raised for an unknown collector name.

Safety controls¶

Active port scanning is off by default. Without an explicit opt-in, the port scanner collector returns an empty result and sends no packets:

export ZETTELFORGE_OSINT_ACTIVE_SCAN=1

Set this variable only when you own the target network or have written authorization to scan it. Accepted values: 1, true, TRUE, yes. Anything else (including an empty string) keeps scanning off.

Even with the flag set, the port scanner requires python-nmap and the nmap binary. If either is missing, the collector returns an empty result and logs a warning instead of raising.

What gets persisted¶

The executor writes canonical entity values to the knowledge graph. Duplicate spellings collapse onto the same node. For example, AS15169 and 15169 resolve to the same canonical ASN node (15169), and Example.COM. and example.com write to the same domain node.

Each persisted tuple also writes a KG edge with an osint: true property and a source property set to the collector name, so you can later query which collector produced a given edge.

Additional enrichment is in progress

Breach, social, and people enrichment collectors use seed types such as EmailAddress, Alias, and Hashtag that are not in SUPPORTED_SEED_TYPES today. Passing those types raises ValueError. Support for those seed types is in progress.