Skip to content

Configure YARA rule ingestion

Use this guide to preview and import .yar and .yara files into ZettelForge. The ingest path stores one memory note per rule, records detection relations in the knowledge graph, and skips an unchanged rule when you import it again.

Prerequisites

  • ZettelForge 2.8.0 installed and configured. Start with the Quickstart if you do not have a working instance.
  • One or more YARA files in a directory such as rules/.
  • Write access to the directory you use for AMEM_DATA_DIR.

The commands below use ZettelForge's bundled technique_loader.yar fixture. Replace that path with your own rule file.

1. Preview a rule

Parse and validate the rule without writing to memory:

python3 -m zettelforge.yara.cli rules/technique_loader.yar --dry-run

Observed output:

[warn] MemoryModule  (technique_loader.yar)  relations=2  mitre=T1218
    warn: missing required CCCS field: id
    warn: missing required CCCS field: fingerprint
    warn: missing required CCCS field: modified

The default warn tier accepts a parseable rule and reports missing or invalid CCCS metadata. Use --tier strict when missing required CCCS fields must reject the rule, or --tier non_cccs when you intentionally import standard YARA without CCCS validation.

The preview reports two relations for this fixture: one ATT&CK technique relation for T1218 and one YARA technique tag. The YARA schema reference defines the full relation mapping and CCCS fields.

CLI dry-run symlink refusal is in progress

ZettelForge 2.8.0 live ingestion refuses symlinked rule files, but the CLI directory dry-run path follows matching symlinks. Preview only directories you trust. For untrusted directory trees, use ingest_rules_dir() as shown below; it skips symlinks and files that resolve outside the selected root.

2. Ingest the directory

Disable optional enrichment for a deterministic rule import, then ingest every .yar and .yara file under rules/:

AMEM_DATA_DIR="$PWD/.zettelforge-data" \
ZETTELFORGE_ENRICHMENT_ENABLED=false \
python3 -m zettelforge.yara.cli rules/

Observed output for a directory containing the fixture:

ingested=1 skipped=0 errors=0

The YARA CLI does not call the rule explainer. Disabling enrichment keeps the surrounding MemoryManager write path offline and prevents background LLM work during this import. Vector embedding still runs when ZettelForge stores the note.

3. Verify ingestion from Python

Use the Python API when you need the stored note, extracted relations, bulk controls, or the directory safety boundary in your own automation:

from zettelforge import MemoryManager
from zettelforge.yara import ingest_rule, ingest_rules_dir

mm = MemoryManager()

note, relations = ingest_rule(
    "rules/technique_loader.yar",
    mm,
    tier="warn",
    sync=False,
)
second, _ = ingest_rule(
    "rules/technique_loader.yar",
    mm,
    tier="warn",
    sync=False,
)

print(f"created={note is not None}")
print(f"relations={sorted({relation['rel'] for relation in relations})}")
print(f"idempotent={second.id == note.id}")

result = ingest_rules_dir(
    "rules",
    mm,
    tier="warn",
    bulk=True,
    flush_timeout=60.0,
)
print(f"ingested={result['ingested']} skipped={result['skipped']}")
print(f"errors={result['errors']}")

Observed output after the CLI and Python API use separate empty data directories:

created=True
relations=['detects', 'tagged_with']
idempotent=True
ingested=0 skipped=1
errors=[]

The second single-file call returns the existing note because the rule ID and content hash produce the same source_ref. The directory call then counts that unchanged rule as skipped. Bulk mode defers per-rule enrichment and calls MemoryManager.flush() once after the directory pass.

Troubleshooting

The preview prints [warn]. The rule is accepted under the default tier. Add the named CCCS fields before you switch to --tier strict.

A strict preview exits with status 1. At least one rule failed CCCS validation or parsing. Fix the reported fields, or choose warn or non_cccs deliberately.

The import prints a FastEmbed model-update warning. Check the final summary. ingested=1 skipped=0 errors=0 confirms that the write completed in this verification run.

A rule file is rejected before parsing. ZettelForge limits each YARA file to 1,048,576 bytes. Split a larger collection into smaller files.