YARA schema reference

Modules: zettelforge.yara, zettelforge.yara.parser, zettelforge.yara.entities, zettelforge.yara.tags, zettelforge.yara.ingest

ZettelForge 2.7.0 — Apache-2.0 license

from zettelforge.yara import (
    YaraRule, YaraParseError,
    parse_yara, parse_file,
    rule_to_entities, resolve_yara_tag,
    ingest_rule, ingest_rules_dir,
)

---

What this reference covers

The YARA subsystem has three layers:

Layer	Module	Responsibility
Parser	yara.parser	Wrap plyara, normalize output to a stable dict shape
Entity	yara.entities / yara.tags	Map a parsed rule dict to YaraRule + KG edges
Ingest	yara.ingest	Orchestrate parse → entity → remember → persist

This page covers all three layers. The CLI wraps the ingest layer; see CLI.

---

Vendored CCCS schemas

Two CCCS YARA metadata schema files are vendored in src/zettelforge/yara/schemas/:

File	Purpose
CCCS_YARA.yml	Field definitions, formats, and optional flags for all CCCS metadata keys
CCCS_YARA_values.yml	Allowed value sets for status, sharing, category, malware_type, actor_type, and hash

These files are a clean-room re-implementation; the upstream CCCS validator_functions.py is not vendored. See schemas/NOTICE.md.

ZettelForge loads both files eagerly at import time. A missing or unreadable schema file raises at startup.

---

Parser

parse_yara(text: str) → list[dict[str, Any]]

Parse YARA source text into a list of normalized rule dicts. A single .yar file may contain multiple rules; one dict per rule.

Uses a module-level plyara.Plyara instance protected by a threading lock (_PARSER_LOCK). The parser is reset before and after each call so state does not leak between invocations.

Each returned dict contains:

Key	Type	Source
rule_name	str	plyara
tags	list[str]	plyara (inline rule tags, e.g. rule Foo : APT)
imports	list[str]	plyara (YARA import statements)
metadata	list[dict]	plyara raw (list of single-key dicts)
meta	dict	added by parser — flattened from metadata, last-write-wins for duplicate keys
strings	list[dict]	plyara
condition_terms	list[str]	plyara
raw_condition	str	plyara
raw_strings	str	plyara
raw_meta	str	plyara
raw_rule	str	added by parser — exact source text carved from start_line/stop_line
start_line	int	plyara (1-indexed, inclusive)
stop_line	int	plyara

parse_file(path: str | Path) → list[dict[str, Any]]

Parse a .yar or .yara file. Raises YaraParseError when:

• the file cannot be stat'd (I/O error),
• the file exceeds MAX_RULE_FILE_BYTES (1 048 576 bytes, 1 MiB), or
• plyara raises on a syntax error (re-raised as ValueError).

rules = parse_file("rules/apt41.yar")
for rule in rules:
    print(rule["rule_name"], rule["meta"].get("status"))

YaraParseError

ValueError subclass raised when a rule file is rejected before it reaches plyara (I/O error, oversize). Syntax errors from plyara are re-raised as plain ValueError.

from zettelforge.yara.parser import YaraParseError, parse_file

try:
    rules = parse_file(path)
except YaraParseError as e:
    print(f"rejected: {e}")
except ValueError as e:
    print(f"syntax error: {e}")

---

CCCS metadata validation

Validation tiers

ZettelForge validates each rule's metadata block against the vendored CCCS schema at ingest time. Three tiers control strictness:

Tier	Behaviour
"warn" (default)	Required-field failures and value violations are recorded as warnings. accepted is always True.
"strict"	Failures on required fields produce errors and set accepted = False; the ingest path drops the rule.
"non_cccs"	All checks skipped. Returns (True, [], []) unconditionally.

Required CCCS fields

These fields are optional: No in CCCS_YARA.yml and must be present in strict tier. In warn tier, their absence is recorded as a warning but does not block ingest.

Field	Format	Validated values
status	uppercase string	TESTING, RELEASED, DEPRECATED
sharing	TLP label	TLP:CLEAR, TLP:GREEN, TLP:AMBER, TLP:AMBER+STRICT (each also allows //COMMERCIAL suffix)
source	string	Any non-empty string; uppercase is convention
author	string	[A-Za-z0-9_.@+-]+
description	string	Any non-empty string
category	uppercase string	INFO, EXPLOIT, TECHNIQUE, TOOL, MALWARE

Auto-generated fields

These fields are optional: Optional in CCCS_YARA.yml. The upstream CCCS validator auto-generates them when absent; ZettelForge does not auto-generate but treats them as required in strict tier to preserve provenance.

Field	Format
id	Base62 UUID (16+ chars)
fingerprint	Hex digest (SHA-1 or SHA-256, 40–64 chars)
version	x.y
modified	YYYY-MM-DD

Optional CCCS fields

All other fields in CCCS_YARA.yml are optional: Yes. They are validated when present but their absence never produces a warning or error.

Field	Type	Description
date	string (YYYY-MM-DD)	Creation date
score	integer (0–100)	Confidence percentage
minimum_yara	string (x.y)	Minimum YARA version required
malware_type	uppercase string	Malware capability (see malware types)
mitre_att	string	MITRE ATT&CK ID(s): T####, T####.###, TA####, G####, S####; comma- or semicolon-separated list accepted
actor	string	Threat actor name
actor_type	uppercase string	APT, CRIMEWARE, or FIN
technique	string	CCCS technique tag (freeform; distinct from mitre_att)
hash	string	Sample hash: 32-char (MD5), 40-char (SHA-1), or 64-char (SHA-256) hex
report	string	Linked report URL or reference
reference	string	External reference (URL, report, individual)
vol_script	string	Reverse-engineering team metadata
al_*	string	Assembly Line internal metadata
credit	string	Author credit note
original_*	string	Preserved metadata from rule conversions

validate_metadata(rule_meta, tier) → ValidationResult

from zettelforge.yara.cccs_metadata import validate_metadata, ValidationResult

result: ValidationResult = validate_metadata(meta_dict, tier="warn")
# result.accepted  — True unless strict tier produced errors
# result.warnings  — list[str]
# result.errors    — list[str]

Malware type values

Valid values for malware_type (validated when present):

ADWARE, APT, BACKDOOR, BANKER, BOOTKIT, BOT, BROWSER-HIJACKER, BRUTEFORCER, CLICKFRAUD, CRYPTOMINER, DDOS, DOWNLOADER, DROPPER, EXPLOITKIT, FAKEAV, HACKTOOL, INFOSTEALER, KEYLOGGER, LOADER, OBFUSCATOR, POS, PROXY, RAT, RANSOMWARE, REVERSE-PROXY, ROOTKIT, SCANNER, SCAREWARE, SPAMMER, TROJAN, VIRUS, WIPER, WEBSHELL, WORM

---

YaraRule dataclass

YaraRule extends DetectionRule with YARA-specific fields.

DetectionRule base fields (inherited)

Field	Type	Description
rule_id	str	Primary identifier. CCCS id when present; otherwise yara_<content_hash[:16]>
title	str	Rule name (rule_name from parser)
source_format	str	Always "yara"
content_sha256	str	SHA-256 of the raw rule text (when available) or of rule_name + strings + condition
description	str \| None	From CCCS description meta
author	str \| None	From CCCS author meta
date	str \| None	From CCCS date meta (YYYY-MM-DD)
modified	str \| None	From CCCS modified meta (YYYY-MM-DD)
references	list[str]	Populated from meta["report"] when present
tags	list[str]	Inline YARA tags (grammar-level, e.g. rule Foo : APT MAL)
level	str \| None	Not populated by YARA ingest (Sigma concept)
status	str \| None	Lowercased CCCS status value
tlp	str \| None	From CCCS sharing meta
license	str \| None	Not populated by YARA ingest
source_repo	str \| None	Not populated by YARA ingest
source_path	str \| None	Not populated by YARA ingest
extra	dict[str, Any]	See extra keys

YaraRule-specific fields

Field	Type	Description
rule_name	str \| None	Raw YARA rule name (same as title)
cccs_id	str \| None	CCCS id meta value
fingerprint	str \| None	CCCS fingerprint meta (SHA-256 over strings + condition)
category	str \| None	CCCS category: INFO, EXPLOIT, TECHNIQUE, TOOL, or MALWARE
technique_tag	str \| None	CCCS technique meta value (freeform; distinct from MITRE IDs)
cccs_version	str \| None	CCCS version meta value, coerced to string
hash_of_sample	list[str]	CCCS hash values (one or more sample hashes)
is_private	bool	True when the rule has the private modifier
is_global	bool	True when the rule has the global modifier
imports	list[str]	YARA import module names (e.g. "pe", "hash", "dotnet")
condition	str \| None	Raw condition expression, stripped of leading/trailing whitespace

Extra keys

Key	Type	Description
cccs_compliant	str	Tier the rule was tagged with: "strict", "warn", or "non_cccs"
cccs_warnings	list[str]	Warnings from metadata validation
cccs_errors	list[str]	Errors from metadata validation
condition_terms	list[str]	Individual condition tokens from plyara
source_line_range	[int, int]	[start_line, stop_line] from plyara
source	str	CCCS source org (when present)
malware_type	str	CCCS malware_type (when present)
actor_type	str	CCCS actor_type (present when actor_type is given but no actor name)

---

rule_to_entities(rule, *, tier) → (YaraRule, list[dict])

Convert a parsed YARA rule dict into a YaraRule entity and a list of KG edges.

from zettelforge.yara.entities import rule_to_entities
from zettelforge.yara.parser import parse_yara

rules = parse_yara(yara_text)
entity, relations = rule_to_entities(rules[0], tier="warn")

KG edge shape

Every relation dict uses the canonical edge shape expected by SQLiteMemoryStore.add_kg_edge:

{
    "from_type": "YaraRule",
    "from_value": entity.rule_id,   # str
    "rel":        str,              # see relation types below
    "to_type":    str,              # entity type of the target
    "to_value":   str,              # primary identifier of the target
    "properties": dict,            # additional relation properties
}

Relation types

Relation	to_type	Trigger	to_value
detects	AttackPattern	CCCS mitre_att meta (one edge per technique ID)	Normalized MITRE ID, e.g. T1218, T1218.001
detects	AttackPattern	Inline tag matches T#### or attack.T####	MITRE technique ID
attributed_to	ThreatActor	CCCS actor meta	Actor name string
tagged_with	YaraTag	CCCS technique meta	Technique name (namespace "technique")
tagged_with	YaraTag	Inline tag (category token or freeform)	Tag name
references_cve	Vulnerability	Inline tag matches CVE_YYYY_NNNN or CVE-YYYY-NNNN	Normalized CVE-YYYY-NNNN

Tag resolution — resolve_yara_tag(tag) → (entity_type, entity_properties)

Inline YARA grammar tags (the colon-separated tokens after the rule name) are resolved before they become KG edges:

Tag pattern	Resolved type	Example
T#### or attack.T#### (case-insensitive)	AttackPattern	T1218 → {"technique_id": "T1218"}
CVE_YYYY_NNNN or CVE-YYYY-NNNN (case-insensitive)	Vulnerability	CVE_2021_44228 → {"cve_id": "CVE-2021-44228"}
Known category token	YaraTag	APT → {"namespace": "category", "name": "APT"}
Anything else	YaraTag	custom_tag → {"namespace": "freeform", "name": "custom_tag"}

Known category tokens (case-insensitive match): APT, CRIME, CRIMEWARE, EXPL, HKTL, MAL, MALWARE, PUA, RAT, RANSOM, RANSOMWARE, SUSP, VULN, WEBSHELL.

---

Ingest API

ingest_rule(rule_source, mm, *, domain, tier, sync) → (MemoryNote | None, list[dict])

Ingest a single YARA rule into a MemoryManager instance.

Parameter	Type	Default	Description
rule_source	str \| Path \| dict	required	Path to a .yar file, raw YARA text, or a pre-parsed plyara dict. When a file contains multiple rules, only the first is ingested.
mm	MemoryManager	required	Must not be None; raises ValueError otherwise.
domain	str	"detection"	Memory domain for the note.
tier	str	"warn"	CCCS validation tier.
sync	bool	True	Run MemoryManager enrichment inline.

Returns (None, relations) when strict-tier validation rejects the rule. relations is always a list (empty when the rule has no extractable entities).

Idempotency. Before calling mm.remember, the ingest path checks for an existing note with source_ref = "yara:{rule_id}:{content_sha256[:12]}". If found, the rule is skipped and the existing note is returned.

ingest_rules_dir(path, mm, *, glob, tier, domain, bulk, flush_timeout) → dict

Walk a directory tree and ingest every YARA rule file.

result = ingest_rules_dir("rules/", mm=mm, tier="warn", domain="detection")
# result == {"ingested": int, "skipped": int, "errors": list[str]}

Parameter	Type	Default	Description
path	str \| Path	required	Root directory to walk.
mm	MemoryManager	required	Must not be None.
glob	str	"**/*.yar"	Glob pattern for rule files. The default also sweeps for **/*.yara.
tier	str	"warn"	CCCS validation tier.
domain	str	"detection"	Memory domain for notes.
bulk	bool	False	Defer enrichment and flush once after all rules. Pass True for large rule sets.
flush_timeout	float \| None	None	Timeout in seconds passed to mm.flush() when bulk=True.

Security. Symlinks are never followed. Files whose resolved path escapes the root directory are skipped with a warning log.

---

CLI

python -m zettelforge.yara.ingest <path> [--tier TIER] [--dry-run] [--domain DOMAIN] [--json]

Flag	Default	Description
path	required	Path to a .yar file or directory
--tier	warn	strict, warn, or non_cccs
--dry-run	off	Parse, validate, and print a summary without writing to memory
--domain	detection	Memory domain for ingested notes
--json	off	Emit machine-readable JSON output

Exit codes: 0 on success; 1 on parse errors, strict-tier rejections, or I/O failures.

Dry-run a directory:

python -m zettelforge.yara.ingest tests/fixtures/yara/ --dry-run

Ingest with strict CCCS validation:

python -m zettelforge.yara.ingest rules/ --tier strict --domain detection

The LLM rule explainer is not invoked by this CLI command in v1. Explainer integration with the async enrichment worker is a v1.1 task.