Extend ZettelForge with custom LLM providers

ZettelForge is designed to work without optional packages. The public extension path today is the LLM provider registry. Use this guide to package a provider, register it through Python entry points, and test it without changing ZettelForge's source.

This guide is verified against ZettelForge 2.8.0.

You use this path when you want ZettelForge to call your own local model, OpenAI-compatible gateway, hosted inference service, or lab backend through the same generate() path that built-in providers use.

Prerequisites

• ZettelForge 2.8.0 installed; follow the Quickstart
• Python 3.10+

---

Build a custom LLM provider

ZettelForge's LLM layer is pluggable. You can register a third-party provider that the core generate() function delegates to, without modifying ZettelForge's source.

How the provider system works

When Python imports zettelforge.llm_providers, the module does two things:

1. Registers the built-in provider names: local (llama-cpp-python or ONNX Runtime GenAI), ollama, mock, and litellm.
2. Scans the zettelforge.llm_providers entry-point group and registers any additional providers found there.

The litellm name is registered even when the optional LiteLLM SDK is absent. Calling that provider without the SDK raises an actionable ImportError; install it with pip install "zettelforge[litellm]" before use.

Third-party load failures are logged at DEBUG and do not interrupt the import. Provider names are unique: a third-party entry point cannot replace a registered built-in provider.

Step 1: Implement the LLMProvider protocol

Your class must satisfy zettelforge.llm_providers.base.LLMProvider, a typing.Protocol:

from zettelforge.llm_providers.base import LLMProviderConfigurationError


class MyProvider:
    name = "myprovider"

    def generate(
        self,
        prompt: str,
        max_tokens: int = 400,
        temperature: float = 0.1,
        system: str | None = None,
        json_mode: bool = False,
    ) -> str:
        if not prompt.strip():
            return ""
        if max_tokens < 1:
            raise LLMProviderConfigurationError("max_tokens must be positive")
        return "example response"

LLMProvider uses duck typing — no inheritance required. The name attribute is the registry key users reference in config.

For setup failures, raise LLMProviderConfigurationError rather than returning an empty string. The public generate() path propagates this error to the caller instead of hiding the invalid configuration.

Step 2: Package and register the entry point

In your package's pyproject.toml:

[project.entry-points."zettelforge.llm_providers"]
myprovider = "myprovider_package:MyProvider"

After you install your package, ZettelForge discovers the entry point when it imports zettelforge.llm_providers. Select the provider in config.yaml:

llm:
  provider: myprovider

Step 3: Handle optional SDK dependencies

If your provider depends on an SDK that is not always installed, lazy-import it:

from zettelforge.llm_providers.base import LLMProviderConfigurationError


class MyProvider:
    name = "myprovider"

    def __init__(self) -> None:
        self._client = None

    def _ensure_client(self) -> None:
        if self._client is not None:
            return
        try:
            import mysdk
            self._client = mysdk.Client()
        except ImportError as exc:
            raise LLMProviderConfigurationError(
                "myprovider requires mysdk. Install with: pip install myprovider-package"
            ) from exc

    def generate(self, prompt: str, **kwargs: object) -> str:
        self._ensure_client()
        return self._client.generate(prompt, **kwargs)

This keeps ZettelForge's core install free of your SDK's dependencies.

Step 4: Test your provider

Use reset_registrations() between tests to prevent state leakage:

from zettelforge.llm_providers.registry import (
    available,
    get,
    register,
    reset_registrations,
)


class MyProvider:
    name = "myprovider"

    def generate(self, prompt: str, **kwargs: object) -> str:
        return f"handled: {prompt}"


def test_my_provider_registered():
    reset_registrations()
    register("myprovider", MyProvider)
    assert "myprovider" in available()


def test_my_provider_generate():
    reset_registrations()
    register("myprovider", MyProvider)
    p = get("myprovider")
    result = p.generate("test prompt")
    assert result == "handled: test prompt"

---

What is not a public extension API

ZettelForge contains internal compatibility hooks that let the core vary behavior when SaaS-backed capabilities are present. Treat those hooks as implementation details, not as a plugin system for user packages.

The public boundary is:

• Use zettelforge.llm_providers entry points for custom LLM backends.
• Do not create a [project.entry-points."zettelforge.extensions"] group; the source does not scan it.
• Do not rely on a package-name prefix to trigger discovery. Installing a package whose name starts with zettelforge_ has no effect by itself.
• If your integration needs a capability that only exists in ThreatRecall.ai, call the ThreatRecall.ai API or design your local code to degrade cleanly.