Extend ZettelForge with custom LLM providers

ZettelForge is designed to work without optional packages. The public extension path today is the LLM provider registry. Use this guide to package a provider, register it through Python entry points, and test it without changing ZettelForge's source.

You use this path when you want ZettelForge to call your own local model, OpenAI-compatible gateway, hosted inference service, or lab backend through the same generate() path that built-in providers use.

Prerequisites

• ZettelForge installed: pip install zettelforge
• Python 3.10+

---

Part 1: Build a custom LLM provider

ZettelForge's LLM layer is pluggable. You can register a third-party provider that the core generate() function delegates to, without modifying ZettelForge's source.

How the provider system works

At startup, zettelforge.llm_providers does two things:

1. Registers the built-in providers: local (llama-cpp-python), ollama, mock, and litellm (if installed).
2. Scans the zettelforge.llm_providers entry-point group and registers any additional providers found there.

Load failures in the entry-point scan are logged at DEBUG and never crash startup.

Step 1: Implement the LLMProvider protocol

Your class must implement zettelforge.llm_providers.base.LLMProvider, a typing.Protocol:

from zettelforge.llm_providers.base import LLMProviderConfigurationError


class MyProvider:
    name = "myprovider"

    def generate(
        self,
        prompt: str,
        max_tokens: int = 400,
        temperature: float = 0.1,
        system: str | None = None,
        json_mode: bool = False,
    ) -> str:
        if not prompt.strip():
            return ""
        if max_tokens < 1:
            raise LLMProviderConfigurationError("max_tokens must be positive")
        return "example response"

LLMProvider uses duck typing — no inheritance required. The name attribute is the registry key users reference in config.

For setup failures, raise LLMProviderConfigurationError rather than returning empty. This surfaces misconfiguration at startup instead of silently failing per call.

Step 2: Package and register the entry point

In your package's pyproject.toml:

[project.entry-points."zettelforge.llm_providers"]
myprovider = "myprovider_package:MyProvider"

After pip install of your package, ZettelForge discovers and registers your provider at startup. Users select it by setting llm.provider = "myprovider" in config.yaml.

Step 3: Handle optional SDK dependencies

If your provider depends on an SDK that is not always installed, lazy-import it:

from zettelforge.llm_providers.base import LLMProviderConfigurationError


class MyProvider:
    name = "myprovider"

    def __init__(self) -> None:
        self._client = None

    def _ensure_client(self) -> None:
        if self._client is not None:
            return
        try:
            import mysdk
            self._client = mysdk.Client()
        except ImportError as exc:
            raise LLMProviderConfigurationError(
                "myprovider requires mysdk. Install with: pip install myprovider-package"
            ) from exc

    def generate(self, prompt: str, **kwargs: object) -> str:
        self._ensure_client()
        return self._client.generate(prompt, **kwargs)

This keeps ZettelForge's core install free of your SDK's dependencies.

Step 4: Test your provider

Use reset_registrations() between tests to prevent state leakage:

from zettelforge.llm_providers.registry import (
    available,
    get,
    register,
    reset_registrations,
)


class MyProvider:
    name = "myprovider"

    def generate(self, prompt: str, **kwargs: object) -> str:
        return f"handled: {prompt}"


def test_my_provider_registered():
    reset_registrations()
    register("myprovider", MyProvider)
    assert "myprovider" in available()


def test_my_provider_generate():
    reset_registrations()
    register("myprovider", MyProvider)
    p = get("myprovider")
    result = p.generate("test prompt")
    assert result == "handled: test prompt"

---

What is not a public extension API

ZettelForge contains internal compatibility hooks that let the core vary behavior when SaaS-backed capabilities are present. Treat those hooks as implementation details, not as a plugin system for user packages.

The public boundary is:

• Use zettelforge.llm_providers entry points for custom LLM backends.
• Do not create a [project.entry-points."zettelforge.extensions"] group; the source does not scan it.
• Do not rely on a package-name prefix to trigger discovery. Installing a package whose name starts with zettelforge_ has no effect by itself.
• If your integration needs a capability that only exists in ThreatRecall.ai, call the ThreatRecall.ai API or design your local code to degrade cleanly.