# Ertir Docs — full text > User documentation for Ertir (formerly TheYgent) — a self-hosted, no-code, local-first platform for building and running AI agents on a visual canvas, with built-in local inference engines (llama.cpp, MLX, vLLM) or any OpenAI-compatible API. Every page of the user documentation, in navigation order. Browsable at https://docs.ertir.dev/latest --- Source: https://docs.ertir.dev/latest/ # Ertir !!! note "Ertir is the new name of TheYgent" Same project, same code, same license. Technical names such as the `THEYGENT_*` environment variables and the `~/.theygent` folder are unchanged, so existing installs keep working. See [the name change](name-change.md). Ertir (formerly TheYgent) is a no-code, local-first platform for building and running AI agents. You compose agents visually on a canvas, chat with them or wire them to triggers, and run them against models on your own machine — or against any OpenAI-compatible API you choose to register. ## Your models, your machines, your choice Ertir's promise is honest and specific: **everything runs where you point it, and you own every hop.** - **Local by default.** Models download to your machine and run on local engines — llama.cpp anywhere, MLX on Apple Silicon, vLLM on CUDA GPUs. Weights, the model registry, and credentials live in local files under `~/.theygent/inference`; agents, runs, and conversations live in a Postgres database you control. - **Hosted APIs only when you add them.** Any OpenAI-compatible server or hosted API can be registered as a model, with its API key stored on your machine as a local credential that is never sent anywhere else. - **The choice is per agent and per call.** Every model gets a logical id (for example `triage-fast`), and agents reference that id — never an engine. You can point one node at a small local model and the next at a hosted one, or swap a model's backing engine later without touching a single agent. There is no vendor in the middle: Ertir never proxies your prompts, payloads, or weights through anyone's servers. It has no telemetry or update checks, and once installed it runs fully offline — [Privacy and data flow](concepts/privacy.md) lists every connection it makes, and when. ## What you can do - **Build agents visually** — drag nodes onto a canvas, connect them with edges, configure them in an inspector, and run them without writing code. See the [editor tour](building/editor.md) and the [node reference](building/nodes/index.md). - **Chat** — a unified chat surface for talking to any registered model or published agent, with streaming answers, a collapsible "thinking" view for reasoning models, voice and image input, and automatic session history. See [chat](chat/index.md). - **Run models locally or remotely** — browse and install models from Hugging Face with fit-to-your-RAM badges, or register a hosted API; engines start lazily and are managed for you. See [models](models/index.md) and [models and engines](concepts/models-and-engines.md). - **Extend agents with tools** — built-in tools, HTTP calls, and any MCP (Model Context Protocol) server, running in your own trust domain: install one from a public hub, run a local process, connect a remote endpoint (with OAuth sign-in when it needs one), or generate tools straight from an OpenAPI spec or GraphQL endpoint. See [MCP servers](mcp/index.md). - **Give agents your documents** — upload files or crawl a docs site into a retrieval source; agents search it with hybrid (semantic + keyword) retrieval, embedded by a model on your own inference plane and stored in your own Postgres. See [RAG sources](rag/index.md). - **Automate** — deploy published agents behind token-authenticated invoke endpoints, HMAC-signed webhooks, and cron schedules. See [triggers](running/triggers.md). - **Run durably** — opt a run onto the durable runtime so it checkpoints each step, survives a crash, and can pause for human approval. See [durable runs](running/durable.md) and [human in the loop](running/human-in-the-loop.md). - **Observe everything** — every run gets a zoomable span waterfall with per-node inputs, outputs, and timing, plus optional OTLP export. See [observability](running/observability.md). - **Benchmark** — test any model or agent from the Bench: per-turn latency and token metrics, saved results, side-by-side comparisons. See [the Bench](running/bench.md). ## Watch it run Screen recordings of the real product, if you would rather see it than read it first. Each one also has a page on the site with its chapters and a written summary: [ertir.dev/videos](https://ertir.dev/videos/). | Video | What it covers | |---|---| | [Self-Hosted AI Agent Orchestration](https://www.youtube.com/watch?v=3LE_VrIbe0I) (23:52) | The whole platform end to end — clone, start the services, install a model, build and run an agent. Also on [Getting started](getting-started/index.md). | | [Voice In, Voice Out](https://www.youtube.com/watch?v=leoPQLaWMeU) (5:54) | A voice agent: speech-to-text, a model, and text-to-speech on the canvas. Also on [Voice](chat/voice.md). | | [Run Local + Remote Models in One Agent](https://www.youtube.com/watch?v=eHIHABKbwVg) (7:01) | One graph routing between a local model and a hosted API. Also on [Remote models](models/remote.md). | | [Give Your LLM a Memory: RAG with File Upload & Web Crawl](https://www.youtube.com/watch?v=jFc4on-kH_w) (9:34) | A retrieval source from an uploaded PDF and one from crawling a docs site, wired into an agent as tools. See [RAG sources](rag/index.md). | | [How to Deploy Ertir to Kubernetes](https://www.youtube.com/watch?v=A0kMmTAr-L4) (6:41) | Build and load the images, apply the manifests, port-forward, and sign in. | ## How it fits together Ertir runs as four local services, brought up together with `make up`: ```mermaid graph LR UI[Interface :5174] --> CP[Control plane :8080] UI --> IP[Inference plane :8081] CP --> IP CP --> PG[(Postgres)] IP --> ENG[Local engines: llama.cpp / MLX / vLLM] IP --> API[Any OpenAI-compatible API you register] CC[Claude Code] --> AG[AI gateway :8082] AG --> ANT[Anthropic API] AG -->|local models| IP AG -->|metadata only| CP ``` - The **interface** is the web app where you build, chat, and observe. - The **control plane** orchestrates agents: it stores agent versions, executes graphs, records runs and sessions, and handles triggers. It talks to models only over HTTP. - The **inference plane** is yours alone: it holds the model registry, downloads weights, spawns and supervises local engine servers, and proxies to any remote API you register. When you chat directly with a model, raw media (voice recordings, images) goes to it straight from your browser; in an agent run — which the control plane orchestrates — that media is uploaded to the control plane as an artifact and handed to nodes by reference. - The **AI gateway** sits between your team's Claude Code and its models to cut what it costs: it records what each exchange cost and what routing would save, then — when you choose — routes work to cheaper Claude models, and to models you run yourself in the shares you set. See [AI Gateway](ai-gateway/index.md). - The **MCP server** gives coding agents — Claude Code, Codex, OpenCode — Ertir as tools: offload work to your local models, run and build agents, and search your documents and code. See [Use Ertir from coding agents](coding-agents/index.md). The [architecture page](concepts/index.md) explains the split and exactly where each kind of data lives. ## Where to next | If you want to… | Go to | |---|---| | Install Ertir and get it running | [Getting started](getting-started/index.md) | | Install a model and have your first conversation | [First chat](getting-started/first-chat.md) | | Build, test, and publish your first agent | [First agent](getting-started/first-agent.md) | | Understand agents, graphs, runs, and versioning | [Concepts](concepts/index.md) | | Look up what every node does | [Node reference](building/nodes/index.md) | | Use Ertir from Claude Code, Codex or OpenCode | [Coding agents](coding-agents/index.md) | --- Source: https://docs.ertir.dev/latest/name-change/ # The name change: TheYgent is now Ertir Ertir is the new name of TheYgent. It is the same project: the same code, the same maintainer and the same license ([Sustainable Use License](https://github.com/al2m4n/ertir/blob/main/LICENSE.md)). Version 2.0.0 is the first release under the new name. ## What changed - **The name.** The app, these docs and the website say Ertir. - **The website and docs addresses.** The website is [ertir.dev](https://ertir.dev) and the docs are at [docs.ertir.dev](https://docs.ertir.dev). Links to `theygent.ai` and `docs.theygent.ai` redirect to the matching page on the new addresses. - **The GitHub repository.** It is [github.com/al2m4n/ertir](https://github.com/al2m4n/ertir). GitHub redirects the old `al2m4n/TheYgent` URLs, so existing clones, issues and pull requests keep working. ## What stays the same - **Your installation and your data.** Agents, versions, runs, conversations, models, credentials and settings are untouched. There is nothing to migrate. - **Technical names.** They keep the old spelling, so scripts and configuration you already have keep working: | What | Name | |------|------| | Environment variables | `THEYGENT_*` (see [environment variables](reference/environment.md)) | | Data folder | `~/.theygent` | | Webhook signature header | `X-TheYgent-Signature` | | AI gateway key header | `X-TheYgent-Gateway-Key` | | Python packages and modules, npm packages, Docker and Kubernetes names | `theygent…` | If a later release renames any of these, its release notes say how to switch, and the old names keep working during the change. --- Source: https://docs.ertir.dev/latest/getting-started/ # Getting started Ertir is a local-first platform for building and running AI agents: three small services run on your own machine, and everything — models, agents, runs, chat history — lives where you put it. This section takes you from a fresh clone to your first working agent in about ten minutes. ## Watch the walkthrough
Self-Hosted AI Agent Orchestration (23:52) — cloning the repository, bringing the stack up, installing a model, and building, running, and tracing an agent. The pages below are the written version of the same path.
## What you need | Requirement | Why | |---|---| | **Python 3.12** | Runs the two backend services. The repo pins the version, and `uv` manages it for you. | | **uv** on `PATH` | The Python package manager. Installs dependencies and the MLX engine servers. | | **pnpm** on `PATH` (with Node.js) | Runs the visual interface. The repo pins pnpm `11.8.0`; Ertir is developed against Node 22. | | **PostgreSQL** | The control plane stores agents, runs, and sessions in Postgres. You supply a running server — Ertir does not provision one. | | **Homebrew** (macOS) | `make engines` uses it to install llama.cpp, whisper.cpp, and ffmpeg. | !!! note "Bring your own Postgres" Any way you already run Postgres works — a native install, a container, a managed instance. All Ertir needs is a database it can reach at the `DATABASE_URL` connection string you set during [installation](installation.md). ### Hardware and platform notes Which local model engines you can run depends on your machine: - **Apple Silicon Mac** — the best-covered platform. `make engines` installs everything: llama.cpp (chat, embeddings, vision), whisper.cpp (speech-to-text), and the MLX servers (chat, vision, text-to-speech). MLX only runs on Apple Silicon. - **Linux, or a Mac without Apple Silicon** — llama.cpp and whisper.cpp run anywhere the binaries exist, including CPU-only machines. Install `llama-server`, `whisper-server`, and `ffmpeg` with your package manager; the MLX servers are unavailable. - **A machine with an NVIDIA GPU** — vLLM is the engine for CUDA hosts. It is deliberately not part of `make engines`; install it on the GPU host with `pip install vllm`. - **No local model at all** — that works too. You can register any OpenAI-compatible server or hosted API as a [remote model](../models/remote.md) and build agents against it. Missing engines are never fatal: the inference plane reports exactly which engine/modality slots your machine can run, and the model catalog only offers models you can actually execute. See [Models and engines](../concepts/models-and-engines.md) for the full picture. ## The three services `make up` starts three local processes: | Service | Port | What it does | |---|---|---| | Inference plane | `8081` | Runs your local model engines and proxies remote APIs. Owns the model registry and the model catalog. | | Control plane | `8080` | Agents, runs, sessions, triggers, MCP servers, observability. Backed by Postgres. | | Interface | `5174` | The web UI you work in. Your browser talks to both planes directly. | ```mermaid graph LR B[Browser
interface :5174] --> CP[Control plane :8080] B --> IP[Inference plane :8081] CP -->|OpenAI-compatible HTTP| IP CP --> PG[(Postgres)] IP --> E[Local engine servers] ``` Everything runs where *you* point it: model weights download to `~/.theygent/inference/models/` on your machine, the model registry stays in a local file, and the control plane's data sits in your own Postgres. Nothing reaches a hosted API unless you explicitly register one — you own every hop. All services bind to localhost by default. Accounts are local too: the first time you open the interface, a short wizard creates the install's admin account, and any further [users, roles, and sign-in providers](../running/users.md) are yours to manage. Once the services are up, you work entirely in your browser at `http://localhost:5174`. Read more in [Architecture](../concepts/index.md). ## The ten-minute path 1. **[Installation](installation.md)** — clone the repo, create `.env`, install the engines and dependencies. 2. **[Running Ertir](running.md)** — `make up`, then confirm everything is healthy with `make status` and `/readyz`. 3. **[Your first chat](first-chat.md)** — install a small model from the Registries page and talk to it in New Chat. 4. **[Your first agent](first-agent.md)** — build input → llm → output on the canvas, test it right there, and publish it as a named agent. --- Source: https://docs.ertir.dev/latest/getting-started/installation/ # Installation This page takes you from nothing to an installed Ertir: clone the repository, create your configuration, and install the local model engines and dependencies. When you are done here, move on to [Running Ertir](running.md) to start the services. ## Before you start Make sure the prerequisites from the [getting-started overview](index.md#what-you-need) are in place: **Python 3.12** (managed for you by `uv`), **uv** and **pnpm** on your `PATH`, a reachable **PostgreSQL** server, and — on macOS — **Homebrew** for the engine install. A quick sanity check: ```bash uv --version pnpm --version ``` !!! note "You supply Postgres" Ertir does not provision a database. Point it at any Postgres you already run — a native install, a container, or a managed instance. All it needs is a database it can reach at the `DATABASE_URL` you set below. ## 1. Clone the repository ```bash git clone https://github.com/al2m4n/ertir.git cd Ertir ``` Run every command below (and every `make` target) from this repository root. ## 2. Create your `.env` Copy the example file and open it: ```bash cp .env.example .env ``` The one value you **must** set is `DATABASE_URL` — an async Postgres connection string in the `postgresql+asyncpg://` form. The example ships with a sensible default: ```bash DATABASE_URL=postgresql+asyncpg://theygent:theygent@localhost:5432/theygent ``` The other keys already have working defaults; you rarely need to change them for a local install: | Key | Default | What it does | |---|---|---| | `DATABASE_URL` | *(the example DSN above)* | Postgres connection for the control plane. Migrations and the control plane fail loudly if this is unset or unreachable. | | `THEYGENT_INFERENCE_PLANE_URL` | `http://127.0.0.1:8081/v1` | Where the control plane reaches the inference plane. Must include the `/v1` suffix. | | `THEYGENT_INFERENCE_PLANE_HOST` / `_PORT` | `127.0.0.1` / `8081` | The inference plane's listen address. | | `THEYGENT_CONTROL_PLANE_HOST` / `_PORT` | `127.0.0.1` / `8080` | The control plane's listen address. | | `THEYGENT_MAX_RESIDENT` | `2` | How many local model engines may be loaded at once. | | `THEYGENT_CORS_ORIGINS` | localhost / 127.0.0.1 on `:5173` and `:5174` when unset | Browser origins allowed to call both planes. The shipped `.env.example` pins it to `http://localhost:5174`; leave it unless you serve the interface from another port. | The full list, including durable-mode and secret-store variables, is in the [environment reference](../reference/environment.md). ## 3. Install the local engines The local model engines (llama.cpp, whisper.cpp, ffmpeg, and the MLX servers) are separate binaries. On macOS, one target installs them: ```bash make engines ``` It is idempotent — rerun it after pulling new code. What it installs: - **`brew install llama.cpp whisper-cpp ffmpeg`** — `llama-server` (chat, embeddings, and vision), `whisper-server` (speech-to-text), and `ffmpeg` (converts microphone recordings from the browser). - **`uv tool install mlx-lm`** — MLX chat (Apple Silicon only). - **`uv tool install mlx-vlm`** — MLX vision (Apple Silicon only). - **`uv tool install mlx-audio` (with extras)** — MLX text-to-speech (Apple Silicon only). vLLM is deliberately **not** installed by this target — it belongs on an NVIDIA/CUDA host. === "Apple Silicon Mac" `make engines` installs the full set above. The MLX servers give you fast local chat, vision, and text-to-speech; llama.cpp and whisper.cpp cover the rest. === "Linux / other" `make engines` prints guidance instead of installing. Install `llama-server` (llama.cpp), `whisper-server` (whisper.cpp), and `ffmpeg` with your package manager. The MLX servers are unavailable off Apple Silicon. === "NVIDIA GPU host" Install vLLM directly on the GPU host with `pip install vllm`. It is not part of `make engines`. !!! tip "Engines are optional if you only use remote models" You can skip local engines entirely and register any OpenAI-compatible server or hosted API as a [remote model](../models/remote.md). To follow [Your first chat](first-chat.md), though, install at least one local engine. Which engine fits which machine is covered in [Engines](../models/engines.md). ## 4. Install dependencies and create the schema The recommended single command, [`make up`](running.md), installs dependencies, applies the database schema, and starts all four services in one step — so from a fresh clone with `.env` set and `make engines` done, you can jump straight to it. If you prefer to run the steps separately: ```bash make install # uv sync (Python) + pnpm install (interface) make migrate # apply the control-plane database schema ``` `make migrate` runs the database migrations against your `DATABASE_URL`; it stops with a clear error if that variable is unset. ## Where things land After installation, Ertir keeps your data in these places, all under your control: - **`~/.theygent/inference/`** — the local model registry, downloaded model weights (under `models/`), and local credentials for remote APIs. - **Your Postgres** (`DATABASE_URL`) — agents and their versions, runs, chat sessions, triggers, MCP registrations, connections, and observability traces. - **`~/.theygent/ai-gateway/`** — the [AI gateway](../ai-gateway/index.md)'s records not yet delivered, its last known configuration, and an organization key you set on it. - **`.run/`** — development log files and process ids (ignored by git). ## Next Head to [Running Ertir](running.md) to bring the services up and confirm they're healthy. --- Source: https://docs.ertir.dev/latest/getting-started/running/ # Running Ertir With the code [installed](installation.md), a single command brings up all four services. This page covers starting them, checking they're healthy, stopping them, and the one optional process — the durable worker — you'll only need for certain agents. ## Start everything: `make up` From the repository root: ```bash make up ``` `make up` runs three phases in order: 1. **install** — `uv sync` (Python dependencies) and `pnpm install` (interface dependencies). 2. **migrate** — apply the control-plane database schema to your `DATABASE_URL`. 3. **start** — launch the four services as background processes. From a fresh clone with `.env` set and `make engines` done, this is the only command you need. Once it finishes, open the interface in your browser: **[http://localhost:5174](http://localhost:5174)** On a fresh install the interface greets you with a **first-run wizard**: pick a username and password to create the install's admin account, and you land in the app signed in. Everything about accounts, roles, and single sign-on is covered in [Users, roles & sign-in](../running/users.md). ## The four services | Service | URL | What it does | |---|---|---| | Inference plane | `http://127.0.0.1:8081` | Runs your local model engines and proxies any remote APIs you register. Owns the model registry and the Hugging Face model catalog. | | Control plane | `http://127.0.0.1:8080` | Agents, runs, sessions, triggers, MCP servers, observability. Backed by your Postgres. | | Interface | `http://localhost:5174` | The web app you work in. Your browser talks to both planes directly. | | AI gateway | `http://127.0.0.1:8082` | Sits between Claude Code and the Anthropic API, recording what each exchange cost and what routing would save. As shipped it is a personal gateway for this machine; see [AI Gateway](../ai-gateway/index.md). | If a port is already taken, `make up` skips that service loudly rather than clobbering it — you'll see a message like `already on :8080 — skipping (use 'make restart')`. Process ids and logs land under `.run/` (`.run/inference-plane.log`, `.run/control-plane.log`, `.run/interface.log`, `.run/ai-gateway.log`). ```mermaid graph LR B[Browser · interface :5174] --> CP[Control plane :8080] B --> IP[Inference plane :8081] CP -->|OpenAI-compatible HTTP| IP CP --> PG[(Postgres)] IP --> E[Local engine servers] ``` ## Check status and logs ```bash make status # which services are up make logs # tail all four service logs ``` ## Health checks Both planes (and the AI gateway, on `:8082`) expose two endpoints: - **`GET /healthz`** — a plain liveness check (is the process up?). - **`GET /readyz`** — a readiness check. ```bash curl http://localhost:8081/readyz # inference plane curl http://localhost:8080/readyz # control plane ``` The **inference plane's** `/readyz` reports which `(engine, modality)` slots this machine can actually run — for example `llamacpp`, `mlx`, `mlx:vision`. A missing binary shows up here as not-ready with an install hint, so you learn about it now rather than as a surprise error on your first model call. The service stays ready as long as *any* engine is available. The **control plane's** `/readyz` checks both your Postgres and the inference plane; it fails if either is unreachable. ## Restart and stop ```bash make restart # stop, then start — use this to pick up backend code changes make down # stop all four services ``` ## When you need the worker (durable runs) Most agents run on the normal interactive path and need nothing beyond the services above. **Durable execution** — runs that checkpoint each step so they survive a crash and can pause for human input — is required for the `human`, `subgraph`, `loop`, and `map` node types, and for durable runs launched from the Bench (the `POST /agents/{id}/durable-runs` path). There are two ways to enable it: - **On your own machine:** set `THEYGENT_DURABLE=1` in your `.env` and restart. The control plane then runs the durable runtime in-process — no separate program. - **In a server or air-gapped deployment:** run the standalone worker, `uv run --package theygent-worker theygent-worker`. It is **not** started by `make up`. The durable runtime keeps its bookkeeping in a separate `dbos` schema inside the same Postgres, and migrates it automatically — there is no extra migration step to remember. Without durable mode, running a durable-only agent returns a `durable_required` error (the interface shows a note telling you to set `THEYGENT_DURABLE=1`). See [Durable runs](../running/durable.md) for the full story. ## Make targets at a glance | Target | Does | |---|---| | `make up` | install + migrate + start (the golden path) | | `make start` | start the four services (no install/migrate) | | `make restart` | stop, then start | | `make down` | stop all four services | | `make status` | show which services are up | | `make logs` | tail all four service logs | | `make install` | install Python + interface dependencies | | `make engines` | install the local model engines (macOS) | | `make migrate` | apply the database schema | ## Next Everything is up — now [install a model and have your first chat](first-chat.md). If something isn't healthy, see [Troubleshooting](../reference/troubleshooting.md). --- Source: https://docs.ertir.dev/latest/getting-started/first-chat/ # Your first chat The quickest way to see Ertir working is to install a small model and talk to it. This takes a few minutes once the services are up. **Before you start:** the three services should be [running](running.md) (`make status` to confirm), and you should have at least one local engine [installed](installation.md) with `make engines`. No local engine? You can register a [remote model](../models/remote.md) instead and skip straight to step 3. ## 1. Open the Registries page In the left sidebar, under **Configuration**, open **Registries**. This page lists the models registered in your inference plane and is where you add new ones. ## 2. Install a small model Click **Add model**, then **Browse Hugging Face →** to open the browse dialog. 1. **Search** for something small — for example `qwen`, `llama`, or `phi`. 2. Set the **Size** filter to **Small (<3B)**. Small, quantized models download fastest and load quickly. 3. The **Engines** chips are limited to engines your machine can actually run — discovery only shows models you can execute. (If no local engine is ready, the page tells you to install one first.) 4. Expand a card to see its **variants** (quantizations). Each carries a **fit badge** sized against your machine's RAM: **fits** (green), **tight** (amber), or **too large** (red). Choose a variant marked *recommended* that fits. 5. Click **Install**. Pick a **logical id** — the short name agents and chat will use to reference this model (one is suggested for you) — then **Download & install**. !!! tip "Or paste an id directly" In the **Hugging Face** tab you can paste a specific model id or URL — for example `mlx-community/Qwen2.5-0.5B-Instruct-4bit` — and press **Add** to jump straight to its variants. Download progress appears in the notification center (bottom-right): bytes transferred, speed, an ETA, and a **Cancel** button. It survives navigation, so you can move on while it runs. The weights download **inside your inference plane** — the bytes never pass through the control plane. When it finishes, the model shows up in the **Installed** table with the logical id you chose. ## 3. Start a chat In the sidebar, open **New Chat**. 1. Leave the **Model / Agent** toggle on **Model**. 2. In the searchable select, pick your model by its logical id. ## 4. Send a message Type in the composer at the bottom and press **Enter** to send (**Shift+Enter** inserts a newline). Your first message lazily starts the engine — the model loads on demand, so the first reply can take a little longer than the ones after it; the engine then stays warm. The answer **streams** in token by token. If you picked a reasoning ("thinking") model, its reasoning appears in a collapsible **Thinking…** block above the answer, which auto-collapses once the answer lands. A few things worth knowing: - The model is **pinned once the conversation starts** — use the **New chat** button to switch to a different model. - Every conversation is recorded as a **session**. Find it again under **Chats** in the sidebar, or in the **Recents** list below the navigation. - Each answer carries a **run** link that opens the full run detail with its execution trace. ## Where to next | You want to… | Go to | |---|---| | Build an agent instead of chatting with a raw model | [Your first agent](first-agent.md) | | Learn everything the chat surface can do (voice, images, thinking) | [Chat](../chat/index.md) | | Understand installing, browsing, and managing models | [Installing local models](../models/installing.md) | | A model won't appear or a call fails | [Troubleshooting](../reference/troubleshooting.md) | --- Source: https://docs.ertir.dev/latest/getting-started/first-agent/ # Your first agent An agent is a small graph of connected nodes. The simplest useful one takes an input, sends it to a model, and returns the answer: ```mermaid graph LR IN[input] --> LLM[llm] --> OUT[output] ``` In this walkthrough you'll build exactly that on the canvas, test it right there, and publish it as a named, versioned agent. (Your work autosaves as a **draft** the whole time — closing the tab mid-build loses nothing.) **Before you start:** you need at least one registered model. If you don't have one yet, follow [Your first chat](first-chat.md) through step 2. ## 1. Open the editor In the sidebar, open **Agents** (the home page), then click **New agent**. The editor opens with a fresh, blank graph: an **input** node already wired straight to an **output** node. It is valid and would run as-is — echoing whatever you give it — but we'll put a model in the middle. The editor has three columns: the **palette** (node types, left), the **canvas** (middle), and the **inspector** (settings for the selected node, right). Along the top is a toolbar with **id**, **name**, and **version** fields and a validation indicator. ## 2. Add an llm node In the palette, under the **Compute & Tools** group, find **llm**. Drag it onto the canvas (or click it to drop at a default spot). The new node arrives pre-seeded with one **user** message containing `$in` — the placeholder that stands for whatever input the node receives. ## 3. Wire input → llm → output Nodes connect through **ports** — the small round handles on a node's sides (inputs on the left, outputs on the right). To insert the llm between input and output: 1. **Delete the existing edge** from input to output: click it and press **Delete** (or right-click it and choose **Delete edge**). 2. Drag from the **input** node's `out` handle to the **llm** node's `in` handle. 3. Drag from the **llm** node's `out` handle to the **output** node's `in` handle. You now have `input → llm → output`, joined by solid data edges. !!! tip "Tidy up" The **Tidy layout** button in the canvas controls (bottom-left) auto-arranges the nodes if the wiring gets messy. ## 4. Configure the model and prompt Click the llm node to select it; its settings appear in the inspector. - **model** — open the picker (*"pick a binding or inference model…"*) and choose your installed model by its logical id. Picking a model from your inference plane **automatically declares it** as a model binding on the graph, so there's nothing else to set up. - **Messages** — the seeded user turn already contains `$in`, which forwards the agent's input to the model. Optionally add a **system** turn: click **Add message**, change its role to *system*, and type an instruction such as *"You are a concise assistant."* System turns automatically float to the top. `$in` is the input-referencing syntax: `$in` is the node's default input, and `$in.` / `$in..` reach named ports and fields. See [Referencing inputs](../building/input-references.md) for the full grammar. ## 5. Confirm it's valid The toolbar's validation indicator turns green (**valid**) when the graph is sound. If it shows red errors or amber warnings, click it to open the **Issues** panel — hovering an issue flashes the node or edge it points at. The usual first-agent fixes are: pick a model, and make sure every required input port is connected. ## 6. Test it on the canvas No need to publish anything to try it: 1. Click **Test** in the toolbar. A test console docks under the canvas. 2. Type an input — say, *"What's the capital of France?"* — and press ++enter++. Watch the canvas: each node pulses as it executes, then keeps a green ring when it succeeds. The model's answer streams into the panel, and a **Trace** tab shows the per-node waterfall — hover a row and the matching node flashes. Not what you wanted? Tweak the prompt or model and run again; the loop never leaves the editor. ## 7. Publish it as a named agent 1. In the toolbar, set the **id** (for example `first-agent`), a **name**, and a **version** (it starts at `0.1.0`). 2. Click **Publish** and confirm. The server strips the visual layout, computes a **content hash**, and stores this as an **immutable version** that anyone using this control plane can see and run. A few consequences worth knowing: - The **id becomes read-only** after the first publish — to "rename", publish under a new id. - To change the agent later, edit and **Publish** again with a bumped **version**. Re-publishing identical content under the same version does nothing; publishing *different* content under the same version is rejected, so bump the version yourself. - Your working **draft is removed** on publish — the registry owns the content now. (Until then, drafts wait for you on the Agents page under **Drafts**.) See [Drafts & publishing](../building/saving-agents.md) and [Agent versioning](../concepts/versioning.md) for more. ## 8. Run it anywhere Once published, the agent runs from outside the editor too: 1. Go to **Agents** in the sidebar. 2. On your agent's card, click **Run**. A **Run · <name>** modal (the Agent Bench) opens. 3. Pick a **Version** (defaults to the latest), enter an **Input** in Text or JSON mode, and click **Run**. The answer streams back just like chat, followed by a **run waterfall** showing each node's timing and its inputs and outputs. The caret next to **Run** offers **Run durably** (checkpointed), which you don't need for a simple agent like this — a plain `input → llm → output` graph runs on the normal interactive path. Agents that use `human`, `subgraph`, `loop`, or `map` nodes are durable-only and need [durable mode](../running/durable.md). ## Where to next | You want to… | Go to | |---|---| | A full tour of the editor — palette, canvas, inspector, code view | [The editor](../building/editor.md) | | Everything the llm node can do — tools, parameters, sessions | [LLM node](../building/nodes/llm.md) | | The full list of node types | [Node reference](../building/nodes/index.md) | | Test and benchmark agents in depth | [The Bench](../running/bench.md) | --- Source: https://docs.ertir.dev/latest/samples/ # Sample agents **Ready-made agents you install in one click** — each one a complete, runnable graph that demonstrates a core Ertir pattern. Install them from **Settings → Samples**, pick which of your models fill each role, then open them on the canvas, run them, change them, or delete them: an installed sample is an ordinary agent, not a special object. Every sample's name starts with `sample-` so they stay easy to spot on the Agents page. !!! tip "Why start here" Reading the [Node Reference](../building/nodes/index.md) tells you what each piece does; the samples show the pieces **composed** — conditional branches, multi-input joins, tool wiring, governance gates — as working graphs you can pull apart in the editor. ## The catalog ### Hybrid reasoner — two models in one run `sample-hybrid-reasoner` · *local model + API model* A local triage model classifies every request and emits `{"route": ..., "question": ...}`; a [router](../building/nodes/router.md) follows the matching handle. Greetings and easy questions are answered by your **local** model; coding, analysis, and long-form work go to the **API** model. Two model bindings, one graph, no proxying — each branch calls its model directly. Seen end to end in [Run Local + Remote Models in One Agent](https://www.youtube.com/watch?v=eHIHABKbwVg) (7:01). ### Private SQL analyst — data privacy by construction `sample-private-sql-analyst` · *local model + API model · creates the `sample-crm-sqlite` connection* The privacy pattern in one graph: 1. The **API model sees only the schema** (it is pasted into the system prompt) and writes one SQLite `SELECT`. 2. A deterministic [guardrail](../building/nodes/guardrail.md) (`regex`, allow-mode) refuses any statement that does not **start with** `SELECT` (no CTEs, no write verbs) — the blocked branch answers with a refusal and the database is never touched. A prefix check is not a SQL parser; the second gate is that `mcp-server-sqlite`'s `read_query` tool only accepts `SELECT` statements at all. 3. The query runs on a **local** SQLite [MCP server](../mcp/index.md) (`uvx mcp-server-sqlite`) against a small demo CRM database the installer seeds for you. 4. A **local model** reads the result rows and answers the question. The rows never leave your machine; the API provider only ever sees the schema and the question you typed (so don't paste sensitive data into the question itself). !!! note "The demo database" The installer writes a deterministic SQLite file (8 customers, 20 orders) under the control plane's artifact directory (`THEYGENT_ARTIFACT_DIR`). Reinstalling never overwrites it. The stdio server needs `uvx` on the control plane's machine — the Docker images ship it. ### Expense approver — human in the loop `sample-expense-approver` · *local model · durable runtime only* A local model assesses the risk of an expense request, then the run **waits for a real human decision** at a [human node](../building/nodes/human.md) — durably: the paused run survives restarts. Resuming with `{"decision": "approve"}` routes into a receipt + a placeholder execution step (an `echo` tool — swap it for an [http tool](../building/nodes/tools.md) that calls your real payment API); `{"decision": "reject"}` routes to a refusal. The shipped wait times out after **24 hours** (`timeout: 86400`, `onTimeout: "fail"`) — raise it, or set `timeout` to `null` to wait forever, if your approvals take longer. !!! warning "Needs the durable runtime" Human nodes run only on the durable runtime — start the stack with `THEYGENT_DURABLE=1` (the Kubernetes deploy enables it by default) and use **Run durably**. See [Durable runs](../running/durable.md) and [Human-in-the-loop](../running/human-in-the-loop.md) for the resume flow. ### Tool belt — step tools vs. LLM-driven tools `sample-tool-belt` · *local model · creates the `sample-countries-graphql` and `sample-open-meteo` connections* The two ways to use a tool, side by side, behind a [rate limit](../building/nodes/ratelimit-quota.md): - A **GraphQL API turned into an MCP server** (`countries.trevorblades.com`) runs as a flow **step**: it always executes, with args bound from the input — the model has no say. - A **REST API turned into an MCP server** (Open-Meteo, generated from an OpenAPI spec) is **wired to the model's `tools` port**: the model decides whether to call it, and works out the capital's coordinates itself. Ask about the weather and it calls; ask about the currency and it doesn't. Pick a local model with tool-calling support for this one. ### Second opinion — multi-model consensus `sample-second-opinion` · *local model + API model* The same question goes to your local model **and** the API model, each answering independently (the graph runs its nodes one after another); a local judge then merges the two answers into one, flagging any disagreement. Multi-provider consensus as a graph — no orchestration code, and the judge (who sees both answers) runs locally. ### PII firewall — privacy as a router `sample-pii-firewall` · *local model + API model* A deterministic PII [guardrail](../building/nodes/guardrail.md) inspects every request **before any model runs**: clean requests may use the API model; anything containing an email, SSN-style id, or card-like number is answered by the **local** model instead. The personal data never leaves the machine — and the run trace shows which path fired. (The default patterns catch emails, US-SSN-shaped ids, and card-like digit runs; tune `spec.patterns` for your own.) ### Visual inspector — vision in a pipeline `sample-visual-inspector` · *vision model* A local vision model inspects an image URL against your acceptance criteria and emits a minified `{"verdict": "pass"|"fail", "reason": ...}`; a `json_schema` guardrail catches a malformed verdict (fences, prose) and routes it to **Rejected**, and a router sends well-formed verdicts down the matching branch. The skeleton of a visual QC line — vision as a *pipeline step with structured output*, not a chat window. ### Voice desk — voice in, voice out `sample-voice-desk` · *speech-to-text + local model + text-to-speech* STT transcribes the audio, a model answers in spoken prose, TTS reads it back. The pickers offer every registered model with the right modality — **pick local models for all three slots and the audio never leaves your machine**. Each media step's `err` port routes to an honest failure output. Input is an audio artifact (`POST /artifacts`) or an audio URL; New Chat's voice mode drives the same shape end-to-end. Built and used in [Voice In, Voice Out](https://www.youtube.com/watch?v=leoPQLaWMeU) (5:54). ### Art department — local image generation `sample-art-department` · *local model + image model* A local model rewrites your rough brief into one vivid image prompt, and a local diffusion model paints it. If generation fails (no image engine resident), the `imagine` node's `err` port routes to an honest text fallback instead of a broken run. ### Handbook Q&A — retrieval as a model capability `sample-handbook-qa` · *local model + embedding model · creates the `sample-handbook` RAG source* The installer creates a retrieval source in your own Postgres (pgvector), seeds it with a fictional company handbook, and wires it to a local model as a **search tool the model decides to call** — answers cite the handbook section they came from. Seeding embeds in the background — check the source's status on the [RAG page](../rag/index.md) after installing; if it failed (e.g. the embedding model wasn't reachable), upload the handbook document to the source again from there, or delete the source and reinstall the sample. ### Web researcher — a crawler the model calls `sample-web-researcher` · *local model · creates the `sample-apify` connection · needs an Apify API token* A local model with a [crawler node](../building/nodes/crawler.md) wired in as its `web_search` tool. The model decides when a question needs fresh facts, calls the crawler with one search query, reads the top three pages as trimmed Markdown, and answers with the URLs it used. The result count, the output format and a $0.05 charge cap per search are fixed on the node, out of the model's reach. The installer creates the `sample-apify` connection **without a token** — samples never ship a secret. Before the first run, open the agent in the editor, select the `web_search` node, and use **Set token** on its connection (or rotate the connection's secret through the [connections API](../reference/api.md)). Until then, each search returns an error the model sees, and it says it couldn't search. Searches bill the Apify account that owns the token. ### Editorial loop — a self-correcting agent `sample-editorial-loop` · *local model · installs 2 agents · durable runtime only* A bounded [loop](../building/nodes/orchestration.md) re-runs a writer-critic child agent — draft, harsh self-critique, rewrite — until the child approves its own draft or three rounds are up. The loop is durable (it survives restarts mid-revision) and every round is a separate child run in the waterfall. The parent pins the child by version: editing the child publishes a new version, and the parent keeps running the pinned one until you re-pin. The loop's stop condition reads the child's JSON output — a round that emits something other than the `{"approved": ..., "draft": ..., "critique": ...}` shape fails the run loudly (the waterfall shows which round), the platform's no-silent-nonsense rule. ### Batch briefing — durable fan-out with a cost ceiling `sample-batch-briefing` · *local model · installs 2 agents · durable runtime only · ships a disabled weekly trigger* A `map` node fans one child agent out per topic (two at a time, `onError: collect`); a crashed worker resumes only the unfinished branches. Each **worker** starts with a token-budget [quota](../building/nodes/ratelimit-quota.md) — quota is scoped per agent over its runs' recorded usage, and every branch runs the worker agent, so the shared daily budget genuinely accumulates; over-budget branches return a budget notice instead of a brief. It ships with a **weekly schedule trigger that is disabled** — arm it with the [triggers API](../running/triggers.md) (`PATCH /triggers/{id} {"enabled": true}`; the agent's API dialog shows the trigger and its id) when you actually want unattended Monday-morning briefs. Like the agent itself, the trigger fires only on the durable runtime. ## Installing 1. Open **Settings → Samples** (admin only — Settings is an admin page; the API itself needs the editor role). 2. Fill the **model slots**. *Local model* lists your engine-served chat registrations; *API model* lists `openai-compatible` ones — add one under [Remote models](../models/remote.md) if the list is empty. The special slots (*Vision*, *Speech-to-text*, *Text-to-speech*, *Image*, *Embedding*) filter your registry by the registered modality — a sample's Install button stays disabled until every slot it declares is filled. 3. Click **Install** on a sample. The installer creates any connections and RAG sources the sample needs (reusing them by name on reinstall), seeds demo data, stamps your chosen models into the graph, publishes the agent(s) through the normal registry gate — children first, so a composition sample's pins always resolve — and registers any shipped trigger **disabled**. Everything the API does: | Method | Path | Purpose | |---|---|---| | GET | `/samples` | The catalog with per-sample install state. | | POST | `/samples/install` | Install samples: `{ids, models: {slot: {logical_id, binding}}}`. | A sample whose model slots are not filled fails loudly with `400 sample_models_required`, and a binding outside `mlx | vllm | llamacpp | openai-compatible` with `400 sample_model_invalid` — nothing is written in either case. Installing an already-installed sample reports `already_installed` — delete the agent first if you want a fresh copy with different models. If one sample in a batch fails, its report entry says `error` (that sample persists nothing) and the rest of the batch still installs. ## Making them yours - **Edit**: open the agent in the editor and publish new versions like any agent — see [Drafts & publishing](../building/saving-agents.md). - **Duplicate**: export the agent from the Agents page, change the `id`/`name` in the JSON, and import it (or paste it into a new agent in the editor's code view). - **Delete**: from the Agents page. The connections a sample created stay (other agents may use them); remove MCP servers on the MCP page if you no longer want them. ## Related pages - [Your first agent](../getting-started/first-agent.md) - [Node Reference](../building/nodes/index.md) - [MCP example agents](../mcp/examples.md) - [Durable runs](../running/durable.md) --- Source: https://docs.ertir.dev/latest/concepts/ # Architecture Ertir is split into a small number of cooperating processes, each with a clear job and a clear home for its data. Understanding the split — the interface, the control plane, the inference plane, and the optional durable worker — tells you where your agents live, where your models run, and exactly when (if ever) anything leaves your machine. ## The pieces | Piece | Default address | What it does | |---|---|---| | Interface | `http://localhost:5174` | The browser UI: build agents, chat, browse runs, manage models. | | Control plane | `http://localhost:8080` | Owns agents, runs, sessions, triggers, connections, and observability. Stores everything in Postgres. | | Inference plane | `http://localhost:8081` | Your models and engines, exposed behind one OpenAI-compatible API. | | Durable worker | — (optional) | Runs long-lived, crash-resumable workflows. Not needed for everyday local use. | The interface, control plane, and inference plane are started together by `make up`; the durable worker is optional and started separately — see [Running the services](../getting-started/running.md). ```mermaid graph LR B["Your browser
(interface :5174)"] --> CP["Control plane
:8080"] B --> IP["Inference plane
:8081"] CP -->|"OpenAI-compatible HTTP"| IP CP --> PG[("Postgres")] W["Durable worker
(optional)"] --> PG W --> IP IP --> LE["Local engines
llama.cpp / MLX / vLLM"] IP -.->|"only if you register one"| RA["Hosted API
(OpenAI-compatible)"] ``` ### The interface The interface is a single-page web app served at `http://localhost:5174`. Its sidebar pages are **Agents**, **Runs**, **New Chat**, **Chats**, **Registries**, **MCP**, and **Settings**, plus the visual [editor](../building/editor.md). It talks to both planes directly from your browser: the control plane for agents, runs, and sessions, and the inference plane for the model registry and catalog. ### The control plane The control plane is the orchestration layer. It: - stores published agents as immutable, content-addressed versions (see [Versioning](versioning.md)), - executes agent graphs and persists every run (see [Runs and sessions](runs-and-sessions.md)), - keeps chat sessions, [triggers](../running/triggers.md) (schedules, webhooks, token invocation), tool/MCP connections with encrypted secrets, and per-run [observability](../running/observability.md) data. It needs two things to be healthy: a reachable Postgres (`DATABASE_URL`) and the inference plane. Crucially, the control plane **never runs a model itself** — every model call is an HTTP request to the inference plane's OpenAI-compatible API. That seam is what lets you swap where inference happens without touching a single agent. ### The inference plane The inference plane is where models actually run. It manages the lifecycle of local engines — llama.cpp, MLX (Apple Silicon), and vLLM (CUDA GPUs) — and can also reach any OpenAI-compatible server or hosted API you register. Two rules define it: - **Callers use logical model ids.** An agent says `"triage-fast"`, never an engine name. Which engine or endpoint serves that id is a registration detail you can change at any time — see [Models and engines](models-and-engines.md). - **Its state is local files, never the control-plane database.** The model registry lives at `~/.theygent/inference/registry.json` and downloaded weights under `~/.theygent/inference/models/`. ### The durable worker (optional) Durable execution makes unattended runs survive a crash and resume from the last completed step, and it unlocks the durable-only node types (`human`, `subgraph`, `loop`, `map`). You get it in one of two ways: - **Embedded** — set `THEYGENT_DURABLE=1` and the control plane runs the durable runtime in-process. This is the normal desktop setup. - **Separate worker process** — `theygent-worker`, for server deployments. It consumes the durable queue and recovers crashed workflows. Either way, durable checkpoints live in a separate `dbos` schema of the same Postgres. Details in [Durable runs](../running/durable.md). ## Where your data lives | Data | Where it lives | |---|---| | Agents, versions, runs, sessions, triggers, connections and encrypted secrets, observability spans | Postgres, at `DATABASE_URL` | | Durable run checkpoints | The `dbos` schema of that same Postgres | | The logical model registry | `~/.theygent/inference/registry.json` | | Downloaded model weights | `~/.theygent/inference/models/` | | Generated audio and images (artifacts) | `THEYGENT_ARTIFACT_DIR` (default: a `theygent-artifacts` folder in the system temp directory) | The storage locations and listen addresses are all configurable — see the [environment variable reference](../reference/environment.md). ## The sovereignty story, honestly Ertir's promise is not "no data ever leaves your machine". The honest version is: **everything runs where you point it, and you own every hop.** - All four processes run locally by default, listening on `127.0.0.1`. Your prompts travel from your browser to the control plane to the inference plane over local HTTP. - Local models run on your hardware; weights download to your disk. - A hosted API is contacted **only when you register one** — as an `openai-compatible` model with a base URL you supply — and then only for calls routed to that model. Nothing is ever proxied through a vendor server. See [Remote models](../models/remote.md). - Because agents reference logical model ids, the local-versus-hosted choice is per model and per agent, and reversible: re-register the id, and every call site follows. - Credentials live in two separate stores, both on your machine and never written into agent documents. **Hosted-API keys** are inference-plane credentials, referenced as `secret://NAME` and never sent to the control plane (see [Remote models](../models/remote.md)). **Tool and MCP connection secrets** live in the control-plane store, encrypted with `THEYGENT_SECRET_KEY`. - Nothing is exported for telemetry unless you opt in by setting `OTEL_EXPORTER_OTLP_ENDPOINT`; otherwise all trace data stays in your Postgres. Every connection Ertir makes on its own, and how to run fully offline, is listed in [Privacy and data flow](privacy.md). !!! warning "Keep the inference plane on loopback" The control plane requires sign-in: a fresh install's API is closed until the first-run wizard creates an admin, and every call after that carries a bearer token — see [Users, roles and sign-in](../running/users.md). The inference plane has no sign-in of its own. It listens on `127.0.0.1` by default, and anything that can reach port 8081 can use your models, so don't expose it to untrusted networks. ## Checking that everything is up Both planes expose `GET /healthz` (liveness) and `GET /readyz` (readiness): ```bash curl http://localhost:8080/readyz # control plane: checks Postgres and the inference plane curl http://localhost:8081/readyz # inference plane: reports which engine/modality slots this host can run ``` A missing engine binary shows up in the inference plane's `/readyz` report — never as a surprise failure on your first model call. ## Related pages - [Agents and graphs](agents-and-graphs.md) — what an agent document contains - [Nodes, ports, and edges](nodes-ports-edges.md) — the building blocks of a graph - [Models and engines](models-and-engines.md) — logical ids, bindings, sources, modalities - [Runs and sessions](runs-and-sessions.md) — run lifecycle and conversational memory - [Running the services](../getting-started/running.md) — start, stop, status, ports --- Source: https://docs.ertir.dev/latest/concepts/agents-and-graphs/ # Agents and graphs In Ertir, an agent *is* a graph: a set of steps (nodes) wired together (edges), plus everything those steps need to run. This page explains what that document contains and why the way you arrange it on the canvas has no effect on what it does. ## An agent is a graph document Behind the visual canvas, every agent is a single JSON document. It describes a **graph** — nodes connected by edges — together with the models the agent may call, the tools it may use, and where each node sits on the screen. Running the agent means walking that graph from its input to its output. You almost never write this document by hand. You build it in the [visual editor](../building/editor.md), dragging nodes and wiring ports; the editor keeps the JSON in sync and offers a raw **code view** as an escape hatch. But it helps to know the shape of what you are building, because the same document is what gets saved as a version, hashed for identity, and executed. ## What the document contains The document is a small envelope of top-level fields. The keys are camelCase, and a typo in any of them is rejected rather than quietly ignored. | Field | What it holds | |---|---| | `schemaVersion` | The document format version. The editor writes `"1.0"`. | | `id` | The agent's stable identifier (for example `agent.greeter`). With `version`, it is the registry coordinate. | | `name` | A human-readable display name. | | `version` | This document's version string (for example `0.1.0`). | | `models` | A map of model **key → binding** — the models this agent may call, each given a short local key that nodes reference. | | `tools` | A map of tool **key → binding** — the built-in, HTTP, or MCP tools this agent may use. | | `nodes` | The list of nodes: the steps of the agent. | | `edges` | The list of edges wiring node ports together. | | `view` | Canvas layout — node positions, viewport zoom, per-node icon overrides. Cosmetic only. | | `contentHash` | A derived fingerprint of the document (see [Layout is cosmetic](#layout-is-cosmetic)). Filled in by the server when you publish. | | `metadata` | Optional free-form data you can attach; ignored by execution. | ### Identity: id and version An agent is identified by its `id`, and each published snapshot by its `version`. Together they form the coordinate the registry uses. When you publish an agent, that exact `(id, version)` pair becomes an immutable version — see [Drafts & publishing](../building/saving-agents.md) and [Versioning](versioning.md). ### The models map Rather than naming a model inside every node, an agent declares the models it may call once, in the `models` map, and nodes reference them by key. Each entry is a **binding**: which engine family serves the model (`mlx`, `vllm`, `llamacpp`, or `openai-compatible`), the logical model id to ask for, and optional generation parameters. There are two levels of naming here, and it is worth keeping them straight: - The **map key** (for example `responder`) is a local alias, meaningful only inside this one agent. Nodes reference this key. - The binding's **`model`** field (for example `chat-small`) is the *logical model id* registered on the inference plane — never an engine name. Which engine or endpoint actually serves it is a registration detail you can change without touching the agent. See [Models and engines](models-and-engines.md) for how logical ids, bindings, and sources fit together. ### The tools map Tools are declared the same way: once in the `tools` map, then referenced by key from `tool` and `mcp_tool` nodes. A binding is one of three kinds — a built-in tool, an HTTP call, or an MCP tool. Crucially, **secrets never live in the document**: an HTTP or MCP tool references a saved connection by id, and the credential is resolved on the server at run time. See the [tools node reference](../building/nodes/tools.md). ### The nodes and edges The `nodes` and `edges` lists are the graph itself — the steps and the wires between them. This is where the agent's behaviour actually lives. The next page, [Nodes, ports, and edges](nodes-ports-edges.md), covers the node kinds, the ports they connect through, and how values flow. ### The view The `view` block records where each node sits, the current zoom, and any custom icons you picked. It exists purely so the canvas looks the same when you reopen the agent. Nothing in it affects execution. ## Layout is cosmetic An agent's identity is a content hash — a `sha256:` fingerprint the server computes over the document. Before hashing, the layout (`view`) is stripped out, defaults are filled in, and keys are sorted. The consequence is simple and worth relying on: - **Rearranging the canvas never creates a new version.** Dragging a node, zooming, collapsing a node, or changing its icon changes only the `view` block, so the hash is unchanged. - **Only real content changes the hash** — editing a message, a config value, a model binding, or an edge. - Formatting and key order of the JSON never matter, and omitting a field that has a default hashes the same as writing that default explicitly. So two people can lay out the same agent completely differently and still have, byte-for-byte in what counts, the *same* agent. See [Versioning](versioning.md) for how the hash underpins immutable versions. ## A complete example A minimal agent that sends its input to a model and returns the answer looks like this. In the editor you would build it by dropping an `input`, an `llm`, and an `output` node and wiring them left to right. ??? example "The full document" ```json { "schemaVersion": "1.0", "id": "agent.greeter", "name": "Greeter", "version": "0.1.0", "models": { "responder": { "binding": "llamacpp", "model": "chat-small" } }, "nodes": [ { "id": "n_in", "type": "input", "kind": "boundary", "config": {}, "ports": { "in": [], "out": [{ "id": "out" }] } }, { "id": "n_llm", "type": "llm", "kind": "activity", "config": { "model": "responder", "messages": [{ "role": "user", "content": "$in" }] }, "ports": { "in": [{ "id": "in" }, { "id": "tools", "role": "tool", "required": false }], "out": [{ "id": "out" }] } }, { "id": "n_out", "type": "output", "kind": "boundary", "config": {}, "ports": { "in": [{ "id": "in" }], "out": [] } } ], "edges": [ { "id": "e_in_llm", "source": "n_in", "sourceHandle": "out", "target": "n_llm", "targetHandle": "in" }, { "id": "e_llm_out", "source": "n_llm", "sourceHandle": "out", "target": "n_out", "targetHandle": "in" } ], "view": { "nodes": { "n_in": { "position": { "x": 0, "y": 0 } }, "n_llm": { "position": { "x": 260, "y": 0 } }, "n_out": { "position": { "x": 520, "y": 0 } } } } } ``` A few things to notice, all covered in more depth elsewhere: - The `llm` node references the model by its map key (`responder`), not by the logical id (`chat-small`) or an engine name. - The message content is `$in` — a reference to the value arriving on the node's input. This is the [input reference language](../building/input-references.md). - Every node carries a `type` and a `kind`; the `input` and `output` nodes are the graph's boundary, and the `llm` node is the work in the middle. See [Nodes, ports, and edges](nodes-ports-edges.md). ## Creating and running agents A brand-new graph in the editor already contains an `input` node wired to an `output` node, so it validates and runs (it simply echoes its input) before you have changed anything. From there you add nodes, wire them, and configure them in the inspector — your work autosaves as a draft, and the editor's Test panel runs the graph in place at any point. When it is ready, publish it to the registry to mint an immutable version. See: - [The editor](../building/editor.md) — the canvas, palette, and inspector - [Drafts & publishing](../building/saving-agents.md) — turning a graph into a named version - [Runs and sessions](runs-and-sessions.md) — what happens when an agent runs ## Related pages - [Nodes, ports, and edges](nodes-ports-edges.md) — the graph's building blocks - [Models and engines](models-and-engines.md) — bindings, logical ids, and sources - [Versioning](versioning.md) — content hashing and immutable versions - [Node reference](../building/nodes/index.md) — every node type in detail --- Source: https://docs.ertir.dev/latest/concepts/nodes-ports-edges/ # Nodes, ports, and edges An agent is a graph of **nodes** wired together by **edges**, and each connection joins two named **ports**. This page explains the three kinds of node, what ports are for, and how values travel along edges — including why some nodes don't run at all on a given pass. ## Nodes come in three kinds Every node has a **type** (what it does — `llm`, `router`, `output`, …) and a **kind** (its broad role in the graph). There are three kinds: | Kind | In plain words | Node types | |---|---|---| | **boundary** | Where data enters or leaves the graph, or where a run pauses. | `input`, `output`, `human`, `subgraph` | | **activity** | A step that does real work — a model call, a tool call, or another outside effect. Results can vary from run to run. | `llm`, `tool`, `mcp_tool`, `transcribe`, `speak`, `imagine`, `ratelimit`, `quota` | | **orchestration** | Deterministic control flow and reshaping — no call to the outside world. | `router`, `transform`, `loop`, `map` | The kind is not decoration: every type has exactly one correct kind, and the editor sets it for you. One type is the exception — a `guardrail` takes its kind from how you configure it. A rule check (a plain, inline check) is orchestration; a model-judge check is an activity, because it makes a real inference call. You never set this by hand; choosing the check type in the inspector picks the kind. !!! note "A few types are durable-only" `human`, `subgraph`, `loop`, and `map` run only on the durable runtime. The ordinary interactive run path refuses them and tells you to run the agent durably. See [Durable runs](../running/durable.md), the [human approval node](../building/nodes/human.md), and the [orchestration nodes](../building/nodes/orchestration.md). The full catalogue of node types, with each one's configuration, is the [node reference](../building/nodes/index.md). ## Ports Nodes connect to each other through **ports**: named connection points, in-ports on one side and out-ports on the other. A node's ports depend on its type — an `llm` has an `in` port and an `out` port (and a special `tools` port); a `guardrail` sends its result out of either a `pass` or a `block` port. Two properties of a port matter when you wire a graph: - **Required vs. optional.** A *required* in-port must be fed by exactly one edge, or the graph fails validation (`required in-port 'x' is not connected`). An *optional* in-port can be left unwired — at run time it simply resolves to nothing (`null`). - **Role.** Most ports carry data. Two roles are special: **control** ports carry pure sequencing, and **tool** ports carry a capability. The editor only lets you connect a port to another port of the same role. Downstream nodes read the value that arrives on their in-ports using the [`$in` reference language](../building/input-references.md) — `$in` for the default `in` port, `$in.` for a named one. ## Edges and their three channels An **edge** connects one node's out-port to another's in-port. The editor derives an edge's **channel** from the ports it joins, and only lets you connect like to like: | Channel | Carries | The rule to remember | |---|---|---| | **data** | A value passed from one node to the next. | At most **one** data edge may feed a given in-port — a second is rejected as an ambiguous multi-input. | | **control** | Nothing — pure sequencing, "run this after that". | Any number allowed. Shown as an animated line on the canvas. | | **tool** | A *capability*, not a value — it tells a model it may call a tool. | Wired from a tool node's `use` handle into an `llm` node's `tools` port. | ### The tool channel: capability wiring By default a `tool` or `mcp_tool` node is a step in the pipeline: a value flows into its `in` port, the tool runs, and a result flows out of `out` (or an error out of `err`). But you can instead wire its `use` handle into an `llm` node's `tools` port. That connection passes no value — it hands the model the tool as something it may call on its own, deciding when to call it and with what arguments. The model can then call it, read the result, and keep going, in a loop, until it has an answer. A tool node is one or the other — a pipeline step **or** a capability offered to a model — never both at once. When you wire it as a capability, the node's own id becomes the function name the model sees. See the [tools node](../building/nodes/tools.md) and the [llm node](../building/nodes/llm.md) for the full story. ## How values flow When a node finishes, the value it produces travels along its outgoing **data** edges to the nodes downstream. But not every edge carries a value on every run. An edge is **live** only if its source node actually ran *and* activated that particular out-handle: - Some nodes activate exactly **one** of several out-handles. A `router` activates the single handle its `select` resolves to. A `guardrail` activates either `pass` or `block`. `ratelimit` and `quota` activate either `allow` or `deny`. The un-taken handle's edges are **dead**. - A node whose incoming data edges are **all dead** is **skipped** — it doesn't run at all, and any downstream port fed only by that dead branch resolves as absent (`null`). - The `input` node has no incoming edges, so it is never skipped — it is where every run begins. This is exactly how you build branches. A `router`, for instance, sends a request down one branch or another by name: ```mermaid graph LR in[input] --> r{router} r -->|"local"| a[llm: local model] --> o1[output] r -->|"remote"| b[llm: hosted model] --> o2[output] ``` Here the router's `select` resolves to either `local` or `remote`. Only that branch's edges are live; the other branch — including its `output` node — is skipped entirely. Notice each branch ends in **its own** `output` node. That is deliberate: at most one output node may execute per run, and if two were live at once the run would fail loudly rather than silently pick one. Because the branches are mutually exclusive, exactly one output runs and the run has a single, unambiguous result. (A single output node fed by two edges wouldn't work either — that is the "one data edge per in-port" rule.) See [Runs and sessions](runs-and-sessions.md) for how a run turns into its final output. ## Related pages - [Agents and graphs](agents-and-graphs.md) — the document a graph lives in - [Referencing inputs](../building/input-references.md) — the `$in` language nodes use to read values - [The router node](../building/nodes/router.md) — building branches by handle name - [The tools node](../building/nodes/tools.md) — steps versus capabilities - [Node reference](../building/nodes/index.md) — every node type in detail --- Source: https://docs.ertir.dev/latest/concepts/models-and-engines/ # Models and engines Ertir separates *what an agent calls* from *what actually runs*. An agent always names a model by a **logical id** — a label you choose, like `triage-fast` — and never an engine. Behind that id, the inference plane holds a registration that says which engine (or hosted endpoint) serves the id, where its weights come from, and what task it performs. This page explains that split and the vocabulary around it: bindings, sources, and modalities. ## Logical model ids A logical id is the name your agents use to reach a model. You register a model once under an id of your choosing, and from then on every agent, chat, and node refers to that id — for example an `llm` node whose model is `triage-fast`. The rule is strict: the id is a logical name, never an engine name. If an agent tried to call `mlx`, `llamacpp`, or `vllm` as if it were a model, the run is rejected up front with a clear error (`engine_name_not_allowed`) before anything executes. Engine names live only in the *registration* for an id, never at a call site. ```mermaid graph LR N["llm node
model: triage-fast"] --> REG["Inference plane
registry"] REG -->|"triage-fast is registered as…"| E["An engine or endpoint
(you decide, and can change)"] ``` This indirection is the whole point. Because the agent only knows the id, you can change what the id points at — a different engine, different weights, a hosted API — without editing a single agent. ## Bindings: which engine backs an id The **binding** says what kind of thing serves a logical id. There are exactly four values, and they never change meaning: | Binding | What backs the id | How Ertir treats it | |---|---|---| | `llamacpp` | A llama.cpp server running GGUF weights. Runs anywhere the binary is installed. | Managed — spawned and supervised for you. | | `mlx` | The MLX family of servers on Apple Silicon. | Managed. | | `vllm` | A vLLM server on an NVIDIA/CUDA GPU host. | Managed. | | `openai-compatible` | Any OpenAI-compatible HTTP server or hosted API you point at with a base URL. | Reached, not managed — no process is started or stopped for you. | The first three are **managed engines**: Ertir starts the server on first use, health-checks it, and stops it when it is idle. The fourth, `openai-compatible`, is a **reachable endpoint**: Ertir just forwards calls to a URL you supply — whether that URL is a hosted API or another local server. For details on each engine and its platform, see [The engines](../models/engines.md); to register a hosted endpoint, see [Remote models](../models/remote.md). !!! note "vLLM is experimental in this build" The `vllm` binding exists and targets CUDA GPU hosts, but it has not yet been verified on the maintainers' hardware. llama.cpp and MLX are the proven local paths. Treat vLLM as experimental until you have confirmed it on your own GPU. ## Sources: where the weights come from For a managed engine, the **source** is orthogonal to the binding — it says where the model's weights come from, not how they run: | Source | Meaning | |---|---| | `hf` | A Hugging Face repo id; the weights are downloaded to your machine. | | `local-path` | A file or directory of weights already on your disk. | | `url` | A location the engine loads weights from. | Binding and source are independent: the same GGUF file on disk (`source: local-path`) or pulled from Hugging Face (`source: hf`) both run under `binding: llamacpp`. A `url` source is not the same as the `openai-compatible` binding — a source points at *weights an engine loads*, while `openai-compatible` points at *a server that already runs the model*. Reachable (`openai-compatible`) registrations have no source at all; they carry a base URL instead. Weights downloaded from Hugging Face land under `~/.theygent/inference/models/` on your machine. See [Installing models](../models/installing.md) for the browse-and-install flow. ## Modalities: what a model does A model's **modality** is the task it performs. It is a third axis, independent of binding and source — a llama.cpp binding can serve chat, embeddings, or vision depending on how it is registered. | Modality | Wire value | Model does | |---|---|---| | Chat | `chat` | Text in, text out — the default. | | Vision | `vision` | Image (plus text) in, text out. | | Embeddings | `embeddings` | Text in, numeric vectors out. | | Transcription | `audio.transcription` | Audio in, text out (speech to text). | | Speech | `audio.speech` | Text in, audio out (text to speech). | | Image generation | `images.generation` | A text prompt in, an image out. | When you register a model, its modality decides which server the engine manager spawns and which data-plane endpoint accepts it. A chat-only model called at the embeddings endpoint returns a clear `modality_not_supported` error rather than a wrong answer. Not every engine supports every modality — transcription, for example, is served by whisper.cpp under the `llamacpp` binding; MLX handles chat, vision, and text-to-speech. The [engines page](../models/engines.md) lists what each one covers. The modality here describes a *model*. It is separate from the `modality` field on an agent's `input`/`output` boundary nodes, which is a declaration about a node's data type — see [Nodes, ports, and edges](nodes-ports-edges.md). ## How engines run Managed engines are started **lazily** — the first call to a logical id spawns its server, and Ertir waits for it to become healthy before forwarding the request. Only a limited number of engines stay resident at once (two by default). When that ceiling is reached, the least-important, least-recently-used engine is stopped to make room; an engine that is in the middle of a request is never evicted. Idle engines are also reaped after a timeout. You do not manage any of this by hand — but it is why the very first call to a large model can take a moment while it loads. The full lifecycle and its knobs are covered in [The engines](../models/engines.md). A reachable (`openai-compatible`) id has no lifecycle: nothing is spawned or evicted, and calls are proxied straight to your URL. ## Swapping a model without touching agents Because agents route by logical id, changing what an id points at is a re-registration, not an edit to any agent. Suppose ten of your agents call `writer`, registered today as a local llama.cpp model. Later you want `writer` to run on a hosted API instead. You re-register the id `writer` as an `openai-compatible` binding with the provider's base URL and your credential. That is the only change. Every agent, chat, and trigger that used `writer` now routes to the hosted endpoint on its next run — no agent is opened, re-saved, or re-versioned. ```mermaid graph LR subgraph Agents["Agents (unchanged)"] A1["agent A → writer"] A2["agent B → writer"] end Agents --> ID["logical id: writer"] ID -->|"today"| L["local llama.cpp"] ID -.->|"after re-registration"| H["hosted openai-compatible"] ``` This works because, at run time, Ertir forwards only the logical id to the inference plane; the inference plane's current registration for that id decides the engine, source, and endpoint. ??? note "What the agent document actually stores" When you pick a model in the editor, Ertir records the logical id on the model binding and copies in the binding value it was registered under at that moment (so the editor knows the model family). That stored binding value is a snapshot — it is not consulted when the run executes. The run resolves purely by logical id, so re-registering the id under a different engine takes effect on the next run without re-saving the agent. Generation parameters (temperature, max tokens, and so on) are the exception: those live on the agent's model binding and travel with each call, so they are part of the agent, not the registration. ## Related pages - [Registries](../models/index.md) — where you register, browse, and install models - [The engines](../models/engines.md) — llama.cpp, MLX, and vLLM in detail - [Remote models](../models/remote.md) — registering a hosted or remote OpenAI-compatible endpoint - [The llm node](../building/nodes/llm.md) — picking a model and setting its parameters in an agent - [Runs and sessions](runs-and-sessions.md) — what happens when an agent calls a model --- Source: https://docs.ertir.dev/latest/concepts/runs-and-sessions/ # Runs and sessions A **run** is one execution — of a plain prompt against a model, or of a whole agent graph. Every run is recorded in the control plane's Postgres database, so it survives restarts and you can inspect it long after the answer has scrolled away. A **session** is optional conversation memory you can attach to runs so a model remembers earlier turns. This page explains the run lifecycle, where a run's output comes from, and how sessions carry memory. ## What a run is Whenever an agent produces an answer, that is a run. Runs start from a few places: - **A plain prompt** — you send a string and a logical model id, and the model completes it. This is the simplest run. - **An agent graph** — the control plane walks the graph node by node, calling models, tools, and MCP servers as the edges dictate, and returns the graph's output. - **A published agent** — the same as above, but the graph comes from an immutable [published version](versioning.md) rather than being pasted in. However it starts, a run gets a row in the database the moment it is accepted. That row holds the status, the resolved model, the final output, any error, timing, and — for a published agent — which version ran. You can read a run at any time, including while it is still producing output. The chat surface, the [Runs list](../running/index.md), and the [run waterfall](../running/observability.md) are all views onto these rows. ## The run lifecycle A run moves through a small set of statuses: | Status | Meaning | |---|---| | `created` | The run row exists and is about to execute. | | `streaming` | The run is executing and producing output. Answer tokens stream out as they are generated. | | `waiting` | The run has paused at a human-approval step and is waiting for input. Only durable runs reach this state. | | `completed` | The run finished. Its output is persisted. | | `failed` | The run errored, or was interrupted. | ```mermaid stateDiagram-v2 [*] --> created created --> streaming streaming --> completed streaming --> failed streaming --> waiting: human node (durable only) waiting --> streaming: input delivered ``` A few behaviors are worth knowing: - **Reasoning is separate from the answer.** When a model "thinks" out loud, those reasoning tokens stream on their own channel and are shown as a collapsible thinking view in [chat](../chat/index.md) — they are never folded into the run's output. - **Interruptions are honest.** If the control plane restarts while an ordinary run is in flight, that run is marked `failed` with a plain reason ("interrupted: control-plane restarted while run was in-flight") rather than left looking alive. Closing the browser mid-stream likewise ends the run as `failed` ("client disconnected mid-stream"). - **Durable runs survive crashes.** A run on the [durable runtime](../running/durable.md) resumes from its last completed step instead of failing, and a `waiting` run can pause indefinitely — across restarts — until you [resume it](../running/human-in-the-loop.md). ## Where the output comes from For a **plain prompt run**, the output is exactly the model's completion. For an **agent graph**, the output is the value that reached the executed `output` node, coerced to a string (structured values are serialized to JSON). The `output` node performs no side effect of its own; it simply marks what the run returns. Two rules keep this unambiguous: - **At most one `output` node may execute per run.** A graph may *contain* several output nodes — for instance, one on a guardrail's allow branch and one on its block branch — but only one can be live in any single run. If a second one executes, the run fails loudly rather than silently picking a winner. Route mutually exclusive branches (with a [router](../building/nodes/router.md) or [guardrail](../building/nodes/guardrail.md)) so at most one output is ever reached. - **Empty output is explained, not hidden.** If the run reaches no output node, or the branch that ran produced nothing, or the model spent its whole token budget thinking, the run still completes but carries an honest note explaining why the output is empty (for example "the graph declares no output node" or "hit the token limit … raise maxTokens"). The interface styles such a run amber rather than pretending it succeeded. See [Runs](../running/index.md) for how these rules surface in the Runs list and detail views. ## Sessions: conversation memory By default a run is one-shot: the model sees only the current prompt, and nothing about the exchange is remembered for next time. (The run itself is still recorded — "one-shot" refers to memory, not persistence.) A **session** turns a series of runs into a conversation. You attach a session to a run by giving it a session id; every run that shares that id becomes part of the same conversation, and the model sees what came before. How the memory works: - **Full replay, verbatim.** Before the current prompt, every prior turn in the session — each user message and each assistant reply — is prepended in order, exactly as it was. There is no summarization and no truncation; the conversation is replayed in full each time. Inside an agent, an `llm` node reads the same prior turns. - **Only real exchanges are stored.** A run appends its user-and-assistant turn pair to the session only when it succeeds with non-empty output. A failed run stores nothing (no dangling half-turn), and an empty-output run adds no blank reply. - **Unknown ids are created for you.** Attaching a session id that does not exist yet simply creates that session — you do not have to set one up in advance. ```mermaid sequenceDiagram participant You participant Run participant Session You->>Run: prompt + session id Session-->>Run: prior turns (verbatim, in order) Run->>Run: model answers with full history Run->>Session: append this turn (only if it succeeded) ``` Sessions are what power the **Recents** list in chat: each ongoing conversation is a session. Deleting a session removes the conversation and its messages but *detaches* — never deletes — the runs that happened in it, so your run history outlives any conversation you clear. !!! tip "Memory is opt-in and per run" Because a session is attached run by run, you decide exactly which runs share memory. Two agents can even share one conversation by using the same session id, and a single agent can hold many independent conversations by using different ids. ## Related pages - [Runs](../running/index.md) — the Runs list, run detail, and output rules in the UI - [Durable runs](../running/durable.md) — crash-resumable runs and the `waiting` state - [Human in the loop](../running/human-in-the-loop.md) — pausing a run for approval and resuming it - [Chat](../chat/index.md) — conversations, sessions, and the thinking view - [Versioning](versioning.md) — the immutable agent versions runs execute against --- Source: https://docs.ertir.dev/latest/concepts/versioning/ # Versioning When you publish an agent, Ertir stores it as an **immutable, content-addressed version**. "Content-addressed" means the version's identity is a fingerprint of what the agent *means* — its nodes, edges, models, tools, and configuration — and nothing else. Rearranging the canvas never creates a new version; changing what the agent does always does. This page explains that fingerprint, what counts as a change, and how versions are pinned and resolved. Versions are one of two tiers the editor works with: while you are still building, the graph autosaves as a mutable **draft** — no hash, no history, nothing pinnable. Publishing is the moment a draft's content graduates into everything below. See [Drafts & publishing](../building/saving-agents.md). ## Immutable versions A published agent has a stable id and a series of versions. Each version, once published, never changes — Ertir will not overwrite it. When you want to change an agent, you publish a new version alongside the old ones; the earlier versions remain exactly as they were, so anything pinned to them keeps behaving identically. This immutability is what makes an agent safe to deploy. A [trigger](../running/triggers.md), a [durable run](../running/durable.md), or a parent agent that composes this one can pin a specific version and trust that it will never shift under them. While an agent exists, its history is append-only: there is no way to edit or remove an *individual* version. You can, however, delete the **whole agent** — from the Agents page or `DELETE /agents/{id}` — which removes it with all of its versions and triggers in one step; past runs and chats survive, keeping the agent id and content hash they ran against. See [Exporting and deleting agents](../building/saving-agents.md#exporting-and-deleting-agents). Immutability is per-version, never a lock on the agent's existence. ## The content hash: meaning, not layout Every version carries a `contentHash` of the form `sha256:…`. It is computed by taking the full agent document, filling in every default, sorting the keys, stripping out the layout, and hashing the result. Two things are deliberately left out of the hash: - the `contentHash` field itself, and - the **`view` block** — the canvas layout: node positions, zoom level, whether a node is collapsed, and any custom node icons. ```mermaid graph TB subgraph Doc["Your agent document"] C["nodes · edges · models · tools · configuration
(the meaning)"] V["view: positions · zoom · collapse · icons
(the layout)"] end C -->|"hashed"| H["contentHash · sha256:…
(the version's identity)"] V -.->|"never hashed"| H ``` Because the layout is never hashed, everything you do to *arrange* the canvas leaves the version untouched: - Dragging nodes to new positions. - Zooming or panning. - Collapsing or expanding a node. - Changing a node's display icon. None of these mint a new version. Neither does reformatting the underlying document — key order and whitespace never matter, and omitting a field that has a default produces the same hash as writing that default out explicitly. Two agents that mean the same thing always have the same hash, and so are the same version. ## What creates a new version Any change to what the agent *does* changes the hash, and you publish that as a new version. That includes: - Editing a node's configuration — a prompt message, a temperature, a router's selection, a guardrail rule. - Adding, removing, or rewiring a node or an edge. - Changing which model an `llm` node uses, or the generation parameters stored on a model binding. - Adding, renaming, or removing a port. - Changing a pinned child agent in a [subgraph, loop, or map](../building/nodes/orchestration.md) node. If the hash is different, it is a different version. ### Publishing rules When you publish, the outcome depends on whether the content and the version label already exist: | You publish… | Result | |---|---| | The **same content** under an **existing** version label | Idempotent — nothing new is stored; you get the version you already have. | | **Different content** under an **existing** version label | Rejected (`version_conflict`) — a published version is immutable. Bump the version label. | | A **new** version label with any content | A new version is created. | The idempotent case matters in practice: re-publishing an agent you have not actually changed, or redeploying the same agent in a script, is a harmless no-op rather than an error. The rejection case is the immutability guard — you cannot quietly rewrite version `1.0` to mean something new; you publish `1.1` instead. See [Drafts & publishing](../building/saving-agents.md) for the publish flow in the editor. ## Pinning and resolving a version Anything that runs a published agent can pin which version it means, in one of two ways: by its exact `content_hash`, or by its version label. If neither is pinned, Ertir uses the **latest** version. The resolution order is always: 1. a pinned content hash, then 2. a pinned version label, then 3. latest. "Latest" means the **most recently published** version — not the highest-numbered label. If you publish a fix to an older line *after* releasing a newer one, that fix is now the latest, because latest tracks publication order, not version numbering. For unattended runs, prefer an explicit pin so "latest" can never surprise you: triggers, in fact, *require* you to pin exactly one of a version or a content hash. A pin that no longer resolves — a mistyped hash or a version that does not exist — fails cleanly with a not-found error, never a crash or a silent fallback to something else. ## Composition is frozen in When one agent composes another — through a [subgraph, loop, or map](../building/nodes/orchestration.md) node — the child is referenced by id plus exactly one pin (a version or a content hash). That pin lives in the parent's configuration, so it becomes part of the parent's content hash. The consequence: the parent's version fully determines which child version runs. If the child later publishes a new version, the parent does not change and does not pick it up — you would publish a new parent version to adopt the newer child. Composition is immutable, all the way down. ## Related pages - [Agents and graphs](agents-and-graphs.md) — what the agent document contains - [Drafts & publishing](../building/saving-agents.md) — the mutable draft tier and the publish flow - [Runs and sessions](runs-and-sessions.md) — the runs that execute against a version - [Triggers](../running/triggers.md) — deploying a pinned version behind a schedule, webhook, or token - [Orchestration nodes](../building/nodes/orchestration.md) — composing pinned child agents --- Source: https://docs.ertir.dev/latest/concepts/privacy/ # Privacy and data flow Ertir's promise is **everything runs where you point it, and you own every hop** — not "no data ever leaves your machine". This page is the checkable version of that promise: where each kind of data lives, every connection the software makes on its own and when, and how to run with no network at all. ## Where your data lives | What | Where it stays | |---|---| | Prompts and responses | Browser → control plane → inference plane, over local HTTP. All of them listen on `127.0.0.1` by default. A hosted model is reached only if you [register one](../models/remote.md), and then directly from the inference plane. | | Model weights | The inference plane's model directory, `~/.theygent/inference/models/` by default (`THEYGENT_INFERENCE_PLANE_MODEL_DIR`). Weights you register by path stay where they are. | | The model registry and hosted-API keys | Local files under `~/.theygent/inference/` (`THEYGENT_INFERENCE_PLANE_STATE_DIR`). Keys are referenced as `secret://NAME` and never sent to the control plane. | | Agents, runs, sessions, traces | Your Postgres, at `DATABASE_URL`. What each agent's runs capture — and for how long — is set by its [I/O policy](../running/observability.md). | | Retrieval documents | pgvector in the same Postgres, embedded by the model you bind to the source. Bind a local embedding model and the documents never leave the machine. See [RAG sources](../rag/index.md). | | Tool and MCP secrets | The control plane's secret store, encrypted with `THEYGENT_SECRET_KEY`. Agents carry references, never values. | | What a crawler collects | A [crawler node](../building/nodes/crawler.md) sends its input to the provider it names, which crawls on its own servers; the results come back into your run. The provider's token stays in the secret store. | | Audio and images | Local artifacts under `THEYGENT_ARTIFACT_DIR` on the control-plane host — never written into run history or exported with traces. | ## Every connection Ertir makes This is the complete list for a running install. Apart from the first row, nothing connects until you open a feature that needs the network, or configure one. | Connection | When | What it carries | |---|---|---| | A public model price list from GitHub | Each time the inference plane starts. The model client library it uses to call engines (LiteLLM) loads it. | None of your data. With no network, the library falls back to the copy bundled with it, and startup continues. | | Hugging Face | When you browse the model catalog, install a model, or start a model you registered by hand with `source: hf`. | Your catalog searches and the files you download. A [gated repository](../models/installing.md#install-from-the-catalog) also sends your Hugging Face token. Catalog installs register as local files, so they load from disk afterwards. | | MCP registries | When you open **Browse hubs** on the MCP page. | Your searches. Replace the public hubs with a registry you host by setting `THEYGENT_MCP_REGISTRIES` — see [MCP servers](../mcp/index.md#installing-from-a-hub). | | The Apify Store | When you open the **Crawlers browser** in the editor. The control plane makes the request; your browser does not. | Your searches and the Actors you open. No token is sent. See [Crawler](../building/nodes/crawler.md#adding-a-crawler). | | Things you configure | When an agent or a setting uses them. | Exactly what you wired: a [hosted API](../models/remote.md) you registered, a remote MCP server, an HTTP tool, a crawler node's provider (which receives the node's input), a site you crawl for retrieval, a sign-in provider, an [OpenTelemetry collector](../running/observability.md), or the Claude Code traffic you route through the [AI Gateway](../ai-gateway/index.md). | Installing Ertir — cloning the repository, installing dependencies and engines, pulling container images — needs the network once. ### What Ertir doesn't have There is no usage analytics, crash reporting, update check, or license activation, and no vendor account: the only account is the local admin you create in the first-run wizard. Trace export is off until you set `OTEL_EXPORTER_OTLP_ENDPOINT` to a collector you run. ## Running offline or air-gapped Once it is installed, Ertir runs with no network access: the canvas, the engines, Postgres, durable runs, and retrieval with a local embedding model. To prepare a host that will never be online: 1. **Install while connected** — or on a connected machine of the same platform that you then move across. Follow [Installation](../getting-started/installation.md) through `make engines`, and run `make up` once so dependencies install and the schema migrates. 2. **Bring the weights.** Install models from the catalog while connected, or copy GGUF files and MLX model folders onto the host and [register them by path](../models/installing.md#register-a-model-already-on-disk). Avoid hand-registered `source: hf` models on an offline host: their engine looks for the weights on Hugging Face when it starts. 3. **Keep retrieval local.** Bind your [retrieval sources](../rag/index.md) to a local embedding model, and ingest by uploading files or crawling sites on your own network. 4. **Mirror what people browse.** If the MCP hub browser should work, set `THEYGENT_MCP_REGISTRIES` to a registry you host; it doubles as an allowlist of servers people can install. 5. **Run durable work in-house.** On a server, run the standalone durable worker next to the control plane — see [Durable runs](../running/durable.md). The startup price-list request fails fast and falls back to the bundled copy; nothing else expects to be online. ## Who can reach your install - **The control plane requires sign-in.** A fresh install's API is closed until the first-run wizard creates an admin; after that, every call carries a bearer token. See [Users, roles and sign-in](../running/users.md). - **The inference plane has no sign-in of its own.** It listens on `127.0.0.1` by default (`THEYGENT_INFERENCE_PLANE_HOST`), and anything that can reach port 8081 can use your models. Keep it on loopback or a private network you trust. - **The AI Gateway** in personal mode refuses any listen address but loopback; in team mode, every developer has a gateway key. See [Running the gateway](../ai-gateway/running.md). --- Source: https://docs.ertir.dev/latest/building/editor/ # The editor The editor is where you build an agent visually: drag node types onto a canvas, wire their ports together, and configure each one in a form — no code required. It is a three-column workspace over a single agent document, with a raw-JSON view for anything the forms don't expose. This page is a tour of every panel and interaction; the individual node types have their own [reference pages](nodes/index.md). If you have never built an agent, walk through [your first agent](../getting-started/first-agent.md) first — it uses this editor end to end. ## Opening the editor The editor lives at `/editor`. You reach it from the **Agents** page (in the sidebar, at `/agents`): - **New agent** opens a blank graph. - Clicking any agent card opens that agent's latest published version for editing. - Opening a **draft** (from the Drafts strip, a card's amber `draft` badge, or a `?draft=` URL) resumes that work-in-progress exactly where it left off. A blank graph is not empty. It arrives pre-seeded with an `input` node wired straight to an `output` node, named *Untitled agent*, version `0.1.0`. That starter graph already passes validation and runs — it simply echoes whatever you send it — so you can test and publish immediately, then build outward. ```mermaid graph LR IN["input"] --> OUT["output"] ``` When you open an existing agent, the sidebar collapses to icons to give the canvas room. Opening an agent that has no published versions shows an error rather than a blank canvas. As soon as you make a real edit, the editor starts **autosaving a draft** in the background — see [Drafts & publishing](saving-agents.md#drafts-the-automatic-tier). You never lose work to a reload again. ## The three columns The workspace is three resizable columns: | Column | Default width | What it holds | |---|---|---| | **Palette** (left) | 280 px | The node types you can add, grouped by category. | | **Canvas** (center) | fills | The graph itself — nodes and the edges between them. | | **Inspector** (right) | 450 px | Configuration for whatever you have selected. | Drag the splitter between columns to resize (palette 150–420 px, inspector 260–620 px); double-click a splitter to reset it. Either side panel collapses to a thin labeled rail — selecting a node or edge auto-expands a collapsed inspector so you never lose the form. The toolbar across the top carries the agent's **id**, **name**, and **version** fields, a **visual / code** toggle, the published content hash (display only), a validation indicator, the draft-save status (*Draft saved just now*, *Saving draft…*), **Revert**, **Test**, and **Publish**. Those last few are covered under [Validation](#validation), [Testing on the canvas](#testing-on-the-canvas), and [Drafts & publishing](saving-agents.md). ## The palette The palette lists every node type you can drop onto the canvas, under a header labeled **Nodes** with a search box ("Search nodes…") and category filter chips. The categories map onto a node's *kind*: | Category chip | Kind | Contains | |---|---|---| | **I/O & Human** | boundary | `human`, `input`, `output`, `subgraph` | | **Compute & Tools** | activity | `imagine`, `llm`, `quota`, `ratelimit`, `speak`, `tool`, `transcribe` | | **Control flow** | orchestration | `guardrail`, `loop`, `map`, `router`, `transform` | Groups are collapsible and show an item count. Types are sorted alphabetically within each category. !!! note "Where is the MCP node?" There is no separate MCP node in the palette. An MCP tool is one of the three *kinds* of the single **tool** node — you pick "MCP" in the inspector after dropping a tool node. A graph you import that already contains an `mcp_tool` node still renders fine. See [MCP tools](nodes/mcp.md). ## The canvas The canvas is a pannable, zoomable surface. Kind colors are consistent everywhere: boundary nodes are green, activity blue, orchestration amber — the same tones appear on the node border, its kind dot, and the palette chips. ### Adding nodes - **Drag** a palette item onto the canvas to drop a node exactly where you release it. - **Click** a palette item to add it at a cascading default position. New nodes get generated ids like `n_llm_1`. A freshly dropped `llm` node is pre-seeded with one user message containing `$in` so it is wired up to receive its input. ### Connecting ports Nodes carry small handles you drag between to make an edge. Handles come in three channels, and you can only connect like to like: | Channel | Handle look | Meaning | |---|---|---| | **data** | round, on the node sides (in on the left, out on the right) | Passes a value along the edge. Required in-ports are blue, optional gray; error out-ports red, normal green. | | **control** | amber squares, top and bottom | Pure sequencing — "run after this", no value passed. Control edges animate on the canvas. | | **tool** | violet squares (an `llm`'s `tools` port on the bottom; a tool node's `use` port on top) | A capability wire — "the model may call this tool". | Trying to cross channels is rejected with a toast: *cannot connect a `` handle to a `` handle — connect like-to-like*. Feeding a second data edge into an in-port that already has one is also rejected — an in-port takes at most one data edge, because two would be an ambiguous input. (Control edges have no such limit.) Wiring a tool node's violet `use` handle into an `llm`'s `tools` port does more than draw a line: it registers that tool as a callable capability, and the tool node's id becomes the function name the model sees. See [tools](nodes/tools.md). #### Adding and editing handles The handles a node starts with come from its type, but they are yours to change — that is how a node takes several inputs, or how a router grows one branch per outcome. Select the node and open **Ports** in the inspector: - **+ data**, **+ control** and (on an `llm`) **+ tool** add a handle to that side. New handles get a fresh id you then rename. - Each row lets you **rename** the handle — every edge already on it is rewired, so wiring survives the rename — **delete** it, mark an in-port **required**, or flag an out-port as the **error branch** (where a step's failures route). - The panel opens automatically for a router, and for any node whose handles already differ from its type's defaults — those were authored deliberately, so they are worth seeing. A node that reads several inputs gives each one its own named in-port and addresses them as `$in.`; see [referencing inputs](input-references.md). ### Moving, deleting, duplicating - **Left-drag on empty canvas** draws a box (marquee) select; dragging any selected node moves the whole selection together. - **Middle-drag, right-drag, or hold Space and drag** to pan; **scroll** to zoom. - **Right-click** a node or edge for a context menu: **Duplicate node** (a fresh copy with a new id, its label suffixed " copy", offset slightly, with *no* edges carried over) and **Delete node** / **Delete edge**. - Selecting something and pressing ++delete++ or ++backspace++ deletes it. ### Canvas controls The bottom-left controls offer **Tidy layout** (a wand that auto-arranges the nodes by longest path — a layout-only change), plus **Undo** and **Redo**. A **minimap** shows the whole graph, and a **?** button in the top-left reveals an interaction legend. A hand-written or imported graph with no saved positions is laid out automatically the first time it renders, so it is never a pile of overlapping boxes. ## The inspector What the inspector shows depends on your selection. ### A node is selected The header shows the node's kind badge, its type, and its id, with a per-node **Wizard / Code** toggle: - **Wizard** is the form. A combined **Label + icon** row comes first (the label placeholder is the node id), then one field per config key. Field labels are humanized from the camelCase keys — `maxToolIterations` becomes "max tool iterations". Required keys get a `*`. Enums render as dropdowns, nullable strings and numbers as inputs, and objects or arrays as a guarded JSON box that shows a parse error inline instead of silently dropping bad input. - **Code** is that one node's exact JSON in an editor — the escape hatch for anything the form doesn't surface. This is how you pin a child agent by `contentHash` instead of by version. Invalid JSON shows *✗ invalid — not applied* and is never committed. Renaming the node id in the JSON keeps it selected. Several node types get purpose-built panels instead of the generic form — the `llm` model and message editors, the tool **Kind** picker, the guardrail **Check** picker, the pinned-body pickers for `subgraph`/`loop`/`map`, the transcribe/speak/imagine parameter forms, and the **input fields** builder on the `input` and `human` nodes. Each is documented on that node's [reference page](nodes/index.md). !!! tip "Everything the shipped samples use is buildable in the Wizard" All thirteen [sample agents](../samples/index.md) — every node, every config field, every named port and every edge — can be built from the forms and the canvas alone. The Code view is an escape hatch, not a requirement. The one exception is pinning a child agent by content hash rather than by version, which no sample uses. !!! warning "Durable-only nodes" Adding a `human`, `subgraph`, `loop`, or `map` node shows a banner: these run only on the durable runtime. Publish the agent, then use **Run durably** (which needs the server's durable mode) or deploy it behind a trigger. Neither the in-editor Test panel nor the plain interactive **Run** path can execute them. See [durable runs](../running/durable.md). #### The label and icon picker The leading icon button in the label row opens an inline icon picker: a grid of suggested icons, a **Search all icons…** box over the full icon set (results capped, so keep typing to narrow), and **Reset to default**. Each node type has a sensible default icon, and any type the editor doesn't have a specific icon for falls back to a gear. An icon is a display choice only. It is stored in the graph's layout block, which is never part of the content hash — **changing an icon never creates a new agent version.** The same is true of dragging nodes and zooming. ### An edge is selected The edge panel shows the edge id, a **Delete edge** button, and its `source.handle → target.handle` readout. A **Channel** toggle switches a data edge ("passes a value along the edge") to control ("pure sequencing — no value passed"). A tool edge's channel is read-only — to change it, delete and re-wire. There is also a *Condition (router-driven, optional)* field; routing in practice is done with the [router node](nodes/router.md)'s `select`, and this field is reserved. ### Nothing is selected With an empty selection, the inspector shows the whole-graph panel: - **Model bindings** — a read-only list of the logical models this agent declares (each as `binding · model`). Bindings are declared for you when you pick an inference model on an `llm` node; see [models and engines](../concepts/models-and-engines.md). - **Tools** — a read-only registry derived from the tool nodes you have wired to an `llm`. - **Connections (tool / MCP auth)** — an editable list of saved connections, plus a **New connection** form. A connection has a **Name**, a **Kind** (`http_auth` or `mcp_server`), a non-secret **Config** JSON, and a write-only **Secret** field. The secret is encrypted on the server and never shown again — the connection stores only a reference, so rotating the secret never changes any agent's content hash. Connections are how [HTTP tools](nodes/tools.md) and [MCP servers](nodes/mcp.md) authenticate. ## Model parameters (the llm node) When an `llm` node is selected, its inspector includes a **Model parameters** section that edits the parameters stored on that model *binding* — literal values shared by every `llm` node that uses the same binding, and part of the agent's hashed content. A note under the form spells this out: *every llm node using this binding shares them. Empty fields send nothing (engine defaults apply).* The chat parameters, each with a **?** tooltip explaining it in plain language: | Parameter | Control | Range / values | |---|---|---| | Temperature | slider | 0–2 | | Top-p | slider | 0–1 | | Max tokens | number | — | | Stop | text | comma-separated stop strings | | Presence penalty | slider | −2 to 2 | | Frequency penalty | slider | −2 to 2 | | Seed | number | — | | **Reasoning** | select | on / off | | **Reasoning effort** | select | low / medium / high | Two more fields appear only when the model advertises the capability: **Tool choice** (auto / none / required — needs tool-calling support) and **JSON / structured output** (text / json_object — needs structured-output support). ### The reasoning toggle The **Reasoning** select turns a model's hidden thinking phase on or off. Choosing "on" or "off" sends the model's `chat_template_kwargs` with `{"enable_thinking": true}` or `{"enable_thinking": false}`; leaving it unset uses the model's own default, and a model without a thinking switch simply ignores it. This is the same reasoning control offered in [the Bench](../running/bench.md) and referenced from [chat](../chat/index.md) — a model's thinking is always kept separate from its answer and out of the run output. ### Presets If you have saved parameter presets (from the Bench), a **Load a preset…** dropdown and a **Load** button appear. Loading a preset **copies** its values into the form — the preset's name never lands in the agent, and later edits to the preset do not follow the agent. The section is hidden when there are no presets for the modality. ## The code view The toolbar's **visual / code** toggle swaps the whole canvas for the full agent document as JSON — including the layout block — in an editor with syntax highlighting, line numbers, a fold gutter, **Fold all** / **Unfold all** / **Format** / **Copy** buttons, and find (++cmd+f++ / ++ctrl+f++). It autocompletes node types, kinds (`boundary` / `activity` / `orchestration`), channels (`data` / `control` / `tool`), bindings (`mlx` / `vllm` / `llamacpp` / `openai-compatible`), sources (`hf` / `local-path` / `url`), and common property names. Two linters run: JSON syntax and the structural graph checks. The status reads *✓ applied* or *✗ invalid JSON — not applied*. **Invalid JSON blocks switching back to Visual, publishing, and test runs** — fix it first. Unparsed text also never reaches the draft (the draft holds the last valid state), so leaving while the JSON is broken warns you that the typed text would be discarded. ## Validation The editor validates continuously and mirrors the server's own checks, so what passes here is what the control plane will run. The toolbar indicator reads **valid** (green), **N warnings** (amber), or **N issues** (red), and toggles a floating **Issues** panel over the top-right of the canvas — hovering an issue flashes the offending node or edge, clicking it selects it. **Errors block Publish and test runs — never the draft.** While any exist, the Publish and Test-panel Run buttons are disabled with *Fix N validation error(s)…*, but the graph keeps autosaving as a draft (a work in progress is allowed to be broken). You will see errors like: | Situation | Message you see | |---|---| | Two nodes share an id | duplicate node id | | A type isn't a real node type | unknown node type | | A node's `kind` is wrong for its `type` | `type 'X' must have kind 'Y', got 'Z'` | | A required config field is empty | `config '' is required` | | A config value has the wrong shape | e.g. `config/foo must be string` | | An `llm` or guardrail names a model that isn't declared | `references undeclared model '' (declare it in Model bindings)` | | A guardrail is half-configured | "a rule guardrail needs a rule"; "pick a judge model"; "add a judge prompt" | | A `subgraph`/`loop`/`map` isn't pinned to one body | `must pin EXACTLY ONE of 'version' or 'contentHash'` | | A loop has no bound | `loop maxIterations must be a whole number ≥ 1` | | An edge points at a handle that doesn't exist | `'' has no out-port ''` | | An in-port is fed twice | `in-port '

' on '' is fed by >1 data edge (ambiguous)` | | A required in-port is unwired | `required in-port '

' is not connected` | | A tool node is wired as both a capability and a step | "a tool node is either a capability (wired to an llm) or a step, not both" | | The graph forms a cycle | "graph has a cycle (not allowed)" | `input` nodes and tool nodes used purely as model capabilities are exempt from the "required in-port not connected" rule. Tool (capability) edges are excluded from cycle detection. **Warnings do not block Publish** — they flag things worth a second look: - graph has no nodes; - a `human` node whose `onTimeout` is `default` but with no default value set (a timeout would flow `null`); - the graph contains durable-only node types — run it with **Run durably**; - more than one `output` node — make sure at most one executes per run (two live outputs fail the run). The code view runs the same structural graph checks as the visual canvas, so the same warnings and errors surface there too. ## Testing on the canvas You don't have to publish — or even finish — an agent to try it. The toolbar's **Test** button docks a test console under the canvas that runs the document **exactly as it sits on the canvas**, draft or not: 1. Give it an input and click **Run** — or press ++enter++ in the input field. The control **follows your input node's `modality`**: a text box for `text`, a validated JSON editor for `json`, a microphone and file attach for `audio`, image attach plus camera for `image`, a file picker for `video`/`file`. Retype the modality on the input node and the control changes immediately — no publish, no round trip. A **JSON** option stays in the mode dropdown whatever the boundary declares, for the multi-input graphs no single modality describes. See [what the modality changes](nodes/input-output.md#what-the-modality-changes). 2. **Watch the graph execute.** As each node runs it pulses on the canvas, then keeps its outcome: a green ring for success, red for an error, dimmed for a branch that was skipped. The answer streams into the panel as it is generated, with a model's thinking shown separately. 3. **Inspect the details.** Once a run exists, the panel grows two tabs — **Output** (the streamed answer) and **Trace**, an embedded run waterfall showing every node's timing; hovering a waterfall row flashes that node on the canvas. The run id next to the tabs links to the full run page. An `audio` or `image` answer plays or shows in the panel rather than printing its artifact reference. 4. **Stop** aborts a run mid-stream; closing the panel does too. A few rules, all mirrored from the server: - **Validation errors disable Run** (the same errors that block Publish) — the graph must be structurally sound to execute, even as a draft. - **Durable-only graphs can't test-run.** A graph containing `human`, `subgraph`, `loop`, or `map` needs the durable runtime — publish it and use **Run durably**. - **Test runs are real runs.** Each one is recorded under **Runs** like any other, so your test history is inspectable later. - Drag the panel's top edge to resize it; double-click to reset. ```mermaid graph LR Build["Build on canvas"] --> Test["Test in the editor"] Test -->|iterate| Build Test -->|happy| Pub["Publish"] Pub --> Run["Run from the Agents page, chat, API, or a trigger"] ``` Published agents also run from the **Agents** page: every agent card has a **Run** button that opens a bench modal. It is a split button — **Run** streams the agent interactively, and the caret menu offers **Run durably** (checkpointed; needs the server's durable mode). A durable-only agent shows a single **Run durably** button, because its nodes can't execute on the interactive path. For the full publish flow — ids, versions, and conflicts — see [drafts & publishing](saving-agents.md). For run output, statuses, and durable execution, see [running agents](../running/index.md) and [durable runs](../running/durable.md). ## Keyboard and mouse reference | Input | Action | |---|---| | ++cmd+s++ / ++ctrl+s++ | Save the draft now (no validation gate — drafts may be broken) | | ++cmd+z++ / ++ctrl+z++ | Undo (history of 100 steps, including drags) | | ++cmd+shift+z++ / ++ctrl+y++ | Redo | | ++esc++ | Deselect | | ++delete++ / ++backspace++ | Delete the selection | | Left-drag empty canvas | Box (marquee) select | | Middle / right / ++space++-drag | Pan | | Scroll | Zoom | | Right-click a node or edge | Context menu (duplicate / delete) | Undo restores node positions too, so a mistaken drag is one keystroke away. Text fields keep their native undo while you are typing in them. !!! note "Leaving with unsaved work" Because drafts autosave, leaving the editor is almost always safe. The *Leave with unsaved changes?* prompt appears only when the very latest edits haven't reached the draft yet (they're still inside the autosave window, or a save failed) — and it offers **Save draft & leave** so one click does both. Closing or reloading the tab in that window triggers the browser's own warning, and the editor still fires a last-moment draft save on the way out. ## Related pages - [Referencing inputs](input-references.md) — the `$in` token language you use in message and tool fields - [Drafts & publishing](saving-agents.md) — autosaved drafts, ids, immutable versions, and running what you publish - [Node reference](nodes/index.md) — every node type, its ports, and its configuration - [Nodes, ports and edges](../concepts/nodes-ports-edges.md) — the underlying graph model - [Models and engines](../concepts/models-and-engines.md) — logical model ids and bindings --- Source: https://docs.ertir.dev/latest/building/input-references/ # Referencing inputs Most nodes need to read a value that flowed in from an upstream node — the run's prompt, a field of a JSON object, a tool's result. You reference those values with one small token language built around `$in`. It is the same grammar everywhere it appears, and every mistake fails loudly at run time rather than passing through silently. This page covers it in full. If you are new to how values move between nodes, read [nodes, ports and edges](../concepts/nodes-ports-edges.md) first. ## The idea Every node reads its inputs through named **in-ports**. A `data` edge feeds a value onto an in-port; the node then references that port with a `$in` token. The token is always resolved **against the node it is written on** — `$in` on an `llm` node means "the value on *this* llm's input port", not some global. ## The three forms | Token | Means | |---|---| | `$in` | The whole value on the node's default in-port, which is named `in`. | | `$in.` | The whole value on a named in-port. | | `$in..` | Drill into a field of that port's value (chain more `.field` steps to go deeper). | Plus one escape: `$$` is a literal `$`. The single rule that ties them together: **the first segment after `$in.` is always a port name.** It is never guessed to be a field. This is what makes the grammar unambiguous, and it is the one thing that trips people up — so it gets its own section. ## The default-port rule (why you often write `$in.in.field`) Most nodes have exactly one input port, and it is named `in`. That naming is deliberate, and it is why `$in` alone works as a shorthand: - `$in` — sugar for "the value on the default `in` port". Use it when you want the *whole* input. - `$in.in` — the same thing, written explicitly (port `in`, no further drilling). - `$in.in.userId` — drill into the `userId` field of the value on port `in`. So when the run input is a JSON object and you want one field of it, you write `$in.in.` — the first `in` selects the port, the second selects the field. Writing `$in.userId` would instead ask for a *port* named `userId`, which almost certainly doesn't exist, and you would get a loud error suggesting `$in.in.userId` instead. !!! tip "Rule of thumb" Want the whole input? `$in`. Want one field of a JSON input? `$in.in.`. ## Worked examples ### The whole prompt into a message The run input is a plain string like `"Summarize this: …"`. An `llm` node with the default `in` port references it directly in a message: ```json { "role": "user", "content": "$in" } ``` ### One field of a structured input You run the agent with an object as its input: ```json { "userId": "u_42", "question": "What is my balance?" } ``` That object lands on the first node's `in` port. To use just the question in a message: ```json { "role": "user", "content": "The user asked: $in.in.question" } ``` `$in.in.userId` would give you `"u_42"` — useful, for example, as a [rate-limit](nodes/ratelimit-quota.md) key. ### Drilling into JSON a model produced A model emits text that happens to be JSON, such as `{"intent": "billing"}`. A downstream node reading `$in.in.intent` gets `"billing"`: while descending a path, any JSON *string* is parsed first, so you can drill into structured output a model returned as text without an intermediate parse step. This is exactly how a [router](nodes/router.md)'s `select` reads a classifier's answer. ### Templating inside longer text Tokens substitute inline anywhere in a string. In a message you can mix literal text and several tokens: ```text Hello $in.in.name — your order $in.in.orderId ships today. ``` A resolved value renders as itself if it is a string, or as JSON if it is an object or array. `$$` gives you a literal dollar sign. ## Where you can use it The same grammar is read by: - **`llm` message content** — inline in text, and inside the URL of an image part (so an image can come from `$in.in.image`). - **`tool` and `mcp_tool` `args`** — each value is either a literal or a `$in` reference. - **`router.select`** — resolves to the *name* of one outgoing handle. - **`loop.condition`** — a reference over each iteration's output. - **HTTP tool `urlTemplate`, `headers`, and `bodyTemplate`.** - **`ratelimit` and `quota` `keyExpr`** — resolves the caller key, e.g. `$in.in.userId`. Anything in an `args` map that is *not* a `$in` reference is a literal — there is no expression language and no code execution. `{"limit": 10}` sends the number `10`; `{"who": "$in.in.name"}` sends the resolved value. ## When a value is missing Absence is handled deliberately, and it differs from an outright error. **A port can legitimately have no value** — an optional in-port left unwired, a port fed only by a branch that wasn't taken this run, or a port explicitly fed `null`. In all of these the port resolves to *absent*, and so does any drill into it. Absent renders as an empty string inline and as JSON `null` in a structured argument. Drilling into an absent value short-circuits to absent rather than erroring, because the author opted into that optionality. **A missing field on a value that *is* present is still a loud error.** If `$in.in` holds `{"name": "Ada"}` and you write `$in.in.age`, the run fails with a message naming the node, the token, and the fields that *are* available. ## Loud failures There is no silent literal pass-through and no green run with nonsense. Depending on where the token sits, an unresolved reference surfaces as a template error (in an `llm`), a router error (in a `router`), or binds the tool's `err` port (in a `tool`/`mcp_tool`) — and the message tells you exactly what went wrong: | Mistake | What you see | |---|---| | Bare `$in` on a node with no `in` port | `node '': bare $in needs a default in-port named 'in' … — address one as $in.` | | A port name that doesn't exist | `node '': no in-port named ''; in-ports: […]` — with `did you mean $in.in.?` when the node has an `in` port | | A field that doesn't exist on a present value | `node '': cannot resolve $in.… : no field '' in in-port ''; available fields: …` | | An unknown token root (for example a stray `$name`) | `node '': unknown template token '$…' — the substitution token is $in, $in., or $in.. …` | Because these fail at run time, the editor's validation cannot catch every dangling reference in advance — it flags what it can (like an `llm` naming an undeclared model), but a mistyped port name inside a message only shows up when the agent runs. Test-run your agent from the [Agents page](../running/index.md) after wiring it. ## Related pages - [Nodes, ports and edges](../concepts/nodes-ports-edges.md) — how edges feed in-ports - [The llm node](nodes/llm.md) — messages, where `$in` is used most - [The router node](nodes/router.md) — `select` resolves to a handle name via `$in` - [The tool node](nodes/tools.md) — `args` templating and the `err` port --- Source: https://docs.ertir.dev/latest/building/nodes/ # Node reference An agent is a graph of **nodes** wired together by edges. Each node does one job — take an input, call a model, run a tool, choose a branch — and passes its result to the next. This page is the map: every node type, what it does, and where to read the details. If you are new to graphs, start with [Nodes, ports and edges](../../concepts/nodes-ports-edges.md) for the underlying model, then come back here to look up a specific node. The [editor tour](../editor.md) shows where these nodes live on screen. ## The palette, grouped by kind Every node has a **kind** — one of three fixed categories that decide how it behaves and where it sits in the editor's left-hand palette: | Kind | Palette group | What these nodes do | |---|---|---| | `boundary` | **I/O & Human** | Enter, leave, or pause the graph. | | `activity` | **Compute & Tools** | Do real work with side effects — call a model, a tool, an engine. | | `orchestration` | **Control flow** | Steer the run deterministically — branch, reshape, gate. | The palette derives its list directly from the set of runnable node types, so the tables below are exhaustive. Search and category chips at the top of the palette filter it; groups are collapsible. ## I/O & Human (boundary) | Type | Purpose | Reference | |---|---|---| | `input` | The agent's entry point — carries the run input into the graph. | [Input & output](input-output.md) | | `output` | The agent's exit — the value that reaches it becomes the run result. | [Input & output](input-output.md) | | `human` | Pauses the run for a person to supply input or approve. Durable-only. | [Human approval](human.md) | | `subgraph` | Runs another published agent as a single step. Durable-only. | [Subgraph, loop & map](orchestration.md) | ## Compute & Tools (activity) | Type | Purpose | Reference | |---|---|---| | `llm` | Calls a language model with templated messages; optional tool-calling and streaming. | [LLM](llm.md) | | `tool` | Runs a built-in tool or an HTTP API — as a pipeline step or a model capability. | [Tools](tools.md) | | `mcp_tool` | Calls a tool exposed by a connected MCP server. | [MCP tools](mcp.md) | | `rag` | Retrieves the most relevant passages from a RAG source — as a step or a model capability. | [RAG](rag.md) | | `crawler` | Runs a hosted scraper or crawler (an Apify Actor) with fixed input and spend caps — as a step or a model capability. | [Crawler](crawler.md) | | `transcribe` | Speech-to-text: turns audio into text. | [Audio & images](media.md) | | `speak` | Text-to-speech: turns text into audio. | [Audio & images](media.md) | | `imagine` | Generates an image from a text prompt. | [Audio & images](media.md) | | `ratelimit` | Allows or denies a request by a request-count limit per key per window. | [Rate limit & quota](ratelimit-quota.md) | | `quota` | Allows or denies a request by a token budget per key per window. | [Rate limit & quota](ratelimit-quota.md) | !!! note "The Tool node has three kinds" `mcp_tool` is not dragged from the palette — it is the **MCP** kind of the one Tool node, chosen in the inspector after you drop a Tool. The other two kinds are a built-in tool and an HTTP call. See [Tools](tools.md) and [MCP tools](mcp.md). ## Control flow (orchestration) | Type | Purpose | Reference | |---|---|---| | `router` | Sends the run down exactly one named branch, chosen at runtime. | [Router](router.md) | | `transform` | Reshapes a value with a deterministic template — no model, no I/O. | [Transform](transform.md) | | `guardrail` | Checks a value and routes it to a `pass` or `block` port. | [Guardrail](guardrail.md) | | `loop` | Runs a published agent repeatedly, feeding each output into the next. Durable-only. | [Subgraph, loop & map](orchestration.md) | | `map` | Runs a published agent once per list element, optionally in parallel. Durable-only. | [Subgraph, loop & map](orchestration.md) | !!! info "The `guardrail` exception" A guardrail's kind depends on how you configure it: an inline rule check is `orchestration`, while a model-judge check is an `activity` (it makes a real model call). The editor sets the right kind for you — you never touch it. In the palette it lives under **Control flow**. See [Guardrail](guardrail.md). ## Durable-only nodes Four node types run **only** on the durable runtime, because they wait, recurse, or fan out over time: `human`, `subgraph`, `loop`, and `map`. A graph that contains any of them cannot run on the plain interactive path (including the editor's Test panel) — publish it, then use **Run durably** (or deploy it behind a trigger). The editor marks these nodes with a "Durable-only" banner. See [Durable runs](../../running/durable.md). ## Reading a node on the canvas Each node card shows an icon, a label (falling back to the node id), and its `type` in small caps. A colored dot and border encode the kind: **green** for boundary, **blue** for activity, **amber** for orchestration. Nodes connect through **handles** on their edges: - **Data handles** are round, on the left (in) and right (out) sides. A required in-port handle is blue; an optional one is gray. An error out-port (`err`) is red. - **Control handles** are amber squares on the top and bottom — pure sequencing ("run after this"), carrying no value. - **Tool handles** are violet squares — they wire a tool node's capability into a model. See [LLM](llm.md). You can only connect like to like (data to data, control to control, tool to tool). More on this in [The editor](../editor.md). ## What isn't here A handful of type ids exist in the underlying schema but do not run yet and never appear in the palette: `agent`, `retriever`, `memory`, `code`, `condition`, and `iterator`. If you hand-write raw IR that uses one, the control plane refuses to run it. Build with the runnable types above. ## Related pages - [Nodes, ports and edges](../../concepts/nodes-ports-edges.md) — the model behind every node - [The editor](../editor.md) — the palette, canvas, and inspector in practice - [Referencing inputs](../input-references.md) — the `$in` templating grammar every node shares - [Drafts & publishing](../saving-agents.md) — turn a graph into an immutable, runnable version --- Source: https://docs.ertir.dev/latest/building/nodes/input-output/ # Input & output nodes Every agent begins at an **input** node and ends at an **output** node. The input node carries whatever you pass to a run into the graph; the output node's value is what the run returns. Both are `boundary` nodes — they mark the edges of the graph and perform no work of their own. A fresh graph in the editor already contains one of each, wired together, so it passes validation and runs (echoing its input) from the start. ```mermaid graph LR IN([input]) --> OUT([output]) ``` --- ## The `input` node The single entry point of the agent. The value you pass when you run the agent appears on its `out` port, ready for downstream nodes to read with [`$in`](../input-references.md). It is a `boundary` node and is **never skipped** — it has no inbound edges, so it always runs first. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `out` | out | — | Carries the entire run input to whatever is wired downstream. | An input node has no in-ports. ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `modality` | enum | `text` | The expected shape of the input: `text`, `audio`, `image`, `video`, `json`, or `file`. | | `schema` | JSON Schema | `null` | An optional description of the input's structure. | The inspector edits `schema` as an **input fields** list — one row per field with a name, a type, a **req** checkbox and an optional description — rather than hand-written JSON Schema. Those names are exactly what a caller sends and what the graph reads with `$in.in.`. A schema that says more than a field list can (an array payload, a nested object) keeps the JSON editor, and a **JSON** link switches to it deliberately at any time. Both fields are **declarative** at run time: the value on `out` is exactly what was passed in, whatever the modality says. What `modality` *does* drive is every place you run the agent from — see [What the modality changes](#what-the-modality-changes) below — and how a trigger maps its incoming payload. Leaving `modality` unset is the same as `text`; that is the default the graph is validated with. In the editor, the modality dropdown's blank option (`— default (text)`) clears the field back to that default. ### What the modality changes {: #what-the-modality-changes } Every surface that runs an agent reads the input node's modality and offers the matching control — the canvas **Test** panel, the **Run** dialog on the Agents page, **New Chat**, and a reopened session all behave identically: | `modality` | What you get | What the run receives | |---|---|---| | `text` | A text box | the string | | `json` | A validated JSON editor (malformed JSON is refused before the run starts) | the parsed object | | `image` | Image attach + camera, plus an optional question | `{"image": "", "text": "…"}` | | `audio` | Microphone record + audio file attach | `{"ref": "art_…", "contentType": "audio/wav"}` | | `video`, `file` | A file picker | `{"ref": "art_…", "contentType": "…"}` | Non-text payloads that are not images ride as an **artifact reference**: the bytes are uploaded once and the run carries only the handle, so multi-megabyte blobs are never journaled through a run's steps. An image is the exception — it goes inline as a data URI, because the [llm](llm.md) node substitutes the value straight into an `image_url` content part without fetching anything. Every one of those surfaces also keeps a **JSON** escape hatch in its input-mode dropdown, whatever the boundary declares: a multi-input agent whose payload no single modality describes can always be exercised by hand. ### Behavior notes - **The whole input lands on `out`.** Downstream nodes read it with the bare `$in` token (the default in-port). To pull a field out of a structured input, drill into it — `$in.in.` — from a node that reads it. See [Referencing inputs](../input-references.md). - **Composing several inputs.** The input node has a single `out` port; it does not split values for you. When an agent needs to combine several upstream values — say a document *and* a question — you give the **consuming** node (typically an [llm](llm.md)) more than one named in-port and wire a separate edge into each. That node then addresses them distinctly as `$in.`. This is how multi-input agents work: a run input like `{"path": "...", "question": "..."}` fans out to nodes whose named ports each pick up their piece. - **Triggers live here, not as nodes.** When you select the input node, the inspector shows a read-only **Triggers** panel listing how this agent can be invoked (schedule, webhook, token). Triggers are added *after* publishing the agent, not as graph nodes — see [Triggers and webhooks](../../running/triggers.md). ### Example An input node feeding two downstream readers that each take a slice of a structured input: ```mermaid graph LR IN([input]) --> A[tool: read file] IN --> B[tool: carry question] A --> LLM[llm] B --> LLM LLM --> OUT([output]) ``` ```json { "id": "n_in", "type": "input", "kind": "boundary", "config": { "modality": "json" }, "ports": { "in": [], "out": [{ "id": "out" }] } } ``` ### Works well with - [The llm node](llm.md) — the usual first consumer of the input - [Referencing inputs](../input-references.md) — `$in`, `$in.`, and field drilling - [Runs and sessions](../../concepts/runs-and-sessions.md) — what a "run input" is --- ## The `output` node The exit point. Whatever value reaches its `in` port is the run's **canonical output** — the value returned by the run and stored against it. It is a `boundary` node and performs no side effect; it just marks where the answer comes from. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value that becomes the run's output. | An output node has no out-ports. ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `modality` | enum | `text` | The shape of the result: `text`, `audio`, `image`, or `json`. | | `schema` | JSON Schema | `null` | An optional description of the output's structure. | For an `audio` or `image` result the value is a **reference** to a produced artifact, not the raw bytes — see [Audio & images](media.md). Every run surface reads that reference back and plays or shows the result instead of printing the handle. A graph may declare several output nodes with **different** modalities — a spoken answer plus a text error branch is the usual shape. What a given run returns is therefore decided from the value it actually produced, not from what the boundary declared: a voice agent whose transcription failed hands back prose on its error branch, and that renders as prose. ### Behavior notes The output node has two rules worth knowing, because both surface as loud failures rather than silent surprises: - **Exactly one in-port.** An output node is a single-value consumer: it takes the value from one in-port. Declaring more than one in-port is a validation error — the editor will not let you publish it, because which value is "the output" would be ambiguous. - **At most one output may run per run.** A graph is allowed to contain **several** output nodes — for example, a happy-path output and a refusal output on the two branches of a [guardrail](guardrail.md). But no more than one may actually execute in a single run. If two output nodes both go live, the run fails with a clear message: *a second output node executed — the run output would be ambiguous.* Route exclusive branches (with a [router](router.md) or a guardrail) so that exactly one output is reachable. - **Empty output is honest.** If the output node's in-port is fed only by a branch that was not taken — for example an upstream tool bound its `err` port and its success path was skipped — the run does not report a green empty success. It carries an honest error note explaining the upstream cause. ### Example A guardrail that routes to one of two outputs — a valid two-output pattern, because only one is ever live: ```mermaid graph LR IN([input]) --> G{guardrail} G -->|pass| LLM[llm] LLM --> OUT([output: answer]) G -->|block| REFUSE([output: refusal]) ``` ```json { "id": "n_out", "type": "output", "kind": "boundary", "config": { "modality": "text" }, "ports": { "in": [{ "id": "in" }], "out": [] } } ``` ### Works well with - [The guardrail node](guardrail.md) — refuse on a separate `block` output before expensive work - [The router node](router.md) — send the run down exactly one branch - [Runs and sessions](../../concepts/runs-and-sessions.md) — where the run output is stored and read back --- Source: https://docs.ertir.dev/latest/building/nodes/llm/ # The LLM node The `llm` node calls a language model. You point it at a model, give it a list of chat messages (with your input templated in), and it streams back a reply. It is the workhorse of most agents. On top of a plain completion it can also drive **autonomous tool-calling**, accept **images** for vision models, replay **session memory**, and surface a model's **reasoning** as a separate stream. It is an `activity` node — it does real work with a side effect (the model call). ```mermaid graph LR IN([input]) --> LLM[llm] LLM --> OUT([output]) ``` ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The input the messages template against with `$in`. | | `tools` | in | no | A **tool channel** port. Wire tool nodes here to let the model call them. | | `out` | out | — | The model's answer. | The `tools` in-port is a violet **tool handle** on the bottom of the node — it carries capabilities, not data. Wiring it is covered under [Tools the model can call](#tools-the-model-can-call). ## Configuration These are the keys on the node itself (camelCase, as the editor and IR store them): | Field | Type | Default | Description | |---|---|---|---| | `model` | string | — (required) | The **binding key** — the local alias of one entry in the graph's `models` map (not the inference-plane logical id, which is that binding's own `model` field). See [agents and graphs](../../concepts/agents-and-graphs.md). | | `messages` | list | — (required) | The chat turns. Each is `{ "role": ..., "content": ... }`. A new node is seeded with one user turn containing `$in`. | | `toolChoice` | enum / object | `"auto"` | Whether the model may call tools: `auto`, `none`, `required`, or a named-function object. Only meaningful when tools are wired. | | `maxToolIterations` | integer | `8` | The maximum number of model-then-tool rounds before the loop stops. Must be at least 1. | | `maxToolResultChars` | integer | unset | The most characters of one tool result the model is shown; the rest is cut, with a note saying how much was left out. Unset uses the platform's `tools.result_max_chars` setting (16,000 by default). Must be at least 1. | | `tools` | list of strings | `[]` | A legacy way to attach tools by key. Prefer wiring the `tools` port instead. | Generation parameters — temperature, max tokens, and so on — are **not** on the node. They live on the model binding and are shared by every llm node that uses it, described next. ## Model binding and parameters ### Picking a model In the inspector, the **model** field is a searchable picker over two sets combined: the model bindings already declared in this graph, and the live logical model ids the inference plane currently offers. Picking a model that is not yet a graph binding **auto-declares** one for you (an entry in the graph's `models` map). You can also type a name freely — useful when authoring offline — and the validator will flag it later if it stays undeclared. A binding always names a **logical id** (like `triage-fast`), never an engine name. Which engine or endpoint actually serves that id is a registration detail you can change without touching the agent. See [Models and engines](../../concepts/models-and-engines.md) and [Installing local models](../../models/installing.md). ### Tuning parameters The **Model parameters** section edits the generation params stored on the binding (`ir.models[].params`). Because they live on the binding, **every llm node that uses the same binding shares them**. Empty fields send nothing, so the engine's own defaults apply. | Parameter | Range / values | What it does | |---|---|---| | Temperature | 0–2 | Sampling randomness. Lower is focused and repeatable; higher is more varied. | | Top-p | 0–1 | Nucleus sampling. Tune this *or* temperature, not both. | | Max tokens | ≥ 1 | Hard cap on generated tokens. Set it generously for reasoning models (see below). | | Stop | comma-separated | Sequences that end generation immediately. | | Presence penalty | −2–2 | Nudges the model toward new topics. | | Frequency penalty | −2–2 | Reduces word-for-word repetition. | | Seed | integer | Best-effort determinism for reproducible runs. On MLX, see the note below. | | Reasoning | on / off | Turns the model's hidden thinking phase on or off. Leave unset for the model's default. | | Reasoning effort | low / medium / high | How much a reasoning model thinks before answering. | | Tool choice | auto / none / required | Shown only if the model advertises tool-calling. | | JSON / structured output | text / json_object | Shown only if the model advertises structured output. `json_object` asks for valid JSON. | Each field has a **?** help tooltip. The last two are **capability-gated** — the editor hides them unless the model reports it supports them, so you never set a switch the model will ignore. !!! note "Seed and prompt reuse on MLX" A local MLX model reuses an agent's earlier prompt from one step to the next, so each step processes only what's new. A request with a **Seed** is handled on its own path, which on hybrid models such as Qwen3.5 and Qwen3.6 turns that reuse off: every step reprocesses its whole prompt. Measured on Qwen3.6-35B-A3B, a tool step's wait for the first token went from about 0.9 seconds to 2.5–3 seconds. Leave **Seed** unset for agents on these models unless you need reproducible runs. The stored keys are the OpenAI-style names (`temperature`, `top_p`, `max_tokens`, `presence_penalty`, `frequency_penalty`, `seed`, `stop`, `tool_choice`, `response_format`). The **Reasoning** switch is stored as `chat_template_kwargs` (`{ "enable_thinking": true }` or `false`) and **Reasoning effort** as `reasoning_effort`. A local model (llama.cpp, MLX) also receives the effort as the chat-template variable `chat_template_kwargs.reasoning_effort`, which is where templates such as gpt-oss read it; a model reached by URL receives only the `reasoning_effort` field. ```json "models": { "default": { "binding": "mlx", "model": "triage-fast", "params": { "temperature": 0.2, "max_tokens": 2048 } } } ``` !!! tip "Presets" A **Load a preset…** dropdown appears when you have saved parameter presets on [the Bench](../../running/bench.md). Loading one **copies** its literal values onto the binding — the preset name never lands in the agent, so later edits to the preset do not follow it. ## Messages and templating The **Messages** panel is a no-code chat-turn list. Each turn has a **role** — `system`, `user`, or `assistant` — and a text box. You can add, remove, reorder, and insert your input with an **Insert input ($in)** helper. System turns automatically float to the top. A freshly dropped node starts with a single user turn containing `$in`. Message text is templated with the shared `$in` grammar: - `$in` — the whole value on the default in-port (`in`). - `$in.` — the whole value on a named in-port. - `$in..` — drill into a field of that port's value. An unknown token, port, or field is a **loud error** at run time, never a silent pass-through — a wrong reference fails the run with a message naming the node and the available fields. The full grammar, including how absent optional ports render, is in [Referencing inputs](../input-references.md). ```json "messages": [ { "role": "system", "content": "You are a concise assistant." }, { "role": "user", "content": "Summarize this:\n\n$in" } ] ``` To compose several upstream values in one prompt, give the node multiple named in-ports and address each one — for example a document on a `file` port and a question on a `question` port: ```json "messages": [ { "role": "user", "content": "Using only this file:\n\n$in.file\n\nAnswer: $in.question" } ] ``` ## Multimodal content (images) For a vision model, a message's `content` can be a **list of parts** instead of a plain string — text parts and image parts mixed: ```json { "role": "user", "content": [ { "type": "text", "text": "What is in this picture?" }, { "type": "image_url", "imageUrl": { "url": "$in.image", "detail": "auto" } } ] } ``` - The image `url` may be an `http(s)` URL or a `data:` URI, and it is **`$in`-templatable** — so a graph can pass an image through at run time (`$in.image`). - `detail` is optional (`low`, `high`, or `auto`). - In the editor, a message that uses this rich form falls back to a per-turn JSON box labelled "content (text + image parts)", so it is never lost when you switch between wizard and code. The model must be a vision model for image parts to mean anything. See [Vision and images](../../chat/vision-and-images.md) and [Installing local models](../../models/installing.md) for finding one. ## Sessions and memory An llm node holds no memory of its own — a single run is stateless. Conversational memory comes from a **session**: when a run belongs to a session, the control plane loads the session's earlier turns and the node **replays them verbatim before its own messages**, so the model sees the prior conversation, then this turn's rendered prompt. Only the model's `content` accumulates into the run's output, and that answer is appended to the session as a new turn — ready to be replayed on the next run. Reasoning is never folded into the stored answer. See [Runs and sessions](../../concepts/runs-and-sessions.md). ## Tools the model can call Wiring a tool node into the `tools` port turns it into a **capability**: the model may call it on its own, decide the arguments, read the result, and continue. This is autonomous tool-calling, and it is entirely visual. 1. Drop a [tool](tools.md), [MCP tool](mcp.md), [rag](rag.md), or [crawler](crawler.md) node onto the canvas and configure it. 2. Drag its violet **`use`** handle (on top of the tool node) into this llm node's violet **`tools`** port (on the bottom). 3. The **Tools the model can call** panel now lists that tool, with a kind badge (builtin / http / mcp / rag / crawler). The panel's **Add MCP tools** and **Add crawler** buttons do all three in one step: pick tools from an MCP server, or a crawler from the store, and they land configured and already wired into this port. A few things follow from wiring a capability: - **The tool node's id becomes the function name** the model sees and calls. Name your tool nodes meaningfully. - When at least one tool is wired, the panel exposes **tool choice** (auto — the model decides / required — it must call a tool / none — it never calls), **max tool iterations** (the round limit, default 8) and **max tool result characters** (see below). - The model runs a **bounded loop**: it answers or calls a tool; each tool result is fed back; it calls again until it answers or hits the iteration limit. - **Each tool result is capped.** The model is shown at most 16,000 characters of one result by default, about 4,000 tokens. A longer result, such as a web page or a crawl, is cut, and a note at the end says how many characters were left out, so the model knows it saw only part of it. Set **max tool result characters** on the node to change the limit for its tools, or change the default for every agent under **Settings → Tools** ([`tools.result_max_chars`](../../reference/settings.md#tools)). The run inspector still shows each tool's whole result, and the tool's span in the waterfall records its full length (`tool.result_chars`) and whether it was cut (`tool.result_truncated`). - There is deliberately no compile-time capability check. A model that does not support tool-calling simply returns a plain answer and ignores the tools. - Local models that write tool calls as text work too: Llama-style calls and gpt-oss's own reply format on MLX are turned into real tool calls, and gpt-oss's reasoning is kept out of the answer (it arrives as the reasoning stream). A tool node is *either* a capability (wired to an llm's `tools` port, the model supplies its arguments) *or* a plain pipeline step (wired with data edges, you template its arguments) — never both. See [Tools](tools.md). ## Streaming and reasoning output The llm node **streams**. As the model generates, tokens arrive incrementally, so a long answer appears as it is written rather than all at once — you see this live in [chat](../../chat/index.md), on [the Bench](../../running/bench.md), and in the [run waterfall](../../running/observability.md). Reasoning models emit a hidden **thinking** phase before their answer. Ertir keeps the two apart: - **Reasoning** is streamed as its own frame — visible progress during a long think — but is **never** merged into the answer. - Only the final **content** becomes the run's output and the session turn. Use the **Reasoning** parameter to turn thinking on or off where the model supports it. !!! warning "Reasoning and the token cap" A reasoning model spends tokens thinking *before* it answers. If **Max tokens** is too low, the model can use its whole budget on hidden reasoning and return an empty visible answer. When that happens the run does not pretend to succeed — it finishes with an honest note pointing at the cause and the fix (raise max tokens). Set the cap generously for these models. ## Behavior notes - **Undeclared model.** Referencing a model that is not in the graph's bindings is a validation error (*references undeclared model '…'*) — declare it in the model picker or the bindings list before saving. - **Loud templating.** Any bad `$in` reference in a message fails the run with a clear reason; it never silently emits a literal `$in`. - **No side effects to undo.** The node reads a model and returns text; retrying it is safe. ## Full example A single-shot summarizer: ```json { "id": "n_llm", "type": "llm", "kind": "activity", "config": { "model": "default", "messages": [ { "role": "system", "content": "You summarize in one sentence." }, { "role": "user", "content": "$in" } ], "toolChoice": "auto", "maxToolIterations": 8 }, "ports": { "in": [ { "id": "in" }, { "id": "tools", "role": "tool", "required": false } ], "out": [{ "id": "out" }] } } ``` ## Works well with - [Tools](tools.md) and [MCP tools](mcp.md) — give the model capabilities to call - [Referencing inputs](../input-references.md) — the `$in` grammar used in every message - [Models and engines](../../concepts/models-and-engines.md) — logical ids and where inference runs - [Guardrail](guardrail.md) — check a prompt or a model answer before or after the call - [Runs and sessions](../../concepts/runs-and-sessions.md) — conversational memory across runs --- Source: https://docs.ertir.dev/latest/building/nodes/tools/ # Tool nodes A **tool** node gives an agent an action beyond generating text: it runs a small built-in function or calls an HTTP API and passes the result on. It is an **activity** node — it has real side effects (network calls), so it is treated as non-deterministic work. There is one Tool node with three flavours, chosen from the **Kind** picker in the inspector: - **Builtin** — a small function Ertir ships. Two are available: `echo` and `http_fetch`. - **REST** — call any HTTP API, optionally authenticated through a saved connection. - **MCP** — a tool exposed by a connected server. Picking MCP swaps the node to the `mcp_tool` type, which has its own page: [MCP tools](mcp.md). This page covers **Builtin** and **REST**. Whichever flavour you pick, a tool node is used in one of two ways — as a step in the graph, or as a capability the model calls on its own. ## Two ways to use a tool A single tool node can only be one of these at a time — it is **either** a capability **or** a step, never both. The editor and the validator enforce that; mixing capability wiring and data wiring on the same node is a validation error. === "As a step (you call it)" Wire a **data** edge into the tool's `in` port and read its result from `out`. The node runs at a fixed point in the graph, once per run. Its arguments come from the graph — you template them with [`$in` references](../input-references.md). Use this when *you* decide when the tool runs (fetch a page, then summarise it). === "As a capability (the model calls it)" Wire the tool's violet `use` handle into an [llm node's](llm.md) `tools` port with a **tool** edge. Now the model decides when — and whether — to call the tool during its reasoning loop, and it supplies the arguments itself. Use this to give an assistant abilities it invokes autonomously. Two rules make capability tools work: - **The node's id is the function name the model sees.** Rename the node (in the inspector's label row, which also renames its id) to give the model a clear, descriptive name like `search_web`. - **A capability tool must describe itself.** It needs a `description`, and a REST capability also needs a `parameterSchema` (see below). The two built-ins already self-describe, so they work as capabilities with no extra config. A capability tool is exempt from the "required in-port must be connected" rule — its `in` port is left unfed on purpose. ## Ports Both Builtin and REST tool nodes share the same ports. | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes* | The node's input, used when the tool runs as a step. *Left unfed when the node is a capability (exempt from the required-port check). | | `out` | out | — | The tool's return value on success. | | `err` | out | — | The error message when the call fails. Its port type is `error`. | | `use` | out | — | The violet capability handle. Wire it into an llm's `tools` port to expose the tool to the model. | ## Built-in tools Built-ins are chosen with the **Builtin** kind. Set the `tool` field to the built-in's name and pass its arguments in `args`. | Field | Type | Default | Description | |---|---|---|---| | `tool` | string | *(required)* | The built-in's name: `echo` or `http_fetch`. | | `args` | object | `{}` | Arguments for the built-in. Values are literals or `$in` references. | There are exactly two built-ins: ### `echo` Returns the value it is given, unchanged. It touches no network — it exists to test that tool dispatch and argument templating work. | Argument | Type | Default | Description | |---|---|---|---| | `value` | any | `null` | The value echoed back verbatim. | ```json { "id": "echo_it", "type": "tool", "kind": "activity", "config": { "tool": "echo", "args": { "value": "$in" } } } ``` ### `http_fetch` Performs an HTTP `GET` and returns the response as an object with three keys: `status` (the integer status code), `body` (the response text), and `headers` (an object). A non-200 status is a **normal return value**, not an error — only a transport failure (a timeout, an unreachable host, a refused connection) binds the node's `err` port. | Argument | Type | Default | Description | |---|---|---|---| | `url` | string | *(required)* | The absolute `http(s)` URL to fetch. | | `timeout_s` | number | `10` | Per-request timeout in seconds. | ```json { "id": "fetch_page", "type": "tool", "kind": "activity", "config": { "tool": "http_fetch", "args": { "url": "$in", "timeout_s": 15 } } } ``` !!! note "Argument names use `snake_case`" `http_fetch` takes `timeout_s`, not `timeoutSeconds`. Built-in argument names are the function's own parameter names. This is distinct from the REST tool's `timeoutSeconds` config field below. ## The REST (HTTP) tool The **REST** kind calls any HTTP endpoint. In the inspector you configure the URL template, the method, an optional connection for authentication, and — for capability tools — a parameter schema. The remaining fields are edited in the node's **Code** view (the per-node JSON editor). | Field | Type | Default | Description | |---|---|---|---| | `tool` | string | *(required)* | A logical name for the tool. | | `urlTemplate` | string | `null` | The request URL. `$in` placeholders are filled in (from the wired input in step mode, or from the model's arguments in capability mode). | | `method` | enum | `null` | One of `GET`, `POST`, `PUT`, `PATCH`, `DELETE`. | | `connection` | string | `null` | The id of an `http_auth` connection that supplies authentication server-side. The IR stores the id, never the secret. | | `headers` | object | `{}` | Extra request headers. | | `bodyTemplate` | object | `null` | The request body for `POST`/`PUT`/`PATCH`. For GraphQL, a `{"query": …, "variables": {…}}` shape. | | `parameterSchema` | object | `null` | A JSON Schema describing what the model fills in. Required when the tool is a capability. | | `responseMap` | string | `null` | A mini-JSONPath (e.g. `$.data.items[0]`) that reshapes the response before it leaves the `out` port. | | `idempotencyKey` | string | `null` | Sent as an `Idempotency-Key` header; `{runId}` and `{nodeId}` are expanded. | | `timeoutSeconds` | number | `null` | Per-request timeout. Defaults to 30 seconds when unset. | | `description` | string | `null` | One line telling the model when to call the tool (capability mode). | | `args` | object | `{}` | Templated values for the URL/body placeholders in step mode. | ### Behavior of an HTTP call - **A non-2xx status is a normal result.** The `out` port receives `{status, body, headers}` for any status the server returns. Only a transport failure (timeout, DNS failure, connection refused) or a failure while setting up authentication binds `err`. - **Redirects are not followed.** A 3xx response comes back as an ordinary value on `out`. This protects your authentication headers from leaking to a redirect target. - **`responseMap` never fails a 2xx.** If the JSONPath does not match anything, the unmapped response is returned instead of an error. - A connection's `baseUrl` (if set) prefixes a relative `urlTemplate`, and a connection's static `headers` merge underneath the node's own headers. ## Authentication with connections and secrets A REST tool never carries a credential in the agent document. Instead it references a **connection** by id, and the credential lives encrypted on your machine. When the tool runs, the connection is resolved server-side, inside the step — the secret is injected into the request there and never appears in the IR or in observability data. ### Create a connection In the [editor](../editor.md), click empty canvas so nothing is selected. The inspector then shows a **Connections (tool / MCP auth)** panel. Choose **New connection**, then: 1. Give it a **Name**. 2. Set **Kind** to `http_auth`. 3. Put non-secret settings — the auth type, header names, base URL — in **Config (non-secret JSON)**. 4. Put the credential in the **Secret (write-only)** field. It is encrypted at rest and never shown again. Back on the REST tool node, pick the connection from the **connection (http auth)** dropdown. !!! tip "Rotating a secret does not change the agent version" A connection's secret is stored under a stable reference, so editing (rotating) the secret leaves every agent's content hash unchanged. See [Versioning](../../concepts/versioning.md) for why the hash matters. ### Auth types Set `auth.type` in the connection's config. The secret you supply is used according to the type: | `auth.type` | The secret is… | What the tool sends | |---|---|---| | `bearer` | the token | `Authorization: Bearer ` | | `api_key` | the key | The key in a header named by `auth.header` (default `X-API-Key`) | | `basic` | the password | `Authorization: Basic base64(:)` | | `oauth2_client_credentials` | the client secret | POSTs `auth.tokenUrl` (`grant_type=client_credentials`, `auth.clientId`, optional `auth.scope`), then sends the returned access token as a bearer | If a connection has no auth type or no secret, the request proceeds without authentication. A connection config for a bearer-token API looks like: ```json { "auth": { "type": "bearer" }, "baseUrl": "https://api.example.com" } ``` …with the token itself supplied in the write-only Secret field. !!! warning "Encryption key" Encrypted secrets are keyed by `THEYGENT_SECRET_KEY`. If you do not set it, an ephemeral key is generated with a loud warning and your stored secrets will not survive a restart. See the [environment reference](../../reference/environment.md). ## The success / error contract On success, a tool binds its return value to `out` (and any other non-error handle). On failure it binds the error **message** to `err` and **the run keeps going** — a tool error is structured output, not a crashed run. Downstream nodes fed only by the un-taken branch are skipped. Wire the `err` port somewhere (an output node, a fallback branch) if you want to handle failures; leave it unwired to simply stop that branch. ## Worked example — a tool as a step Fetch a URL, then summarise the page with an llm. ```mermaid graph LR in([input]) --> fetch["tool · http_fetch"] fetch -->|out| sum["llm · summarize"] sum --> out([output]) ``` The `input` value (a URL string) flows into `http_fetch` via `$in`; the fetched `{status, body, headers}` object flows to the llm, whose message drills into it with `$in.in.body`. The tool's `err` handle is left unwired here, so a transport failure leaves the `out` branch dead and the run completes with an honest "upstream error" note instead of a value. ## Worked example — a tool as a capability Give an assistant the ability to search, and let it decide when to call it. ```mermaid graph LR in([input]) --> llm["llm · assistant"] llm --> out([output]) tool["tool · search_web"] -. tool .-> llm ``` The `search_web` node's `use` handle is wired into the llm's `tools` port (the dotted **tool** edge). Because it is a capability, its URL placeholders are filled by the model's function arguments, and its `parameterSchema` tells the model what to supply: ```json { "id": "search_web", "type": "tool", "kind": "activity", "config": { "tool": "search_web", "urlTemplate": "https://api.example.com/search?q=$in.query", "method": "GET", "connection": "con_01hzy…", "description": "Search the web for a query and return the top results.", "parameterSchema": { "type": "object", "properties": { "query": { "type": "string", "description": "the search query" } }, "required": ["query"] } } } ``` The `$in.query` placeholder in the URL and the `query` property in the schema must line up: every placeholder needs a matching property, and every property needs a placeholder. A mismatch is a validation error caught before the agent runs. ## Works well with - [The llm node](llm.md) — the `tools` port that turns a tool into a model capability. - [MCP tools](mcp.md) — tools hosted by external servers, wired the same way. - [Input references](../input-references.md) — the `$in` grammar for templating arguments and URLs. - [Observability](../../running/observability.md) — inspect a tool node's input and output in the run waterfall. --- Source: https://docs.ertir.dev/latest/building/nodes/mcp/ # MCP tools An **mcp_tool** node calls one tool on an external MCP server. MCP — the Model Context Protocol — is an open standard for exposing tools (a filesystem browser, a database client, a computer-vision model, a company API) to an agent. Where a [built-in or REST tool](tools.md) is defined inside Ertir, an MCP tool lives in a separate server process that you register once and then reference from any agent. It is an **activity** node. ## Where the node comes from `mcp_tool` is not a separate item in the node palette. To add one, drop a **Tool** node onto the canvas and choose **MCP** in the inspector's **Kind** picker. (Agents you import that already contain `mcp_tool` nodes still render normally.) ## Ports Same shape as the other tool nodes. | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes* | The node's input, used when the tool runs as a step. *Left unfed when the node is a capability (exempt from the required-port check). | | `out` | out | — | The tool's return value on success. | | `err` | out | — | The error message when the call fails. Its port type is `error`. | | `use` | out | — | The violet capability handle. Wire it into an llm's `tools` port to let the model call the tool. | ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `tool` | string | *(required)* | The tool name the server exposes. In the inspector this is a **searchable picker** over the tools the chosen server actually reports; it degrades to a free-text field when the server can't be reached, so authoring is never blocked. | | `server` | string | `null` | A name-keyed MCP server's name (registered through the admin API). Set **this or** `connection`, never both. | | `connection` | string | `null` | The id of an `mcp_server` connection — anything created on the [MCP page](../../mcp/index.md): hub installs, remote servers with auth, OpenAPI/GraphQL servers, stdio servers with secret env. Set **this or** `server`, never both. | | `args` | object | `{}` | Arguments for the tool in step mode. `$in` references are allowed. In capability mode the model supplies them. | | `description` | string | `null` | An optional "use this when…" hint for the model in capability mode. MCP tools already describe themselves through the server, so this is a nudge, not a requirement. | You must set **exactly one** of `server` or `connection`. The inspector shows both pickers and picking one clears the other automatically; the tool picker works the same against either target. ## The MCP page Servers are set up and managed on the **MCP** page in the sidebar, which has [its own section in these docs](../../mcp/index.md). The short version: **Browse hubs** installs a published server from an MCP registry, and **Add server** defines one by hand — a local **stdio** subprocess, a **remote** HTTP/SSE endpoint, or a generated server derived from an **OpenAPI** spec or a **GraphQL** endpoint. Everything lands in one list with transport and connected badges and per-row **Tools** / **Warm** / **Close** / **Delete** actions (plus **Connect** for OAuth sign-in). Below the list, the **tool tester** runs a single tool through a throwaway one-node graph so you can check it works before wiring it into an agent. When a tool returns image detections (bounding boxes with labels and scores), they are drawn over the image — handy for computer-vision servers. Everything the MCP page creates is an `mcp_server` **connection**, referenced from an `mcp_tool` node via `connection`. The `server` field remains for name-keyed registrations made through the [admin API](../../reference/api.md) — plain stdio or remote servers that carry no secret. ## How tools are discovered You do not list a server's tools by hand — the server reports them. When a server first connects (on first use, or when you press **Warm** or **Tools**), Ertir calls the server's tool-listing method and caches the result for the connection's lifetime. That cache also drives validation. When you reference a tool in an `mcp_tool` node: - If the server is currently connected, the tool name is checked against its reported tools up front — an unknown name is rejected before the agent runs (`mcp_tool_not_found`). - If the server is not connected, the agent is accepted and the check happens lazily at run time — a missing tool binds the node's `err` port instead of failing the run. An unregistered server name is always rejected up front (`mcp_server_not_found`). ## Secrets in server config MCP servers run in your trust domain — the same posture as your inference engines. How a credential reaches the server depends on the transport: - **stdio servers** receive secrets through their environment: the Add server form's **Secret env** lines are stored encrypted and injected into the subprocess at launch (non-secret **Env** lines are stored plainly). Neither is ever echoed back or logged with its values. - **Remote and generated servers** get their authentication built server-side from the connection's encrypted secret — a bearer token, an API-key header, basic credentials, a whole header map, OAuth2 client credentials, or interactively-minted OAuth tokens. The secret never appears in the agent document. The [MCP page](../../mcp/index.md) is the normal place to set all of this up. A connection can also be created from the [editor](../editor.md): click empty canvas so nothing is selected and use the **Connections (tool / MCP auth)** panel, with the credential in the write-only **Secret** field. ## Calling MCP tools from an llm Like any tool node, an `mcp_tool` can be a **step** (wired with data edges, arguments templated with `$in`) or a **capability** the model calls itself. To make it a capability, wire its violet `use` handle into an [llm node's](llm.md) `tools` port with a **tool** edge. Two things make MCP capabilities especially convenient: - **The node's id becomes the function name** the model calls — rename the node to give the model a clear name. - **MCP tools self-describe.** The server already provides each tool's schema, so — unlike a REST capability — you do not write a `parameterSchema`. An optional `description` on the node nudges the model on when to use it. ## Behavior notes - **Error contract.** A tool-level error the server reports, an unknown tool name, or a connection failure all bind the node's `err` port; the run continues. A tool error is structured output, not a run failure. - **Reconnect once, then fail honestly.** On a transport failure the manager retries the connection once; if it still fails, the node binds `err`. - **Lazy connect and reuse.** A server connects on first use and stays connected for the control plane's lifetime (there is no idle-timeout teardown — server processes are cheap). **Warm** pre-connects; **Close** tears the connection down but keeps the registration. - **Per-call timeout.** A single tool call is capped at 120 seconds by default, tunable with `THEYGENT_MCP_CALL_TIMEOUT_S` (see the [environment reference](../../reference/environment.md)). - **Wrong node type.** A plain `tool` node pointing at an MCP binding is rejected up front (`tool_binding_mismatch`) — use an `mcp_tool` node instead. ## Worked example — a filesystem tool as a capability Give an assistant read access to files through a filesystem MCP server, and let it decide when to read. ```mermaid graph LR in([input]) --> llm["llm · assistant"] llm --> out([output]) fs["mcp_tool · read_file"] -. tool .-> llm ``` Add a `filesystem` server on the MCP page (**Add server → Stdio**: command `npx`, args `-y @modelcontextprotocol/server-filesystem /tmp`), then add a Tool node, choose **MCP**, pick the server, and pick `read_file` from the tool list: ```json { "id": "read_file", "type": "mcp_tool", "kind": "activity", "config": { "tool": "read_file", "connection": "con_01hzy…" } } ``` (A name-keyed server registered through the admin API is referenced with `"server": "filesystem"` instead.) Wiring its `use` handle into the llm's `tools` port exposes it as a `read_file` function the model can call. To use the same node as a **step** instead — reading one fixed file each run — feed its `in` port and template the path, for example `"args": { "path": "$in.in.path" }` when the run input is an object like `{"path": …, "question": …}`. See [Input references](../input-references.md) for the `$in` grammar. ## Works well with - [MCP servers](../../mcp/index.md) — the MCP page: browse hubs, the four ways to add a server, OAuth sign-in. - [The llm node](llm.md) — the `tools` port that turns an MCP tool into a model capability. - [Tool nodes](tools.md) — built-in and REST tools, wired the same two ways. - [Input references](../input-references.md) — the `$in` grammar for templating arguments in step mode. - [Nodes, ports, and edges](../../concepts/nodes-ports-edges.md) — how tool-channel edges differ from data edges. --- Source: https://docs.ertir.dev/latest/building/nodes/rag/ # RAG A **rag** node retrieves from a [RAG source](../../rag/index.md) — a named document collection you filled by uploading files or crawling a site. It runs the source's hybrid search (semantic similarity fused with keyword full-text) and returns the best-matching passages, each with its document, heading path, and scores. It is an **activity** node. Like a [tool node](tools.md), it has two lives: a **step** in the dataflow, or a **capability** the model calls on demand. The capability wiring is what powers "answer from my documents" agents — the model chooses when to search and what to ask. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | no | The query in step mode (used when `query` is not set). Left unfed in capability mode — the model supplies the query. | | `out` | out | — | The retrieval result on success. | | `err` | out | — | The error message when retrieval fails. Its port type is `error`. | | `use` | out | — | The violet capability handle. Wire it into an llm's `tools` port to let the model search. | ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `source` | string | *(required)* | The RAG source's id, picked from a dropdown in the inspector. The id is stable: re-ingesting the source never changes this agent's version. | | `topK` | int | `5` | How many passages to return (1–50). | | `minSimilarity` | float | `null` | Optional quality floor on semantic similarity (0–1). Keyword-only matches are not filtered by it — exact-term hits are kept on purpose. | | `query` | string | `null` | Step mode only: the query as a [`$in` template](../input-references.md). Leave empty to use the `in` port's whole value. Ignored in capability mode. | | `description` | string | `null` | Capability mode: the "use this when…" hint the model reads to decide when to search. Defaults to a generic knowledge-base description — a specific one ("Search the payments API documentation") makes the model call it at the right times. | ## As a capability (the model searches) 1. Drop a **rag** node and pick its source. 2. Drag its violet **`use`** handle into the llm node's violet **`tools`** port. The model now sees a function named after the node's id taking one argument, `query`. When it calls, Ertir runs the retrieval and feeds the matches back as the tool result; the model reads them and answers (usually citing the `uri` of the passages it used). `topK` and `minSimilarity` stay authored knobs — the model cannot override them. !!! tip "Name the node meaningfully" The node **id is the function name** the model sees. `docs_search` beats `n_rag1` — models call well-named functions more reliably. ## As a step (deterministic retrieval) Wire the node inline with data edges: `input → rag → llm → output`. The query is the `in` port's value, or the rendered `query` template — for example `"$in.in.question"` when the run input is an object. The result binds to `out` and flows downstream, so a prompt can inject the passages itself (`$in` templating on the llm's messages). ## The result shape ```json { "source_id": "rag_01ABC…", "source_name": "product-docs", "query": "how do I install it", "matches": [ { "text": "Install the gadget by plugging …", "heading": "Setup > Installation", "uri": "https://docs.example.com/setup/", "title": "Setup", "score": 0.032, "similarity": 0.81, "document_id": "rdoc_…", "chunk_id": "rchk_…", "position": 3 } ] } ``` `score` is the fused hybrid rank (use it for ordering); `similarity` is the raw semantic similarity when the vector leg matched (`null` for a keyword-only hit). ## Failure modes The rag node follows the tool ok/err contract — a retrieval failure binds a clear message to `err` and the run continues: - The source exists but has **no embedded content yet** (still ingesting, or every ingest failed). - The **embedding model is unreachable** at query time. - The model behind the source's logical id started returning a **different vector dimension** (the id was repointed at another model) — re-ingest the source. One check happens earlier: an agent whose rag node references a **deleted or unknown source** is rejected before a run is created (`rag_source_not_found`). Both runtimes execute the node identically; on a [durable run](../../running/durable.md) each retrieval is a journaled step, so a resumed run never re-searches completed steps. ## Related pages - [RAG sources](../../rag/index.md) — creating and managing the collections this node searches - [LLM](llm.md) — the `tools` port and the tool-calling loop - [Referencing inputs](../input-references.md) — the `$in` grammar the `query` template uses --- Source: https://docs.ertir.dev/latest/building/nodes/crawler/ # Crawler A **crawler** node runs a hosted scraper or crawler — an [Apify](https://apify.com/store) **Actor** — and returns the results it collects: web search results with the pages as Markdown, places from a map, a site's pages, product listings. It is an **activity** node. Like a [tool node](tools.md) and a [rag node](rag.md), it has two lives: a **step** in the dataflow, or a **capability** the model calls on demand. The difference from giving a model an Actor through the [MCP hub](../../mcp/apify.md) is who decides what each call does. Here, the person building the agent chooses which fields the model fills, fixes everything else on the node, and caps what a call may cost and return. ## Adding a crawler Crawlers come from the **Crawlers browser**: - On an llm node, press **Add crawler** under **Tools the model can call**. The installed node is wired into that llm's `tools` port. - Or drop a `crawler` node from the palette and press **Browse crawlers** in its inspector. The browser searches the Apify Store. Each card shows what the store reports about the Actor: | On the card | What it tells you | |---|---| | Success rate | The share of the Actor's public runs over the last 30 days that succeeded. An Actor that times out on most runs reads as such before you install it. | | Pricing | The pricing model and its headline price — for pay-per-event Actors, the price of the main billed event (a fetched page, a scraped place) on the free plan. | | Users | How many users ran it in the last 30 days. | | Category, author | Where it sits in the store, and who publishes it. | Open an Actor to see its input fields. The fields a model can usefully fill are listed with checkboxes; the suggested ones come pre-ticked — the required fields and the few the Actor can't do without. Result counts, paid add-ons and anything with a default stay unticked, because a model offered those sets the spend. The first time, the browser asks for your **Apify API token** and stores it as a connection — encrypted, and never shown again. Later installs reuse that connection. Press **Add crawler** and the configured node lands on the canvas. !!! note "The browser goes through Ertir" Store searches go from your control plane to Apify's public Store API, which needs no token, and are cached for a few minutes. Your browser never calls Apify directly. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | no | Step mode: the value the fixed input's `$in` references read. Left unfed in capability mode. | | `out` | out | — | Step mode: the dataset items on success. | | `err` | out | — | The error message when the crawl fails. Its port type is `error`. | | `use` | out | — | The violet capability handle. Wire it into an llm's `tools` port to let the model call the crawler. | ## Configuration The inspector edits all of it; the Code view shows the stored config. | Field | Type | Default | Description | |---|---|---|---| | `provider` | string | `apify` | The crawler service. `apify` is the only one today. | | `actor` | string | *(required)* | The Actor's id, such as `apify/rag-web-browser`. **Change** in the inspector re-points the node at another Actor and keeps its id and wiring. | | `connection` | string | *(required)* | The connection whose secret is your Apify API token. The node stores the connection's id, never the token. A connection without a token shows a **Set token** field in the inspector. | | `input` | object | `{}` | The **fixed input**, sent on every call. The inspector renders it as a form generated from the Actor's own input schema — text, numbers, dropdowns, URL lists — with an **Advanced** section for proxy and other settings. An empty field uses the Actor's default. | | `expose` | list | `[]` | The fields the model fills when the node is a capability. | | `scalar` | list | `[]` | List fields the model fills as **one value per call**, wrapped into a one-item list for the Actor. On by default for exposed list fields — see [Local models](#local-models). | | `parameterSchema` | object | — | The function schema the model sees, generated from the Actor's input schema. Ticking a field regenerates it; you never edit it by hand. | | `description` | string | the Actor's description (from the browser) | The "use this when…" hint the model reads to decide when to call it. | | `limits` | object | see below | What one call may cost, take and return. | ### Limits | Limit | Default | Description | |---|---|---| | `timeoutSeconds` | `120` | How long the Actor run may take. Interactive runs stop at 300 seconds whatever this says — see [Long crawls](#long-crawls). | | `maxChargeUsd` | `0.25` | The most one call may charge. Never above the platform ceiling, **Settings → Crawlers**. | | `maxItems` | `10` | The most results a pay-per-result Actor may bill. | | `itemLimit` | `10` (`5` from the browser) | How many dataset items are read back. | | `resultChars` | `6000` | Capability mode: how many characters of results the model reads, shared equally across the items. | | `fields` | all | Keep only these top-level item fields — for example `markdown` and `searchResult` for web search. | ## As a capability (the model crawls) Wire the node's violet **`use`** handle into an llm's violet **`tools`** port (the browser does this for you when opened from an llm). The model sees a function named after the node's id, with only the exposed fields as arguments. When it calls: 1. Ertir builds the Actor input: the fixed input, plus the model's values for the exposed fields. A value for any other field is dropped, so a model can't raise a result count or switch on a paid add-on by inventing an argument. 2. The Actor runs under the node's limits. 3. The model gets the results back as text: only the chosen `fields`, each item given an equal share of `resultChars`, long items cut with a marker. A crawl can return megabytes; the model reads what fits. !!! tip "Name the node meaningfully" The node **id is the function name** the model sees. The browser names it after the Actor (`rag_web_browser`); rename it to what it does for your agent — `web_search` is easier for a model to choose. ## As a step (a fixed crawl) Wire the node inline with data edges: `input → crawler → llm → output`. The fixed input is the whole Actor input. A value that is exactly a [`$in` reference](../input-references.md) reads the node's `in` port — for example `{"query": "$in"}` searches for the run input, and `{"startUrls": "$in.in.urls"}` crawls a list of URLs from an object input. Any other string is sent as written, `$` included, so page functions and selectors are safe. The dataset items — at most `itemLimit`, projected to `fields` — bind to `out` as a list and flow downstream. ## Spend Apify bills the account that owns the token. Three things bound what a crawler node can spend: - **The node's charge cap**, `limits.maxChargeUsd`, sent with every run. It is the cap that matters for pay-per-event Actors, which most popular Actors are. - **The platform ceiling**, **Settings → Crawlers** (`crawler.max_charge_usd`, default $1.00). Every run's cap is held under it, so no agent can raise its own. Lowering it applies to the next call. An administrator can cap it from the environment with `THEYGENT_CRAWLER_MAX_CHARGE_USD`. - **The fixed input**: result counts and add-ons the model can't touch. The [run waterfall](../../running/observability.md) shows what each crawl returned and cost: a crawler node's row (or, in capability mode, the tool call's row inside the llm) shows the number of items and the charge Apify reported. Only durable runs report the charge — the synchronous call an interactive run uses names no run to ask about. ## Long crawls An interactive run — the editor's **Test** panel, chat, `POST /graphs/runs` — waits for the crawl in one request, which Apify holds for at most **300 seconds**. Ertir caps the Actor's own timeout there too, so a run never keeps going after the agent has given up. A crawl that needs longer fails with a message saying so. A [durable run](../../running/durable.md) starts the crawl, checks on it about once a minute, and reads the results when it finishes, so a crawl can take as long as `timeoutSeconds` and survives a restart in between. A resumed run keeps polling the crawl it already started instead of starting (and paying for) another. ## Local models Small local models are better at filling one or two simple fields than many. Two defaults help: - **Few exposed fields.** Each exposed field adds to every prompt; an Actor offered with every field can cost thousands of tokens per call. - **One value per call** (`scalar`). Local models write list arguments unreliably — and on MLX, a list written as a bare string can drop the request. A scalar field takes one search term or URL; Ertir wraps it into the list the Actor wants, and unpacks a model's JSON-encoded list if it writes one anyway. ## Failure modes The crawler follows the tool ok/err contract: a failure binds a clear message to `err` (or, in capability mode, goes back to the model as the tool result), and the run continues. - The connection is missing, disabled, or **has no token** — set it on the node's connection. - Apify **refuses the input** — the message is Apify's, naming the field. - Apify **refuses the token** — check it on the connection. - The crawl **exceeds 300 seconds** on an interactive run — run the agent durably, or lower the work per call. - A durable crawl **ends without succeeding**, or is still running after `timeoutSeconds` — the message names Apify's run id so you can look it up in the Apify console. ## Privacy A crawler node sends its input — the fixed input and, in capability mode, the model's arguments — to the provider it names, and the crawled results come back into your run. Your token stays in the control plane's encrypted secret store and is sent only to the connection's API address. See [Privacy and data flow](../../concepts/privacy.md). ## Related pages - [Apify through the MCP hub](../../mcp/apify.md) — every Actor field offered to the model, no node config - [LLM](llm.md) — the `tools` port and the tool-calling loop - [Tools](tools.md) — connections and the error contract - [Samples](../../samples/index.md) — the Web researcher sample agent --- Source: https://docs.ertir.dev/latest/building/nodes/router/ # Router node The **router** node sends the run down exactly one of several named branches. It reads a value from its input, matches it to the name of one of its outgoing handles, and activates only that handle — every other branch, and everything downstream of it, is skipped for this run. It is a **control-flow** (`orchestration`) node. Use it when an earlier node has already *decided* which path to take (a classifier's label, a field on the input, a guardrail's verdict) and you want the graph to follow that decision. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value to route on. A router declares **exactly one** in-port; more than one is a run error. | | `out` | out | — | The default outgoing branch. Add more out-ports (one per route) to actually branch — see [Adding branches](#adding-branches). | ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `select` | string (required) | `""` | An [input reference](../input-references.md) that must resolve to the **name of one of this node's out-ports**. This is handle-name routing, not an expression language — the value *is* the branch name. | ## How routing works `select` is resolved against the in-port value the same way every other reference is: - `$in` — the whole input value (use this when the input is already the branch name, e.g. a classifier that outputs just `"premium"`). - `$in.in.` — a field of the input object (use this when the input is an object like `{"tier": "premium"}` and you route on `tier`). Whatever `select` resolves to must be a string that **equals one of the node's out-port names**. The router then: 1. activates that one out-handle, 2. forwards the in-port value **unchanged** along it, and 3. marks every other out-handle dead. The router does not transform the value — it only chooses the path. To reshape the value on the way through, put a [transform node](transform.md) after it. ### Downstream edge-skipping Only edges leaving the selected handle carry a value. A node is **skipped** when it has incoming data edges and *all* of them are dead — so an entire branch that the router did not choose never runs. This cascades: skip the first node of a branch and everything reachable only through it is skipped too. ```mermaid graph LR IN[input] --> R{router} R -->|premium| P[premium pipeline] --> O[output] R -.->|standard branch dead this run| S[standard pipeline] -.-> O ``` Because branches are mutually exclusive, a router is the standard way to guarantee **at most one `output` node runs per run** — a rule the control plane enforces (two live outputs fail the run). Point each branch at its own output, and only the chosen one executes. ## Behavior notes - **No fallback.** If `select` resolves to nothing, to a non-string, or to a name that is not one of the node's out-ports, the run fails with a clear `router_error` (HTTP 422) naming the value and the handles it declares. The router never guesses a default branch. - **Exactly one in-port.** A router consumes a single value. Declaring more than one in-port is *not* caught by static validation — the graph passes and a run is created — but the router then fails that run with a `template_error` (HTTP 422) the moment it executes. - **Add the branches yourself.** A freshly dropped router has only the `out` handle. Routing to two or more named branches means adding those out-ports (below). ### Adding branches The wizard form edits `select` but does not add out-ports. To create named branches, select the router and switch the inspector to **Code**, then add the handles to `ports.out`: ```json { "id": "n_route", "type": "router", "kind": "orchestration", "config": { "select": "$in.in.tier" }, "ports": { "in": [ { "id": "in" } ], "out": [ { "id": "premium" }, { "id": "standard" } ] } } ``` Save the node, and the new `premium` / `standard` handles appear on the canvas ready to wire. For this example the input object must carry a `tier` field whose value is either `"premium"` or `"standard"`. ## Worked example An upstream llm classifies a support message into `billing`, `technical`, or `other`, emitting just that word. The router forwards the original message down the matching branch. ```mermaid graph LR IN[input] --> CLS[llm: classify] --> R{router select: $in} R -->|billing| B[billing agent] --> OUT[output] R -->|technical| T[technical agent] --> OUT R -->|other| G[general reply] --> OUT ``` The router's config is `{ "select": "$in" }`, and its out-ports are `billing`, `technical`, and `other`. Each branch ends at the same `output` node; only the taken branch reaches it, so exactly one answer is produced. ## Works well with - [Guardrail node](guardrail.md) — the other way to branch: split `pass` vs `block` before expensive work. - [Transform node](transform.md) — reshape a value once a branch is chosen. - [Input references](../input-references.md) — the `$in` grammar `select` uses. - [Nodes, ports, and edges](../../concepts/nodes-ports-edges.md) — how live and dead edges drive execution. --- Source: https://docs.ertir.dev/latest/building/nodes/transform/ # Transform node The **transform** node reshapes a value without an LLM. You give it a small JSON template; it copies the shape verbatim, and wherever a string leaf is an [input reference](../input-references.md) it substitutes the referenced value. It is a **control-flow** (`orchestration`) node — deterministic, with no model call and no I/O. Reach for it to adapt one node's output to the next node's expected input: pick a field out of an object, rename keys, wrap a value in a new structure, or assemble an object from several inputs. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value(s) to reshape. References in the template resolve against this node's in-ports. | | `out` | out | — | The reshaped value. There is **no `err` port** — a bad template fails the run (see below). | ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `expr` | string (required) | `""` | A JSON document that is the output shape. String leaves that are exactly an input reference are replaced with the referenced value; everything else is kept literally. | ## The operations you can express `expr` is parsed as JSON, then walked leaf by leaf: - A string leaf that is **exactly** a reference — `$in`, `$in.`, or `$in..` — is replaced with that value, keeping its real type (a number stays a number, an object stays an object). - **Every other string is a literal.** A string like `"Hello $in.in.name"` is *not* interpolated — it is stored as-is. References must stand alone as the whole leaf value. (This deliberately protects strings that contain a `$`, such as a GraphQL query.) - Numbers, booleans, and `null` pass through unchanged. - Objects and arrays are copied structurally, resolving each of their leaves the same way. From those rules you can: | Goal | `expr` | |---|---| | Take one field out of the input object | `"$in.in.city"` | | Build a new object from input fields | `{ "q": "$in.in.query", "topK": 5 }` | | Rename / reorder keys | `{ "userId": "$in.in.id", "name": "$in.in.fullName" }` | | Wrap a value in a structure | `{ "messages": [ { "role": "user", "content": "$in" } ] }` | | Combine several named in-ports | `{ "a": "$in.left", "b": "$in.right" }` | !!! note "Not a code node" The transform node runs a fixed JSON template — it never executes user code, calls a network, or reads files. It is deterministic and replays identically on a durable resume. Richer expression languages are not part of it today; if the template form can't express your reshape, do it in an [llm node](llm.md) or an [http tool](tools.md). ## Behavior notes - **Failures are loud.** If `expr` is not valid JSON, or a reference in it can't be resolved (an unknown port, or drilling into a field that isn't there), the run fails with a `transform_error` (HTTP 422). There is no `err` port and no silent pass-through — a broken reshape stops the run rather than emitting nonsense. - **Typed substitution.** Because a whole-leaf reference keeps the value's type, `{ "n": "$in.in.count" }` yields a real number `{ "n": 3 }`, not the string `"3"`. - **Multiple inputs.** If several data edges feed different named in-ports, reference each by its port name (`$in.`). See [multi-input references](../input-references.md). ## Worked example An http tool returns a large JSON body; you only want two fields to hand to the llm. ```mermaid graph LR IN[input] --> API[http tool] --> TX[transform] --> LLM[llm] --> OUT[output] ``` With the tool's result on the transform's default in-port, `expr` extracts and renames: ```json { "temperature": "$in.in.current.temp_c", "summary": "$in.in.current.condition.text" } ``` The llm node downstream then references `$in.in.temperature` and `$in.in.summary` in its message text. ## Works well with - [Input references](../input-references.md) — the `$in` grammar every leaf uses. - [Router node](router.md) — choose a branch; transform reshapes the value on it. - [Tools](tools.md) — adapt a tool's raw response before the next step. - [Nodes, ports, and edges](../../concepts/nodes-ports-edges.md) — how values flow between ports. --- Source: https://docs.ertir.dev/latest/building/nodes/guardrail/ # Guardrail node The **guardrail** node is a check that runs *before* expensive work and splits the graph two ways: input that passes flows out `pass`; input that fails flows out `block`. Wire `block` to an `output` to refuse a request before any model or tool runs. A guardrail comes in two flavours, chosen by its **check type**: - **Rule** — a deterministic, inline check (a regex, a length bound, an allow/deny list, a PII scan). No model call, so the node is a **control-flow** (`orchestration`) node. - **Model** — an LLM judge decides pass or block. That is a real inference call, so the node is a **compute** (`activity`) node. The editor stamps the right `kind` for you when you pick the check type; you never set it by hand. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value to check. | | `pass` | out | — | Carries the input **through unchanged** when the check passes. | | `block` | out | — | Carries the `onBlock` payload (or, if none is set, the original input) when the check fails. | Exactly one of `pass` / `block` activates per run; the other branch's edges are dead and everything downstream of it is skipped. ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `check` | object (required) | `null` | The check to run. Must be set — an unconfigured guardrail is a validation error. Its shape depends on the check type (below). | | `onBlock` | object | `{}` | The payload sent out the `block` port when the check fails. Leave it empty to pass the original input through instead. | The inspector gives `onBlock`'s usual shape — a refusal `message` — its own field, with the whole-payload JSON box beside it for anything richer. Typing a message never disturbs other keys already in the payload. ### Rule checks (`check.type: "rule"`) `check.rule` has a `kind` and a `spec`: | Rule `kind` | `spec` keys | Passes when | |---|---|---| | `regex` | `pattern`, `mode` (`allow` \| `deny`, default `allow`) | `allow`: the pattern matches. `deny`: the pattern does **not** match. | | `length` | `min`, `max` (either optional) | The text length is within the bounds. | | `allow` | `values` (list) | The value is one of `values`. | | `deny` | `values` (list) | The value is **not** in `values`. | | `json_schema` | `required` (list of keys) | The value is an object that has all the `required` keys. (A minimal shape check — nothing else about the value is validated.) | | `pii` | `patterns` (optional list) | **No** pattern matches. With no `patterns`, built-in email, US-SSN, and card-number-like patterns are used, so a value containing PII is blocked. | Non-string values are stringified before a string-shaped rule (regex, length, allow/deny, pii) runs. ### Model checks (`check.type: "model"`) `check.model` describes an LLM judge: | Field | Type | Default | Description | |---|---|---|---| | `model` | string (required) | — | A declared model binding (same picker and auto-declare behavior as the [llm node](llm.md)). An engine name here is rejected up front. | | `prompt` | string (required) | — | The judge question. Keep it a terse yes/no, e.g. "Is this request about booking a table? Answer yes or no." | | `passOn` | string | `"yes"` | The answer that means **pass**. Anything else routes to `block`. | The judge answer is compared case- and whitespace-insensitively: the check **passes when the answer starts with `passOn`**. ## Behavior notes - **`pass` carries the input through untouched.** Downstream nodes see exactly what went into the guardrail. - **`block` carries `onBlock`.** If `onBlock` is left empty, `block` instead carries the original input, so you always have something to return. - **Exactly one branch runs.** This is the standard exclusive-branch pattern that keeps at most one `output` node live per run — one output on the `pass` path, one on the `block` path, and only the taken one executes. - **A model check spends tokens.** The judge call is a normal model turn; its token usage counts toward any downstream [quota gate](ratelimit-quota.md) just like any other model call. - **Unknown rule kinds fail closed.** If a rule's `kind` is one the runtime doesn't recognise, the guardrail blocks rather than letting the value through. (The editor validates known kinds before you can save.) ### What a blocked run looks like A block is a normal, successful outcome — not an error. The run stays `completed`; the `block` branch simply activates instead of `pass`. If you wire `block` to its own `output`, the run's output is your refusal payload. In the [run waterfall](../../running/observability.md) the guardrail's `block` handle shows as the live path and the `pass` branch as skipped. If you leave `block` unwired and the check fails, the taken branch reaches no output node — the run completes with an honest empty-output note explaining that no output ran. Wire `block` to something so a blocked caller gets a clear answer. ## Worked example Refuse database questions with a rule check before touching any tool or model. ```mermaid graph LR IN[input] --> G{guardrail: regex deny} G -->|pass| WORK[tool + llm pipeline] --> OK[output: answer] G -->|block| NO[output: refusal] ``` The guardrail's config, using a deny-mode regex so the value **passes only when it does not match**: ```json { "check": { "type": "rule", "rule": { "kind": "regex", "spec": { "mode": "deny", "pattern": "(?i)(postgres|mysql|mongodb|redis|sqlite|database)" } } }, "onBlock": { "message": "I am not allowed to provide that information" } } ``` A prompt mentioning a database matches the pattern, so in `deny` mode the check fails and the `block` output returns the `onBlock` message. Any other prompt passes through to the real pipeline. ## Works well with - [Rate limit and quota nodes](ratelimit-quota.md) — the other pre-work gates: cap request rate and token spend. - [Router node](router.md) — branch on a value that is already a decision, rather than a pass/block check. - [LLM node](llm.md) — a model guardrail reuses the same model bindings. - [Observability](../../running/observability.md) — see which branch a run took. - [Input references](../input-references.md) — how the checked value is addressed. --- Source: https://docs.ertir.dev/latest/building/nodes/ratelimit-quota/ # Rate limit and quota nodes Both of these are **gate** nodes: they sit in front of expensive work and split the run into `allow` and `deny`. They are **compute** (`activity`) nodes. The difference is what they count: | | **ratelimit** | **quota** | |---|---|---| | Counts | **requests** — one hit each time the node runs | **tokens** — model usage already recorded on the run's spans | | Trips when | the hit count in the window passes `limit` | recorded usage reaches `budgetTokens` | | Backed by | a fixed-window counter in Postgres | a read of the agent's usage spans (never re-metered) | | Typical use | "no more than 30 calls a minute per user" | "no more than 100k tokens an hour for this agent" | Both keep the input value on `allow` and emit a clean deny result on `deny` — they never hang or error. ## Shared ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value passing through the gate. | | `allow` | out | — | Carries the input through **unchanged** when the gate lets it pass. | | `deny` | out | — | Carries `{ "denied": true, "reason": "…" }` when the gate trips. | Exactly one of `allow` / `deny` activates per run; the other branch is dead and everything downstream of it is skipped. ## The key: `keyExpr` **Only `ratelimit` counts per key.** Its request counter is bucketed by `keyExpr`, so different callers get separate request budgets. `keyExpr` decides the key: - An [input reference](../input-references.md) such as `$in.in.userId` resolves a per-caller key from the input — each distinct value gets its own bucket. - Any other string is used **as a literal** — a single shared bucket for everyone. (An `$in` reference that can't be resolved falls back to the literal string, so a bad reference never crashes the run.) For unattended runs, the natural key comes from the [trigger](../../running/triggers.md) that fired the agent — resolve it from the input the trigger delivers. **`quota` ignores `keyExpr` today.** The token budget is summed per *agent*, not per key — every caller draws from the same budget no matter what `keyExpr` is set to. The field is still required by the schema, but it has no effect on a quota gate (per-user token attribution is not yet available). ## ratelimit Denies once too many requests hit the same key inside a fixed time window. ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `keyExpr` | string (required) | `""` | The per-caller key (see [above](#the-key-keyexpr)). | | `limit` | integer (required) | `0` | Requests allowed per key per window. Set a real positive value. | | `windowSeconds` | integer (required) | `0` | The window length in seconds. Set a real positive value. | | `policy` | `deny` \| `wait` | `deny` | Leave at `deny`, which emits the `deny` port. `wait` (a bounded suspend) is named in the schema but **not yet implemented** — use `deny`. | ### What counts and when it trips Each time a run reaches the node, it records **one hit** against `::` for the current window and then denies once the count is **greater than** `limit` — so `limit` requests pass and the next one denies. The deny reason reads `rate limit exceeded (/s)`. !!! note "Fixed windows, not sliding" The window is `floor(now / windowSeconds)`; the count resets to zero at each boundary. This is a simple fixed window — a burst straddling a boundary can briefly exceed the nominal rate. It is a lightweight throttle, not a precise sliding-window limiter. ## quota Denies once the agent has already spent its token budget in the window. It **reads** usage that model calls recorded; it does not re-measure anything. ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `keyExpr` | string (required) | `""` | Required by the schema but **inert** for quota — the budget is summed per agent, not per key (see [above](#the-key-keyexpr)). | | `budgetTokens` | integer (required) | `0` | Token budget per window. Set a real positive value. | | `windowSeconds` | integer (required) | `0` | The window length in seconds. Set a real positive value. | | `source` | constant | `"spans"` | Fixed. Usage is read from existing spans; there is no other source. | ### What counts and when it trips The gate sums `gen_ai.usage.total_tokens` across the **agent's** model calls within the last `windowSeconds` and denies once that total is **at or above** `budgetTokens`. The deny reason reads `token budget exceeded (/s)`. - Every model call in the agent contributes — including a [model guardrail's](guardrail.md) judge call. - Scope is **per agent** today: the total is summed across all of the agent's model calls, regardless of `keyExpr` (per-user attribution is not yet available). - If no usage was recorded — for example an upstream that reports none — the sum is `0` and the gate allows. ## Shared behavior notes - **Deny is a clean result, not an error.** A tripped gate keeps the run `completed` and sends `{ "denied": true, "reason": "…" }` out `deny`. The run never hangs and never returns a 500. - **Wire the `deny` port.** If you leave `deny` unwired, the deny branch reaches no output and the run finishes with an honest empty-output note. Route `deny` to its own `output` (a "try again later" message) so a throttled caller gets a clear answer. - **`allow` passes the input straight through** to the protected pipeline. - **Gates are soft policy.** On a durable resume, a re-run step can re-count a rate-limit hit, and quota reads whatever usage has been journaled so far. Treat these as guardrails against runaway cost, not exact financial controls. ## Worked example Throttle a public agent to 30 requests per user per minute (keyed per caller), then cap the whole agent at 50k tokens per hour, before the model runs. ```mermaid graph LR IN[input] --> RL{ratelimit} RL -->|allow| Q{quota} RL -->|deny| BUSY[output: slow down] Q -->|allow| LLM[llm] --> OUT[output: answer] Q -->|deny| CAP[output: budget reached] ``` The two gate configs: ```json { "keyExpr": "$in.in.userId", "limit": 30, "windowSeconds": 60, "policy": "deny" } ``` ```json { "keyExpr": "$in.in.userId", "budgetTokens": 50000, "windowSeconds": 3600, "source": "spans" } ``` Each `deny` branch ends at its own `output`, so a throttled or over-budget caller gets a specific message and the model is never reached. ## Works well with - [Guardrail node](guardrail.md) — the content check that pairs with these spend checks. - [Triggers](../../running/triggers.md) — where the per-caller key comes from on unattended runs. - [Observability](../../running/observability.md) — the token usage quota reads is recorded here. - [Input references](../input-references.md) — the `$in` grammar `keyExpr` uses. --- Source: https://docs.ertir.dev/latest/building/nodes/human/ # Human node The `human` node pauses a run and waits for a person to supply input before the graph continues. Use it for approvals, manual review, or any step where an agent should hand control to you and resume once you respond. It is a **boundary** node, and it is **durable-only**: it runs only on the durable runtime, because a run that waits for a human may sit paused for minutes, hours, or days — and must survive a restart in the meantime. See [Durable runs](../../running/durable.md) for why, and [Human in the loop](../../running/human-in-the-loop.md) for the end-to-end approval workflow. ## Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The value that reaches the gate. It is advisory context — captured in the trace and available to describe what needs review — not what flows out. | | `out` | out | — | Carries the value **the person delivered on resume**, once input arrives. | The node's incoming value is *not* passed straight through. When you resume the run, the value you deliver is what flows out of `out` into the downstream nodes. ## Configuration | Field | Type | Default | Description | |---|---|---|---| | `prompt` | string | `null` | Advisory text describing the input you expect (shown as context; does not change execution). | | `inputSchema` | JSON object | `null` | Advisory schema for the expected input. Only its `required` keys are enforced on resume. | | `timeout` | number (seconds) | `null` | How long to wait. `null` waits indefinitely. | | `onTimeout` | enum `fail` \| `default` | `fail` | On timeout: `fail` fails the run; `default` binds `default` and continues. | | `default` | any | `null` | The value used when `onTimeout` is `default` and the timeout elapses. | The inspector edits `inputSchema` as an **awaited fields** list — a row per field with a name and a **req** checkbox — rather than hand-written JSON Schema, since in practice this node declares which keys the resuming person must send. A field left untyped contributes its requiredness only, which is exactly the `{"required": ["decision"]}` shape a decision gate wants. ## Behavior - **The run pauses as `waiting`.** When the walker reaches a `human` node, the run's status becomes `waiting` and its `awaiting_node` field records the node id. A waiting run is excluded from the crash-recovery sweep, so it can stay paused across restarts. - **You resume by delivering input.** Once input arrives, the run flips back to `streaming`, the delivered value flows out of `out`, and execution continues. A graph may contain more than one `human` node — the run pauses at each in turn. - **The resume input is validated against `inputSchema.required`.** If your input is an object missing a required key, the resume is rejected (`resume_schema_mismatch`) and the run stays paused. - **A duplicate resume is harmless.** Delivery is keyed per node, so a double-click or client retry cannot accidentally satisfy the *next* human gate — the extra message is discarded. - **On timeout.** With `onTimeout: "fail"` (the default) the run fails with a message naming the node and the elapsed time. With `onTimeout: "default"` the value in `default` is bound and the run continues. If you set `onTimeout: "default"` but leave `default` empty, the editor warns you, and a timeout flows `null`. ## Resuming a paused run There are two ways to deliver input. **From the interface.** A waiting run shows a resume panel — on the run detail page (under **Runs**) and in the agent's bench modal. The panel reads *"Paused — awaiting input at ``"* with a **Text**/**JSON** mode toggle, an input box, and a **Resume** button. JSON mode is parsed before it is sent, so a malformed object is caught in the tab rather than at run time; an empty JSON box means "no input" (`null`). **Over the API.** Post the input to the run: ```bash curl -X POST http://localhost:8080/runs//resume \ -H 'Content-Type: application/json' \ -d '{"input": {"approve": true}}' ``` The `input` field is the value delivered to the gate. A successful resume returns **202** with `{runId, status: "resuming", awaitingNode}`. This surface uses the ordinary interactive authentication, not the token-invoke bearer. The common error responses: | Status / code | Meaning | |---|---| | 400 `durable_required` | The control plane is not running in durable mode. | | 404 `run_not_found` | No run with that id. | | 409 `run_not_waiting` | The run is not currently paused at a human gate. | | 409 `awaiting_node_missing` | The run is waiting but records no target node to resume. | | 422 `resume_schema_mismatch` | The input object is missing a key required by `inputSchema`. | | 410 `workflow_gone` | The durable workflow no longer exists; the run is marked `failed`. | ## Running a graph that contains a human node Because the node is durable-only, the plain interactive **Run** path refuses it up front with `durable_required`. Run it durably instead: - **From the interface:** publish the agent, then on the Agents page use **Run durably**. An agent that contains any durable-only node shows a single **Run durably** button. This needs the control plane started with `THEYGENT_DURABLE=1`; otherwise you get a note explaining how to enable durable mode. - **Over the API:** `POST /agents/{id}/durable-runs`, then poll `GET /runs/{id}` and resume when it reports `waiting`. ## Example: approve a draft before it ships An agent drafts a message, pauses for you to approve or edit it, then returns whatever you delivered. ```mermaid graph LR input --> llm[llm draft] llm --> human[human review] human --> output ``` The `human` node's configuration: ```json { "type": "human", "kind": "boundary", "config": { "prompt": "Review the draft. Reply with the approved text.", "inputSchema": { "required": ["text"] }, "timeout": 3600, "onTimeout": "fail" } } ``` When this run reaches the gate it pauses as `waiting`. You resume with, for example, `{"text": "Approved copy goes here."}` — the `inputSchema` requires a `text` key, so an object without it is rejected — and that value flows out `out` to the `output` node. ## Works well with - [Human in the loop](../../running/human-in-the-loop.md) — the full waiting/resume/approval workflow. - [Durable runs](../../running/durable.md) — why this node needs durable mode, and how to enable it. - [Guardrail node](guardrail.md) — gate *automatically* before deciding whether a human even needs to see the request. - [Orchestration nodes](orchestration.md) — the other durable-only nodes (`subgraph`, `loop`, `map`). --- Source: https://docs.ertir.dev/latest/building/nodes/orchestration/ # Orchestration nodes: subgraph, loop, map These three nodes compose *other published agents* into a larger one: `subgraph` runs another agent once, `loop` runs one repeatedly, and `map` runs one across every item in a list. They are how you build agents out of smaller, reusable agents instead of one giant graph. All three are **durable-only** — they run only on the durable runtime. The reason is practical: each of them drives one or more child runs, which can take real time, and the durable runtime journals every completed step so a crash resumes from where it left off instead of restarting the whole thing. A bounded loop halfway through its iterations, or a fan-out with some branches still running, must survive a restart without redoing finished work. See [Durable runs](../../running/durable.md) for the full picture, and [Agents and graphs](../../concepts/agents-and-graphs.md) for what "a published agent" means. !!! note "How you pin the child agent" Every one of these nodes runs a **published agent version**, pinned so composition is immutable. In the inspector you pick the child **agent** and then a **body version** (newest first). Pinning by content hash instead of version is available through the node's **Code** view. You must pin exactly one of `version` or `contentHash` — the editor flags it as an error otherwise. Publishing a new version of the child later does *not* change what a pinned node runs. --- ## subgraph — run another agent once Runs a published agent as a single step and returns its output. A **boundary** node. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The input passed to the child agent. | | `out` | out | — | The child agent's output. | | `err` | out | — | Carries the failure if the child run fails (`error`-typed handle). | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `agent` | string | — (required) | The published agent id to run. | | `version` | string | `null` | The child version to pin. Pin exactly one of `version` / `contentHash`. | | `contentHash` | string | `null` | Pin the child by content hash instead (set via the Code view). | | `maxDepth` | integer | `8` | Maximum nesting depth. A subgraph that nests deeper than this fails honestly rather than recursing without bound. | ### Behavior - The child agent runs to completion; its output flows out `out`. - If the child run fails, the failure is bound to `err` — wire `err` to an `output` (or a recovery branch) to handle it instead of failing the parent. - The pin is folded into the parent's own content — composing a child version is itself immutable. Exceeding `maxDepth` (for example, a subgraph that transitively includes itself) is a clean failure, not a hang. --- ## loop — run an agent repeatedly Runs a published agent in sequence, feeding each iteration's output into the next, up to a bounded number of iterations. An **orchestration** node. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The input to the first iteration. | | `out` | out | — | The final iteration's output. | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `agent` | string | — (required) | The published agent to run each iteration. | | `maxIterations` | integer | — (required, ≥ 1) | The hard cap on iterations. There are no unbounded loops; the default of `0` is invalid and must be raised to at least 1. | | `condition` | string | `null` | An optional `$in` reference over the iteration's output; the loop stops early when it resolves truthy. | | `version` / `contentHash` | string | `null` | Pin exactly one. | | `maxDepth` | integer | `8` | Maximum nesting depth. | ### Behavior - Each iteration's output becomes the next iteration's input. - The loop stops at whichever comes first: `maxIterations` reached, or `condition` resolving truthy on an iteration's output. The last output flows out `out`. - Completed iterations are journaled — on a crash-resume they are never re-run. `condition` uses the same [input-reference grammar](../input-references.md) as everywhere else. For example, `$in.done` stops the loop as soon as an iteration returns an object with a truthy `done` field. --- ## map — fan out across a list Runs a published agent once per element of a list — the fan-out primitive. An **orchestration** node. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | Must be a **list**. One child run is started per element. | | `out` | out | — | The collected results (one per element). | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `agent` | string | — (required) | The published agent to run for each element. | | `concurrency` | integer | `null` | Maximum child runs in flight at once. `null` means unbounded. | | `onError` | enum `fail_fast` \| `collect` | `fail_fast` | `fail_fast` stops on the first failing element; `collect` gathers successes alongside per-element errors. | | `version` / `contentHash` | string | `null` | Pin exactly one. | | `maxDepth` | integer | `8` | Maximum nesting depth. | ### Behavior - The input must be a list; each element becomes the input to one child run. - `concurrency` bounds how many run at the same time; leave it `null` to let them all run at once. - With `onError: "fail_fast"` the map fails as soon as any element fails. With `onError: "collect"` it finishes and returns every success plus a per-element error for the ones that failed. - On a crash, only the incomplete branches resume — finished elements are not re-run. --- ## Example: summarize each document in a batch A parent agent takes a list of documents and maps a saved `summarize-one` agent over them. ```mermaid graph LR input --> map[map · summarize-one] map --> output ``` The `map` node's configuration: ```json { "type": "map", "kind": "orchestration", "config": { "agent": "summarize-one", "version": "1.2.0", "concurrency": 4, "onError": "collect" } } ``` Give this agent a list as input (a multi-input agent drills in with `$in.in.` — see [input references](../input-references.md)); it runs up to four summaries at a time and collects the results, recording per-element errors instead of failing the whole batch. ## Running an agent that uses these nodes Because all three are durable-only, run them the durable way: - **From the interface:** publish the agent, then use **Run durably** on the Agents page (a single **Run durably** button appears for any agent containing a durable-only node). This needs the control plane started with `THEYGENT_DURABLE=1`. - **Unattended:** deploy the agent behind a [schedule or webhook trigger](../../running/triggers.md) — in durable mode, triggered runs are durable automatically. ## Works well with - [Drafts & publishing](../saving-agents.md) — the child agents these nodes compose must be published and versioned first. - [Versioning](../../concepts/versioning.md) — why pinning a child by version or content hash keeps composition immutable. - [Durable runs](../../running/durable.md) — the runtime these nodes require, and how to enable it. - [Human node](human.md) — the fourth durable-only node, for pausing on human input. --- Source: https://docs.ertir.dev/latest/building/nodes/media/ # Media nodes: transcribe, speak, imagine These three **activity** nodes call multimodal models: `transcribe` turns speech into text, `speak` turns text into speech, and `imagine` generates an image from a text prompt. They are the building blocks of voice agents and image-generating agents. They pass audio and image data **by reference**, never as inline blobs. A `transcribe` node reads an audio *reference*; `speak` and `imagine` produce an audio or image *reference*. The bytes themselves are stored as local artifact files — they are never written into run history or session turns, and a resumed durable run replays the reference rather than the media. ## Artifacts, in brief An **artifact** is a blob of media (audio or image) stored on your machine and addressed by an id like `art_...`. Generated bytes live on the local filesystem under `THEYGENT_ARTIFACT_DIR` (default: a `theygent-artifacts` folder in the system temp directory). To turn a reference back into playable/viewable media, the chat client downloads it from `GET /artifacts/{ref}`, which serves only stored `art_...` ids — it cannot read arbitrary files. See [Voice input/output](../../chat/voice.md) and [Image input and generation](../../chat/vision-and-images.md) for how these nodes show up in chat. ## Models these nodes need Each media node references a model by its **logical id** — the same way an [llm node](llm.md) does — and each needs a model registered with a matching **modality**: | Node | Required modality | Local engine that serves it | |---|---|---| | `transcribe` | `audio.transcription` | whisper.cpp (`whisper-server`) | | `speak` | `audio.speech` | the MLX audio server (`mlx_audio.server`) on Apple Silicon | | `imagine` | `images.generation` | stable-diffusion.cpp (`sd-cli`) via the llama.cpp binding, or FLUX (`mflux`) on Apple Silicon via the MLX binding | If a node's model binding names an engine instead of a logical id, the run is rejected up front (`engine_name_not_allowed`). See [Installing models](../../models/installing.md) and [the engines](../../models/engines.md) for how to register each kind. --- ## transcribe — speech to text Reads audio and returns its transcript. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `audio` | in | yes | An audio reference: a stored artifact id, an `http(s)` URL, or a local file path. | | `text` | out | — | The transcript text. | | `err` | out | — | An `error`-typed handle carrying a fetch or transport failure. | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `model` | string | — (required) | A logical model id with the `audio.transcription` modality. | | `params` | JSON object | `{}` | Passed through to transcription. Common keys: `language`, `prompt`, `temperature`, `response_format` (`json` \| `text` \| `verbose_json`). | ### Behavior The node fetches the audio bytes from the reference and streams them to the inference plane's transcription endpoint — the transcription *model call* goes straight to your inference plane, not through a control-plane inference route. (The audio itself is stored as a control-plane [artifact](../../chat/voice.md); it is the model call, not the storage, that bypasses the control plane.) On a fetch or transport failure the error binds to `err` and the run continues; wire `err` to handle it. The inspector gives this node a dedicated parameter panel with per-field help. --- ## speak — text to speech Turns text into an audio artifact. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `text` | in | yes | The text to synthesize. | | `audio` | out | — | A reference to the produced audio artifact. | | `err` | out | — | An `error`-typed handle carrying a synthesis failure. | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `model` | string | — (required) | A logical model id with the `audio.speech` modality. | | `params` | JSON object | `{}` | Voice controls: `voice` (omitted → the engine's own default), `speed`, `format` (`mp3` \| `wav` \| `opus` \| `flac` \| `aac` \| `pcm`). | ### Behavior The node calls the inference speech endpoint, stores the returned audio as a new artifact, and emits the reference on `audio`. On the wire, the node's `format` key becomes the request's `response_format`, and it sets the artifact's MIME type. When `params` omits it, `format` defaults to `mp3`. !!! tip "Only set a voice your engine actually has" **`voice` has no default** — when you leave it out, no voice is sent and the engine synthesizes with its own. That is deliberate: voice vocabularies are engine-specific, so a name from one engine is not a voice on another. A hosted-API name like `alloy`, for instance, does not exist on a local kokoro build. Getting this wrong is worth recognising, because the symptom does not look like a bad parameter: a speech server asked for a voice it does not have accepts the request, answers `200`, and then aborts the response body with no message. The run reports *the speech engine aborted mid-synthesis* and names the voice it asked for. If **every** input fails, it is the voice (or an unloadable model), not the text. The trade-off runs the other way for a **hosted** TTS API, where `voice` is a required request field: leave it out and the provider rejects the call with its own error, relayed verbatim. Name a voice that provider offers when you point a `speak` node at one. Ertir does not guess on your behalf in either direction — a guess is what breaks the local case silently. --- ## imagine — text to image Generates an image from a text prompt. ### Ports | Port | Direction | Required | Description | |---|---|---|---| | `in` | in | yes | The text prompt describing the image. | | `out` | out | — | A reference to the produced image artifact. | ### Configuration | Field | Type | Default | Description | |---|---|---|---| | `model` | string | — (required) | A logical model id with the `images.generation` modality. | | `params` | JSON object | `{}` | Generation controls: `size` (`WxH`, snapped to multiples of 64, default `512x512`), `n` (clamped 1–4), `steps`. | ### Behavior The node sends the prompt to the inference image endpoint, stores the produced bytes as an image artifact, and emits the reference on `out`. Image generation loads weights per request and is serialized on the inference plane (one render at a time), so it can be slow — the run stream sends keepalives during the wait so the connection is not mistaken for a dead one. Unlike `transcribe` and `speak`, `imagine` has **no `err` port**. If generation fails, the node emits nothing, and the run surfaces an honest empty-output reason naming the cause (the run stays `completed` with a note rather than a hard failure). In the inspector this node uses the generic form — a plain `model` field plus a raw JSON `params` box. --- ## Example: a voice agent An audio-in / audio-out agent transcribes your speech, answers with a language model, and speaks the answer back. This is exactly the shape [chat's voice agents](../../chat/voice.md) run. ```mermaid graph LR input["input (audio)"] --> transcribe transcribe -->|text| llm llm --> speak speak -->|audio| output["output (audio)"] ``` - The `input` boundary node declares `modality: "audio"`, so chat gives it a microphone composer; the recorded clip is uploaded as an artifact and the run input becomes that reference. - `transcribe` reads the reference and produces text on `text`. - The `llm` node answers (see [the llm node](llm.md)). - `speak` synthesizes the answer and emits an audio reference. - The `output` boundary node declares `modality: "audio"`, so the run's output is the audio reference, which chat downloads into a playable bubble. An image-generating agent follows the same idea with `imagine` feeding an `output` node whose `modality` is `image`. ## Works well with - [Input and output nodes](input-output.md) — the boundary `modality` that turns an agent into a voice or image agent. - [Voice input/output](../../chat/voice.md) and [Image input and generation](../../chat/vision-and-images.md) — how these nodes appear in chat. - [Installing models](../../models/installing.md) and [The engines](../../models/engines.md) — registering the transcription, speech, and image models. - [The Bench](../../running/bench.md) — test a speech, transcription, or image model on its own before wiring it in. --- Source: https://docs.ertir.dev/latest/building/saving-agents/ # Drafts & publishing The editor keeps your work in two tiers, with two different verbs: - **Drafts save themselves.** As you edit, the graph autosaves to a **draft** on the control plane — a mutable work-in-progress that survives a reload, a crash, or a week away. A draft may be half-wired or outright invalid; that is the point. - **Publishing is deliberate.** **Publish** turns the graph into a named, **immutable agent version** that you can run by reference, invoke over HTTP, or deploy behind a trigger. Publishing is content-addressed: the server hashes the document, so two identical graphs are the same version and any real change is a new one. This page covers both tiers: how drafts behave, the id/name/version fields, the publish flow, and where published agents go. ## Drafts: the automatic tier You never click anything to save a draft. A couple of seconds after your first real edit, the editor creates one and keeps it updated as you work; the toolbar shows where you stand — *Saving draft…*, then *Draft saved just now*. Press ++cmd+s++ / ++ctrl+s++ to flush a save immediately. If a save ever fails, the toolbar shows **Draft save failed — retry**; click it to try again. A few properties worth knowing: - **Drafts are allowed to be broken.** Validation gates *publishing*, never the draft — a graph with unwired ports or half-configured nodes still autosaves, so you can stop mid-thought and come back. - **Layout is saved too.** Unlike a published version's content hash, a draft keeps your node positions, so reopening it looks exactly like you left it. - **The URL follows the draft.** Once a draft exists, the editor's URL carries `?draft=` — reloading the tab reopens that exact session. - **One draft per editing session.** Editing a published agent creates a draft *linked to that agent*; a brand-new graph creates a standalone one. Publishing removes the draft — the registry owns the content from then on. ### Where drafts live The **Agents** page (`/agents`) shows a collapsible **Drafts** strip above the published grid, most recently edited first, with the draft's name, node count, and when it was last saved. **Open** resumes the editing session; **Discard** deletes the draft after a confirmation (published versions are never affected). A published agent that has draft edits also wears a small amber **draft** badge on its card. If you open a published agent that has a lingering draft (from an earlier session, or another tab), the editor shows a banner — *A draft of this agent has unpublished changes* — with **Open draft** and **Discard it**, so you never unknowingly edit beside your own unfinished work. !!! note "Drafts are not versions" A draft has no content hash, no version history, and no run-by-reference identity. Nothing can pin, invoke, or compose a draft — only published versions have those properties. Think of the draft as your workbench and the registry as the shelf. ## The id, name, and version fields The editor toolbar carries three text fields that identify the agent: | Field | What it is | |---|---| | **id** | The agent's stable identifier (for example `agent.triage`). It is the registry coordinate together with the version. Once the agent exists in the registry this field is locked — an id is never renamed (only [deleting the whole agent](#exporting-and-deleting-agents) frees it). | | **name** | A human-readable display name. You can change it on any new version. | | **version** | The version string you are publishing, for example `0.1.0`. You bump this yourself. | A brand-new blank graph starts as `agent.untitled`, *Untitled agent*, version `0.1.0`. Set a real id and name before your first publish, because the id can't change afterward. ## Publishing 1. **Make the graph valid.** Publish is disabled while any [validation errors](editor.md#validation) exist; the button reads *Fix N validation error(s) before publishing.* Warnings do not block publishing. 2. **Set the id, name, and version** in the toolbar. (For an existing agent the id is already locked; just adjust the version.) 3. **Click Publish** and confirm. The confirmation spells out what you are about to do: create an immutable version of this id at this version, visible to everyone who can reach this control plane. The server strips the canvas layout, computes the content hash over the rest, stores the version, and returns that hash — which the toolbar then displays. **Your browser never computes the hash;** the server is the single source of truth for it. After a successful publish the URL re-points to the version you just published, the working draft is removed, and the status badge flips to **published**. The status area tracks where you stand: **not published** for a graph that has never been published, the draft-save state while you are editing, and **published** (green) when the canvas matches the registry. **Revert** discards changes back to the last published snapshot and is enabled only while the content has diverged from it. !!! note "Layout is not content" Dragging nodes, zooming, collapsing a panel, or changing a node's icon all live in the layout block, which is *not* hashed. None of them mark the graph modified or create a new version — only a real change (a config value, a message, an edge) does. (The draft still records layout, so your arrangement is never lost — it just never mints a version.) ## Versions are immutable A published version is frozen. You never edit a version in place; you publish a new one. Because the identity is a content hash, the registry enforces a few rules on publish: - **A new id** creates the agent and its first version. - **A new version string on an existing agent** publishes a new version. - **Re-publishing the exact same content under the same (id, version)** is accepted as a no-op — publishing twice does nothing new. - **Different content under a version string that already exists** is rejected. You will see a *version conflict* toast; bump the **version** field and publish again. The editor smooths over one detail: if you build what looks like a new graph but its id already exists in the registry, the publish is automatically redirected to "add a version" to that agent instead of failing. ```mermaid graph TD Pub["Publish"] --> New{"id exists?"} New -->|no| Create["Create agent + first version"] New -->|yes| Ver{"same content
under this version?"} Ver -->|identical| NoOp["Accepted, no change"] Ver -->|different| Conflict["Version conflict — bump the version"] Ver -->|new version string| AddVer["Publish new version"] ``` For the deeper story on why the hash makes composition and history reliable, see [agent versioning](../concepts/versioning.md). ## Where published agents appear A published agent shows up in several places: - **The Agents page** (`/agents`) lists every published agent, newest first, with its latest version and version count — with your drafts in the strip above them. This is the home for running and re-opening agents. - **Re-open for editing** by clicking the agent's card — the editor loads its latest version. You can also open a specific version directly. - **New Chat** lets you pick a published agent as a conversation target, with a version picker. See [chat](../chat/index.md). - **Subgraph, loop, and map** nodes compose *other* published agents by id and pinned version — so once published, an agent can become a building block inside another. See [subgraph, loop and map](nodes/orchestration.md). ## Testing before you publish You don't need to publish anything just to try a graph: the editor's **Test** panel runs the document exactly as it sits on the canvas — draft, unpublished, whatever — streaming the output and lighting each node as it executes. See [the editor's testing section](editor.md#testing-on-the-canvas). ## Running a published agent Beyond the in-editor test panel, published agents run from the **Agents** page: each card has a **Run** button that opens a bench modal: - **Run** streams the agent interactively — you type an input and watch the answer stream back. - The caret next to it offers **Run durably**, which executes the agent on the durable runtime (checkpointed, crash-resumable). This requires the server's durable mode. - An agent that contains a durable-only node (`human`, `subgraph`, `loop`, or `map`) shows a single **Run durably** button, because those nodes can't run on the interactive path. See [running agents](../running/index.md) for run statuses and output rules, and [durable runs](../running/durable.md) for what durable mode buys you. ### Running from the API Published agents are also reachable over HTTP on the control plane (default `http://localhost:8080`). The interactive endpoint takes the agent id and an input: ```bash curl http://localhost:8080/agents/agent.triage/runs \ -H "Authorization: Bearer tyk_your-api-key" \ -H "Content-Type: application/json" \ -d '{"input": "cancel my subscription", "stream": false}' ``` You can pin a specific version with `"version"` or a specific hash with `"content_hash"`; otherwise the latest published version runs. There is also a token-authed, non-interactive `POST /agents/{id}/invoke` for unattended callers, and a fire-and-poll `POST /agents/{id}/durable-runs` for durable execution. Drafts have their own small CRUD surface (`/drafts`) that the editor drives. The full surface is in the [API reference](../reference/api.md). You don't have to assemble any of this by hand: every agent on the Agents page carries an **API access** action (the `` icon in the card footer, or the **API** button in list view) that opens a per-agent dialog with these endpoints pre-filled — the exact URLs for *this* agent, which credential opens each one, a copy-able curl for each (the example body is derived from the agent's input fields), and any [triggers](../running/triggers.md) already deployed for it. An agent built around a durable-only node shows the durable-run recipe instead of the interactive one. Authentication is the platform's own and is the same for every agent: interactive runs accept a signed-in session or a personal [API key](../running/users.md#personal-api-keys), while the unattended invoke endpoint takes an API key or the deploy-wide `THEYGENT_INVOKE_TOKEN` (sessions deliberately don't open it). The only narrower credential is a webhook trigger's per-trigger signing secret. ## Exporting and deleting agents Beyond Run and API access, every agent on the Agents page carries two more actions — a download icon and a trash icon in the card footer (grid view), or **Export** / **Delete** buttons in the Actions column (list view). ### Export an agent as JSON **Export** downloads the agent's **latest version** as a single JSON file, named like `theygent-agent-agent.triage-v0.2.0.json` — the full graph with its canvas layout embedded. The file is self-contained: hand it to someone else (or your other machine) and import it through **Settings → Import / Export**, where it lands as a published agent again. The content hash in the file is informational only — the receiving server validates the document and recomputes the hash itself, exactly like a publish. ### Delete an agent **Delete** asks for a typed confirmation: the dialog names the agent, and the **Delete** button stays disabled until you type the agent's name exactly. Confirming removes the agent and **all** its versions and triggers in one step (a deployed schedule stops firing). Past runs and chats are kept — they still show the agent id and content hash they ran against — and any draft that was editing the agent survives in the Drafts strip. Deletion cannot be undone; the only way back is re-importing or re-publishing the agent. ### Select mode: bulk export and delete The **Select** button in the page header switches the page into selection mode: each card (or row) gains a checkbox, with **Select all** over the currently filtered set. The bulk bar then offers: - **Export selected** — one zip containing the selected agents with **every** version of each (not just the latest), importable through **Settings → Import / Export** like any bundle. - **Delete selected** — one typed confirmation (type the number of selected agents to arm the button), then the agents are deleted one at a time with a running progress count; a failure on one never stops the rest. !!! note "Versions are immutable — the agent is deletable" While an agent exists, its history is append-only: you can never edit or remove a *single* published version, so anything pinned to one keeps behaving identically. What you can do is delete the **whole agent**, versions and all. Runs are never deletable; sessions, triggers, connections, MCP servers, and **drafts** can be deleted individually as before. To retire one version, publish a new one instead. ## Related pages - [The editor](editor.md) — building, validating, and testing before you publish - [Agent versioning](../concepts/versioning.md) — content hashing and immutability, in depth - [Import & export](../import-export/index.md) — moving whole installs (or agent bundles) between machines - [Running agents](../running/index.md) — statuses, output, and the run list - [Triggers and webhooks](../running/triggers.md) — deploying a published agent behind a schedule or webhook --- Source: https://docs.ertir.dev/latest/chat/ # Chat **New Chat** is the one conversation surface for everything you have registered: talk to any model in your inference plane or to any published agent, with streaming answers, a separate display for a reasoning model's thinking, and automatic session recording. If you have not installed a model yet, start with [your first chat](../getting-started/first-chat.md). ## Starting a conversation 1. Open **New Chat** in the sidebar (the `/chat` page). 2. Pick the target kind with the **Model / Agent** toggle. 3. Pick the target in the searchable select ("Select a model…" or "Select an agent…"). For an agent there is also a **Version** picker — it defaults to the latest version, labeled `v · latest · `. 4. Type your message and press ++enter++. ++shift+enter++ inserts a newline, and pressing Enter mid-composition with an input method editor never sends. If the pickers are empty you will see "No models registered yet — install one under Registries first" or "No published agents yet — build one on the canvas first" — see [installing models](../models/installing.md) and [building your first agent](../getting-started/first-agent.md). Once the conversation starts, the model — or the agent and its version — is **pinned**: the picker locks so every turn goes to the same target. The **New chat** button starts a fresh conversation (and a fresh session) when you want to switch. ## Model or agent? | Target | What each turn is | Notes | |---|---|---| | **Model** | A direct conversation with one model, addressed by its logical id | Text chats run through the control plane so history is replayed server-side; requests default to `max_tokens: 2048` | | **Agent** | A run of the published agent's graph, with your message as the input | The agent's whole pipeline executes — tools, routing, guardrails and all | Badges next to the picker tell you what you selected: - A model with a non-chat modality shows a violet modality badge (for example `audio.transcription` or `images.generation`). - An agent whose graph takes or produces media gets a **voice**/**audio**, **vision**, or **image** badge, read from its input and output nodes. - Embeddings models refuse to chat: "Embeddings models don't chat — bench them from Registries." Test them on [the Bench](../running/bench.md) instead. ## Where your messages go ```mermaid graph LR U[New Chat] -->|text chat with a model or agent| CP[Control plane :8080 - runs and sessions] U -->|voice, vision, image generation with a model| IP[Inference plane :8081 - /v1/*] CP -->|logical model id| IP ``` - **Text conversations** (a chat model, or any agent) are streamed **control-plane runs**: each turn creates a run, the server replays the session history, and the reply is appended to the session. - **Media conversations with a model** — a transcription, speech, vision, or image-generation model — go **directly to your inference plane's data endpoints**. Raw audio and images never transit the control plane; only a readable text record of the turn is written to the session afterwards. See [voice](voice.md) and [vision and images](vision-and-images.md). Everything stays on the machines *you* configured: the inference plane is yours, and hosted APIs are only involved if you [registered one](../models/remote.md). ## Streaming Answers stream token by token with a pulsing cursor. The transcript follows the live edge; scroll up and it stops following, with a jump-to-latest button to catch up again. While a reply streams, the **Send** button becomes **Stop generating**. Stopping keeps the partial answer, marked as stopped — it is not treated as a failure. Navigating away mid-stream also aborts the request, and the server cancels the underlying run so nothing keeps generating in the background. ## Thinking (reasoning models) A reasoning model "thinks" before it answers. Ertir keeps that thinking apart from the answer everywhere: - The thinking renders in a collapsible **"Thinking…"** block above the answer. It is open and follows its live edge while streaming (scroll up to unpin), auto-collapses once the answer starts, and can be reopened at any time. - Both wire shapes are handled — a separate reasoning stream and inline `…` tags — so it works the same across engines. - Thinking never becomes part of the run output or the session transcript. It is captured alongside the run's trace (subject to the capture policy) and is inspectable per node in the [run waterfall](../running/observability.md). New Chat itself has no parameter form. To turn thinking on or off for a model that supports it, use the **Reasoning** parameter (`chat_template_kwargs` → `{"enable_thinking": true|false}`) where parameters are editable: on [the Bench](../running/bench.md) per exchange, or on an agent's [llm node](../building/nodes/llm.md) model binding. !!! note "An answer that never arrives" If a turn finishes with no visible content, the chat shows a note instead of an empty bubble: a reasoning model may have spent the whole token budget thinking. The run still counts as `completed`, with the note shown in amber. Raise the max-tokens budget (on the llm node for agents, or bench the model to find a workable value). ## Per-message details Your turns sit on the right; the assistant's on the left, rendered as markdown, with image thumbnails and playable audio where the turn carried media. - Each control-plane turn header carries a **"run \"** link straight to the [run detail page](../running/index.md). - Under every run-backed answer a collapsed **Trace** disclosure embeds the run waterfall — timings per node, token usage, gaps. It loads nothing until you open it. See [observability](../running/observability.md). - Per-turn problems show as amber notes on the turn rather than breaking the conversation. ## Sessions and Recents Every conversation is recorded as a **session** — the stored unit of chat history, with ids like `ses_1f0c…`. All chat surfaces record into sessions: New Chat, a model bench conversation, and an agent bench chat alike, so everything you said lands in one place. Recording is best-effort: if writing a turn fails, the chat itself is never interrupted. - **Recents** (bottom of the sidebar) lists your latest sessions, labeled by title, preview text, or the agent/model name. It is hidden while the sidebar is collapsed to icons. - **Chats** (`/sessions`) is the full list: columns **Session / Target / Messages / First message / Last activity**, with search over id, preview, and target, and infinite scroll. - Opening a session shows the stored transcript and lets you **continue the conversation with the same target**: text targets continue through control-plane runs, a voice session continues through the direct transport. A session recorded before targets were stored is readable but not continuable. - **Delete** removes the session but keeps its runs — they only lose the session link. The confirm dialog says exactly that. Once a session exists, a **"session \"** link at the top right of New Chat jumps to its detail page. For how sessions relate to runs and memory, see [runs and sessions](../concepts/runs-and-sessions.md). ## Composer reference | Control | Appears for | What it does | |---|---|---| | **Send** / **Stop generating** | always | Sends the message; aborts the in-flight reply while streaming | | **Attach image** / **Capture from camera** | vision targets | Stages an image chip above the input — see [vision and images](vision-and-images.md) | | **Record from microphone** / **Attach audio file** | transcription targets and voice agents | Stages an audio chip (with an inline player); for pure transcription targets the text input is hidden and audio is required — see [voice](voice.md) | | **Attach file** | agents whose input boundary is `video` or `file` | Stages the file and sends it as an artifact reference | | JSON editor | agents whose input boundary is `json` | Replaces the prose box; the payload must parse before Send is enabled | Staged attachments are removable chips above the input; nothing is sent until you press Send. What the composer offers for an **agent** is read from its input node's `modality`, and a target that needs an attachment keeps **Send** disabled until you stage one — so a vision agent can't be sent a bare sentence its graph has no image for. The same rule drives the canvas [Test panel](../building/editor.md#testing-on-the-canvas) and the Agents page's [Run dialog](../running/bench.md#benching-an-agent); see [what the modality changes](../building/nodes/input-output.md#what-the-modality-changes). Reopening a stored conversation re-reads the boundary from the version it was recorded against, so a voice session comes back with its microphone. ## Related pages - [Voice in chat](voice.md) — microphone input, speech output, voice agents - [Vision and images](vision-and-images.md) — chatting about images and generating them - [Runs and sessions](../concepts/runs-and-sessions.md) — the data model behind the transcript - [The Bench](../running/bench.md) — test a model with full parameter control before you chat with it --- Source: https://docs.ertir.dev/latest/chat/voice/ # Voice Voice shows up in [chat](index.md) two ways: you can talk to a voice **model** directly — record speech and get a transcript, or type text and get spoken audio back — or you can chat with a **voice agent** that listens, thinks, and answers out loud. This page covers both, and the speech-to-text and text-to-speech engines you need to install for them. Raw audio always goes **directly to your inference plane**, never through the control plane. Only a short readable record of the turn ("🎤 voice message", "🔊 audio") is written to the session afterwards.

Voice In, Voice Out (5:54) — building the transcribe → llm → speak pipeline described below, then talking to it.
## Recording and attaching audio The composer grows two audio controls whenever the target speaks or listens: - **Record from microphone** — press it to start recording (the icon becomes a stop square); press again to stop. Recording uses your browser's media recorder, preferring `audio/webm;codecs=opus` (falling back to `audio/webm`, then `audio/mp4`). - **Attach audio file** — pick an existing clip from disk instead. Either way the clip stages as a removable chip above the input, with an inline player and its duration. Nothing is sent until you press **Send**. For a **pure transcription target** (a speech-to-text model, or a voice agent), the text box is hidden entirely and replaced with the prompt "Record or attach audio, then send." — audio is required before Send activates. ## Talking to a voice model directly Pick **Model** in New Chat and select a model whose modality is `audio.transcription` or `audio.speech`. A violet modality badge appears next to the picker, and the turn goes straight to the inference data plane. - **Speech-to-text** (`audio.transcription`) — record or attach a clip and send. The transcript comes back as the assistant's reply. Under the hood this is a multipart `POST /v1/audio/transcriptions` with your logical model id. - **Text-to-speech** (`audio.speech`) — type text and send. The reply is a playable audio bubble, produced by `POST /v1/audio/speech`. The whole clip comes back as one response and plays once it has fully arrived. This is the same transport [the Bench](../running/bench.md) uses to test these models, so what you hear in chat is exactly what the model produces. !!! note "Voice names are engine-specific" Text-to-speech engines have their own voice names. If you do not pass a `voice`, the inference plane invents none — the engine speaks in its own default. Set a voice your engine actually supports (an OpenAI voice name like `alloy` will not exist on a local engine). ## Voice agents A **voice agent** is a published agent whose graph starts with an audio input and ends with audio output. A typical pipeline transcribes the speech, sends the text to a language model, and speaks the answer: ```mermaid graph LR IN[input · modality audio] --> T[transcribe] T --> L[llm] L --> S[speak] S --> OUT[output · modality audio] ``` To use one, pick **Agent** in New Chat and select it. Ertir reads the agent's input and output nodes and gives it a **voice**/**audio** badge and the voice composer (mic and attach, text disabled). Then: 1. You record or attach a clip and send. 2. The clip is uploaded to the control plane as an **artifact** (`POST /artifacts`) and passed to the run as a reference — the raw bytes are handed straight to the inference plane, never journaled into run history. 3. The graph's **transcribe** node streams the audio to `POST /v1/audio/transcriptions` and produces text. 4. The **llm** node answers. 5. The **speak** node calls `POST /v1/audio/speech`, stores the produced audio as a new artifact, and returns its reference. 6. The run's output is that artifact reference; the chat downloads it (`GET /artifacts/{ref}`) into a playable audio bubble. Voice-agent runs are deliberately session-less on the server — the client records readable labels ("🎤 voice message" for your turn, "🔊 audio" for the answer) so the session shows the exchange without storing blobs. If a run finishes with no audio, you see "The agent produced no playable audio for this turn." ### The transcribe and speak nodes If you are building the agent yourself, these two [media nodes](../building/nodes/media.md) carry the voice work: | Node | Config | Notes | |---|---|---| | `transcribe` | `model`, `params` (`language`, `prompt`, …) | Audio in → text out. `model` is a logical id bound to a speech-to-text model. | | `speak` | `model`, `params` (`voice`, `format`, `speed`) | Text in → an audio artifact reference out. | The `speak` node defaults `format` to `mp3` when its params omit it; `format` maps to the wire's `response_format`. **`voice` has no default** — omit it and none is sent, so the engine speaks in its own. Because voice names are engine-specific, name one only when your text-to-speech engine actually offers it; see [the speak node](../building/nodes/media.md#speak-text-to-speech) for what an unknown voice looks like when it fails. ## What models you need Voice needs two kinds of engine, and Ertir forces a specific split for each: | Job | Engine | Binary | Notes | |---|---|---|---| | Speech-to-text (`audio.transcription`) | whisper.cpp's `whisper-server`, under the `(llamacpp, audio.transcription)` slot | `THEYGENT_WHISPERCPP_BIN` or on `PATH` (`brew install whisper-cpp`) | Needs whisper's native `ggml-*.bin` weights — a llama-style GGUF conversion will not load. With `ffmpeg` installed it transcodes browser webm/opus recordings; without it, only WAV works. | | Text-to-speech (`audio.speech`) | `mlx_audio.server`, under `(mlx, audio.speech)` | `THEYGENT_MLX_AUDIO_BIN` | Apple Silicon only; loads the voice model per request. | There is **no** MLX transcription slot and **no** llama.cpp speech slot: transcription is whisper.cpp only, and speech is Apple-Silicon MLX only. Install both local servers with `make engines`, or point the env vars at binaries you already have. See [the engines page](../models/engines.md) for the full engine matrix, and [installing models](../models/installing.md) to add the actual weights. If a required engine binary is missing, the model returns a clear `503 engine_unavailable` and the inference plane's `GET /readyz` lists the slot as not ready with an install hint — never a surprise crash on your first message. ## Where the audio lives Audio from **voice agents** — both your recorded input and the spoken output — is stored as local artifacts on the machine running the control plane, under `THEYGENT_ARTIFACT_DIR` (default: a `theygent-artifacts` folder in the system temp directory). Artifact ids look like `art_`. The `GET /artifacts/{ref}` route serves only stored `art_…` ids, so it cannot be used to read arbitrary files. The bytes are never written into run history or exported with telemetry, which is why a durable run that resumes after a crash simply replays the reference. Direct voice **model** chats create no artifacts: a transcript is plain text, and text-to-speech audio plays from the bytes the request returned. ## Continuing a voice session Reopen a voice conversation from **Chats** and the composer keeps talking to the same voice target, so you can pick up where you left off — a voice **model** through the direct data-plane transport it was recorded on, and a voice **agent** by re-reading the boundary from the version the conversation started on, so the microphone comes back rather than a text box. If that version has since been deleted, the conversation continues on the agent's latest. See [runs and sessions](../concepts/runs-and-sessions.md) for how sessions store and replay a conversation. ## Related pages - [Chat](index.md) — the shared conversation surface and its composer - [Vision and images](vision-and-images.md) — the visual counterpart: image input and image generation - [Audio & image nodes](../building/nodes/media.md) — building `transcribe`, `speak`, and `imagine` into a graph - [Engines](../models/engines.md) — llama.cpp, MLX, whisper.cpp and where each runs --- Source: https://docs.ertir.dev/latest/chat/vision-and-images/ # Vision and images [Chat](index.md) handles two visual jobs: **showing an image** to a model or agent and asking about it (vision), and **generating an image** from a text prompt. Both run directly against your inference plane — images never transit the control plane as raw payloads — and both work with a plain model or with a published agent. ## Chatting about an image (vision) ### Attaching or capturing an image When the target can see, the composer grows two image controls: - **Attach image** — pick an image file from disk. - **Capture from camera** — opens a live-preview modal (it prefers a rear-facing camera). **Capture** snapshots a single JPEG frame; the camera stream is released when you close the modal. The image stages as a removable chip above the input. Nothing is sent until you press **Send**. ### Vision models Pick **Model** in New Chat and select a model that advertises the `vision` capability. Image attach turns on automatically. Your image travels as an OpenAI `image_url` content block (a data URI) on the **same** `POST /v1/chat/completions` request as your text — there is no separate vision endpoint — and the answer streams back like any chat reply. Two local engines serve vision: | Engine | Program | How the image side works | |---|---|---| | `(llamacpp, vision)` | `llama-server` with a multimodal projector | The projector is taken from `params.mmproj`, else an `*mmproj*.gguf` next to the weights, else an automatic projector. | | `(mlx, vision)` | `mlx_vlm.server` (Apple Silicon) | A different program from the MLX chat server. | Installing a vision model is automatic: when you [install](../models/installing.md) an image-text-to-text model from the catalog, it registers with `vision` modality and spawns the vision server on first use. A hosted, OpenAI-compatible vision model works too — [register it](../models/remote.md) with `modality: vision`. !!! tip "Grounding models draw boxes" On [the Bench](../running/bench.md), when a vision model answers with structured JSON detections, the boxes are drawn over the image you sent. The same overlay renders detections from computer-vision tools in the MCP tool tester. ### Vision agents A **vision agent** is a published agent whose input node declares `modality: image`. Select it in New Chat and it gets a **vision** badge and a composer that takes an image *and* a question ("Attach an image and ask about it…"). The run travels through the control plane, and its [llm node](../building/nodes/llm.md) folds the image and text into one multimodal message: ```mermaid graph LR IN["input · modality image
(image + question)"] --> L[llm] L --> OUT[output · text] ``` The run input is `{image, text}`; the llm node references `$in.in.image` and `$in.in.text` and turns them into an `image_url` content part. An llm message's content can be a plain string or a list of text and `image_url` parts, and both work identically whether the agent runs interactively or [durably](../running/durable.md). ## Generating an image (text → image) ### With a model directly First [install](../models/installing.md) a text-to-image model from the catalog — its `images.generation` modality is derived automatically from the model's task tag. Then pick it in New Chat (**Model**); the composer prompt becomes "Describe an image to generate…". Sending calls `POST /v1/images/generations`, which returns the image bytes inline (`response_format: "b64_json"`, so no cross-machine URL), and the reply is an image bubble. The only metric is total latency. A few behaviors worth knowing: - **Size, count, steps.** `size` is `WxH`, snapped to multiples of 64 (default `512x512`); `n` is clamped to 1–4; `steps` passes through. - **Generation is serialized** — one render at a time, so concurrent requests queue — and weights load per request, so every image pays the model-load cost. A render whose caller hangs up is stopped, and a queued request whose caller has left never starts, so an abandoned image never holds up the next one. A slow render is kept alive with periodic keepalives so a long, delta-free generation is not mistaken for a dead connection. ### With an imagine-backed agent In a graph, the [`imagine` node](../building/nodes/media.md) does the generating: text in → an image **artifact reference** out, feeding an `output` node whose `modality` is `image`. ```mermaid graph LR IN[input · text prompt] --> IM[imagine] IM --> OUT[output · modality image] ``` Its config is `{model, params}`, where `size`, `n`, and `steps` ride through the params. In chat, such an agent gets an **image** badge; the answer downloads the artifact into an image bubble and records the label "🖼 image" to the session. If a run finishes without an image, you see "The agent produced no image for this turn." ### Image engines Both image engines are command-line generators wrapped by a small bundled OpenAI-compatible server: | Engine | Backed by | Binary | Defaults | |---|---|---|---| | `(llamacpp, images.generation)` | stable-diffusion.cpp's `sd-cli` | `THEYGENT_SDCPP_BIN` or on `PATH` | 20 steps | | `(mlx, images.generation)` | `mflux-generate` (FLUX, Apple Silicon) | `THEYGENT_MFLUX_BIN` (e.g. `uv tool install mflux`) | 3 steps | `sd-cli` runs anywhere its binary exists; the FLUX path is Apple Silicon only. If the underlying CLI is missing, the model shows as not-ready on `GET /readyz` with an install hint rather than failing with a 500. See [the engines page](../models/engines.md) for the full matrix. ## Artifact storage Images (and audio) produced by **agents** are stored as local artifacts on the machine running the control plane, under `THEYGENT_ARTIFACT_DIR` (default: a `theygent-artifacts` folder in the system temp directory). Artifact ids look like `art_`, and `GET /artifacts/{ref}` serves only stored `art_…` ids — passing anything else returns `400 invalid_artifact_ref`, and an unknown id returns `404 artifact_not_found`, so the route cannot read arbitrary files. The bytes are never journaled into run history or exported with telemetry, which is why a resumed durable run replays the reference instead of the image. Direct image-**model** generation creates no artifact: the bytes come back inline and the reply renders straight from them. ## Related pages - [Chat](index.md) — the shared conversation surface and its composer - [Voice](voice.md) — the audio counterpart: microphone input and spoken output - [Audio & image nodes](../building/nodes/media.md) — building `transcribe`, `speak`, and `imagine` into a graph - [Installing local models](../models/installing.md) — adding vision and image-generation models - [Remote models](../models/remote.md) — registering a hosted, OpenAI-compatible vision model --- Source: https://docs.ertir.dev/latest/models/ # Registries **Registries** is where you manage the models your agents and chats can call. It lists every model registered in your inference plane, lets you probe what each one can do, and gives you a browse-and-install catalog for pulling new models from Hugging Face — all without a model's weights ever passing through the control plane. Open it from the left sidebar, under **Configuration → Registries**. This page is a tour of the whole surface. For the step-by-step procedures — installing from the catalog, registering a model already on disk, and deleting one — see [Installing local models](installing.md). For pointing an id at a hosted or remote API, see [Remote models](remote.md). ## What Registries talks to The Registries page reads and writes your **inference plane** directly from the browser (by default `http://localhost:8081`). Everything you see here — the registered models, the Hugging Face catalog, download jobs, and local credentials — lives on your machine. The control plane (agents, runs, sessions) is never involved in registering a model or downloading weights. Every model is registered under a **logical id** — a short name you choose, like `triage-fast` — and your agents reference that id, never an engine. If this idea is new, read [Models and engines](../concepts/models-and-engines.md) first; the rest of this page assumes it. ## Your registered models The header shows **Resident Models: N / \** — how many engines are currently loaded in memory versus the ceiling (two by default). Below it, the **Installed** table lists every registered model: | Column | What it shows | |---|---| | **Logical id** | The name agents and chat use to reach this model. | | **Engine** | The binding, shown as a friendly label: `MLX`, `llama.cpp`, `vLLM`, or the raw value for a reachable endpoint (`openai-compatible`). | | **Model** | The weights or upstream model name. A local-path model shows only the file or directory name; hover for the full path. | | **State** | `resident` (loaded in memory now) or `cold` (registered but not loaded). | | **Capabilities** | Capability badges, once probed (see below). | | **Actions** | Bench, Warm, Evict, Delete. | The engine and state badges are clickable **filters**, and a search box narrows the list by logical id or model name. ### Reading capabilities A registered model's capabilities aren't known until the engine has loaded it, so the **Capabilities** cell starts empty. Click the search-icon **Probe capabilities** button on a row to find out — this loads the model into its engine (it warms the engine) and reads back what it supports. The badges you may see: | Badge | Meaning | |---|---| | `reasoning` | The model produces a separate "thinking" stream. | | `tools` | It supports tool/function calling. | | `structured` | It can return structured JSON output. | | `vision` | It accepts images as input. | | `k ctx` | Its maximum context window. | | `approx` | The report is an approximation read from model metadata rather than the live server. | If nothing is reported, the cell reads **none reported**. Probing a reachable (`openai-compatible`) endpoint loads nothing: a llama.cpp server is asked for its context size and chat template (marked **approx**); any other endpoint advertises only the modality you declared, with the rest unknown (**approx**). ### Row actions Each row carries four actions: - **Bench** — opens the model bench in a modal, a tester panel per modality with live metrics. See [The Bench](../running/bench.md). - **Warm** — loads the model into its engine now, so the next call skips the cold-start wait. - **Evict** — unloads the model from memory (freeing it for other engines). A model with a request in flight is drained rather than killed. - **Delete** — unregisters the model. This asks for confirmation and warns that any agent referencing the id will fail; it cannot be undone. Covered in [Installing local models](installing.md#deleting-a-model). Clicking a row (anywhere but its buttons) opens the model's registration — a catalog detail for Hugging Face installs, or an editable settings modal for everything else. ## Adding a model The **Add model** button opens a dialog with two tabs: - **Hugging Face** — paste a model id or URL, or open the full browse dialog to search the catalog. This is the path for pulling a new open model to your machine. - **Hosted / local** — register a model manually: a hosted or remote OpenAI-compatible endpoint, or weights already sitting on your disk. Both flows end with you choosing a logical id. The procedures are in [Installing local models](installing.md) (catalog and on-disk) and [Remote models](remote.md) (hosted endpoints). ## Browsing the Hugging Face catalog From **Add model → Browse Hugging Face →** you get a searchable catalog of open models. One rule shapes everything you see here: **discovery only shows models you can actually run.** The listing is filtered to the engines that are ready on your machine, so a model you have no engine for never appears. If no local engine is ready at all, the dialog tells you to install one first. The controls: - **Search** — free text, e.g. `qwen`, `llama`, `phi`. - **Size** — Any / Small (<3B) / Medium (3–15B) / Large (>15B), by parameter count. - **Sort** — Trending (default) / Most downloaded / Most liked. - **Engines** — chips that limit the listing to specific ready engines (at least one is always active). - **Task chips** — single-select, applied on the server: Chat, Vision, Image generation, Embeddings, Speech-to-text, Text-to-speech. - **Capability chips** — multi-select, applied to the loaded page: Reasoning, Tools. Scrolling grows the page up to 100 entries. ### Reading a catalog card Each card summarizes a model before you download anything: - Title, repository reference, and parameter count (e.g. "7B"). - A **✓ installed** badge if you've already registered it. - A lock and **gated** marker — "needs a Hugging Face token" — for repositories that require authentication. - License, a coarse "updated" time, download and star counts, and a **View on Hugging Face ↗** link. - A **task badge** (vision, embeddings, speech-to-text, text-to-speech, image generation). - **Capability badges at browse time** — reasoning, tools, vision, and context window, derived from the model's Hugging Face metadata *without downloading it*. These carry an **approx** marker: "static hint from model metadata — the probe confirms it on install." Because the browse hint and the post-install probe use the same signals, they won't contradict each other. ### Variants and fit Expand a card to see its **variants** — the concrete things you can install. For a llama.cpp model these are the individual GGUF quantizations (labels like `Q4_K_M` with a quality note); for an MLX model it's the whole repo at its precision (e.g. "4-bit"). One variant is marked **recommended** (the largest that fits your machine, or the smallest if none do). Every variant carries a **fit badge** sized against your machine's RAM, so you can tell before downloading whether it will run: | Fit badge | Meaning | |---|---| | **fits** (green) | Comfortably within memory. | | **tight** (amber) | Will load but leaves little headroom. | | **too large** (red) | Likely exceeds your memory — it may fail to load or run slowly. | | **size unknown** (slate) | The size couldn't be determined. | Hover a badge for the reasoning, e.g. "~5 GB needed · 16 GB RAM". Picking a too-large variant is allowed but warns you first. ### Incomplete image models Some image-generation repositories publish only **part** of a model. A modern text-to-image model is three pieces — a denoiser, a VAE, and a text encoder — and many repositories ship quantizations of the denoiser alone, with the other two living in separate repositories. The files look like everything else: multi-gigabyte, `.gguf`, tagged for text-to-image. Ertir loads one self-contained checkpoint, so a denoiser on its own can't render. Those variants show an **incomplete** badge instead of a fit badge, their **Install** button is disabled, and none of them is ever marked recommended. Hover the badge for what's missing. Look for a repository that publishes a complete checkpoint — a single file that bundles all three parts. If a repository slips past this check, the install still succeeds and the refusal arrives when you register or first use the model, naming the same missing parts. ### Download progress When you install, the download runs **inside your inference plane** — Ertir never sees the bytes. Progress appears as a card in the notification center (bottom-right): a progress bar, bytes done and total, live speed, an ETA, and a **Cancel** button. It moves through `downloading → registering → done`, survives navigation and page reloads (it re-attaches to active jobs), and finishes with "Installed ✓". On completion the model is auto-registered under the logical id you chose and appears in the Installed table. The full install walkthrough — including the install dialog and choosing a logical id — is on the next page. ## Moving models to another machine The **Model registry** section of **Settings → [Import / Export](../import-export/index.md)** carries your registrations between installs. A registry export contains the logical-id → binding map plus, for catalog installs, the Hugging Face repo and variant each one came from — **never the weights themselves, and never credential values** (credential names travel so the target knows what to ask you for). On import: - **Catalog-installed models re-download automatically.** The target's inference plane pulls the same repo and variant from Hugging Face — with the exported registration preserved (modality, parameters, lifecycle), only the weights path pointing at the new machine. Machine-specific auxiliary paths in the parameters (such as a vision projector file) are dropped with a `params_path_dropped` warning and re-located next to the new weights automatically. Progress shows in the notification center like any install. - **Hosted and remote registrations just work** — an `openai-compatible` binding is a URL plus a credential name; re-enter the credential on the target and it's live. - **Hand-registered local paths can't follow you.** A model registered against a file path on the old machine (with no catalog provenance) is registered anyway but flagged `weights_unavailable` — it fails at first use until you put weights at that path or reinstall it. - **Already-registered logical ids are skipped**, never overwritten. ## Related pages - [Installing local models](installing.md) — the step-by-step install, on-disk registration, and deleting models - [Remote models](remote.md) — registering a hosted or remote OpenAI-compatible endpoint - [The engines](engines.md) — llama.cpp, MLX, and vLLM, and which one runs on your machine - [Models and engines](../concepts/models-and-engines.md) — logical ids, bindings, sources, and modalities - [Import & export](../import-export/index.md) — moving the registry (and everything else) between machines - [The Bench](../running/bench.md) — testing and comparing models --- Source: https://docs.ertir.dev/latest/models/installing/ # Installing local models This page covers getting a local model registered and running: installing one from the Hugging Face catalog, registering weights you already have on disk, where those weights live, and how to delete a model. For an orientation to the whole page, see the [Registries tour](index.md); for hosted APIs, see [Remote models](remote.md). **Before you start:** installing a *local* model needs a matching engine ready on your machine — run `make engines` (see [Installation](../getting-started/installation.md#3-install-the-local-engines)). The catalog only offers models you have an engine for. If you have no local engine, register a [remote model](remote.md) instead. ## Install from the catalog The catalog browses Hugging Face and downloads straight into your inference plane. 1. Open **Registries** from the sidebar (under **Configuration**). 2. Click **Add model**, then **Browse Hugging Face →**. 3. **Search** for a model — e.g. `qwen`, `llama`, `phi` — and narrow with the **Size**, **Sort**, **Engines**, and task/capability filters (**Reasoning**, **Tools**, **MoE**). Only models you can run appear. 4. **Expand a card** to see its variants (quantizations). Each has a **fit badge** against your RAM — prefer one marked *recommended* that shows **fits**. 5. Click **Install** on the variant. In the dialog, confirm the **Logical id** — the short name agents and chat will use ("how agents reference it"). One is suggested; for a llama.cpp variant the quantization is appended so different quants get distinct ids. 6. Click **Download & install**. The download appears in the notification center (bottom-right) with a progress bar, speed, ETA, and a **Cancel** button. It runs entirely on your machine and survives navigation. When it finishes, the model is auto-registered and shows up in the **Installed** table under your logical id, ready to chat with or wire into an agent. !!! tip "Quick-add by id or URL" If you already know the exact model, skip the browse dialog: in **Add model → Hugging Face**, paste the id or URL (for example `mlx-community/Qwen2.5-0.5B-Instruct-4bit`) and press **Add** to jump straight to its variants. !!! tip "Mixture-of-experts models bigger than memory" Cards marked **MoE** are mixture-of-experts models (the **MoE** chip shows only those). When you install the MLX variant of one that won't fit in your memory, the dialog offers **Stream experts from disk**, ticked for you: the model then keeps only the experts in use in memory and reads the rest from your SSD, so it can run — slower, with the same answers. You can turn it off later in the model's settings. See [expert streaming](engines.md#moe-models-bigger-than-memory-expert-streaming). !!! note "Modality is detected for you" A catalog install sets the model's modality from the repository's declared task — a vision model registers as `vision`, an embeddings model as `embeddings`, and so on — so the right server spawns automatically on first use. You don't pick it by hand. ??? note "Gated repositories" Some repositories are **gated** and need a Hugging Face token — the catalog marks them with a lock and "needs a Hugging Face token" before you click. If you have logged in to Hugging Face on this machine (or set a token in your environment), gated downloads use that ambient login. There is no token field in the install dialog. ## Register a model already on disk If you already have model weights on your machine — a GGUF file, or an MLX model directory — you can register them without going through the catalog. 1. **Add model → Hosted / local** tab. 2. Enter a **Logical id** (e.g. `triage-fast`). 3. Choose the **Binding** for the weights: `llamacpp` (a GGUF file) or `mlx` (an MLX model directory). Which one your file is for depends on its format — see [The engines](engines.md). 4. In **Model**, enter the path to the weights on disk. 5. Save. Models registered this way use **source `local-path`** — Ertir loads them from where they are and never copies or moves them. The first call to the logical id spawns the engine against that path. You can also register a local model over the management API directly. The inference plane listens on `http://localhost:8081`, and registration is a `PUT` under the logical id (the wire format is camelCase): ```bash curl -X PUT http://localhost:8081/admin/models/triage-fast \ -H 'Content-Type: application/json' \ -d '{ "binding": "llamacpp", "source": "local-path", "model": "/path/to/model-Q4_K_M.gguf", "modality": "chat" }' ``` ## Where weights live on disk Catalog downloads land under your inference plane's model directory: ```text ~/.theygent/inference/models// ``` The repository's `org/name` is sanitized to `org_name` (the slash becomes an underscore). To keep weights somewhere else — an external volume, say — set `THEYGENT_INFERENCE_PLANE_MODEL_DIR` before starting the inference plane. The registry that maps logical ids to weights is a separate file, `~/.theygent/inference/registry.json`; its parent directory is set by `THEYGENT_INFERENCE_PLANE_STATE_DIR`. Both are covered in the [environment reference](../reference/environment.md). Models registered with **source `local-path`** are *not* copied here — they stay wherever you pointed the registration. Catalog installs also record *where they came from* (the repository and variant), so a registry export can re-download the same weights on another machine — the weights themselves are never part of an export. See [Moving models to another machine](index.md#moving-models-to-another-machine). ## Editing a registered model Click a model's logical id in the **Installed** table to open its settings. For a model registered from a Hugging Face id (source `hf`) this reopens its catalog detail, with an **Edit registration** button for the settings; for anything else it opens the editable settings directly. What you can change depends on the binding: - **Logical id** and **Binding** are read-only. Renaming isn't an edit — register under a new logical id instead. - **Model**, **Source** (or **Base URL** for reachable endpoints), and **Modality** are editable. - For a **managed** engine with a lifecycle, you can set the **Idle timeout (seconds)** and tick **Keep warm (never auto-evict)** to keep the model resident. - For an **MLX chat** model, **Stream experts from disk** turns [expert streaming](engines.md#moe-models-bigger-than-memory-expert-streaming) on or off, and **Expert cache (GB)** sizes its memory cache (blank means half your memory, less the model's core). Models that stream show a **streams experts** badge in the table. - For a **reachable** endpoint, you can edit the **Credential** and a **Params (JSON)** block. ## Deleting a model To unregister a model, use the **Delete** action on its row (the trash icon). A confirmation dialog warns: > agents referencing `` will fail. This cannot be undone. Deleting evicts the engine if it's resident, then removes the registration. It does **not** delete the downloaded weight files from disk — those stay under your model directory; remove them yourself if you want the space back. Any agent, chat, or trigger that still points at the deleted logical id will fail with a `model_not_found` error until you register something under that id again. !!! warning "There's no undo" Deletion removes only the registration, but agents don't know the id is gone until they run. If several agents share a model id, deleting it breaks all of them at once. Consider [re-registering the id](../concepts/models-and-engines.md#swapping-a-model-without-touching-agents) under a different model instead of deleting. ## Related pages - [Registries](index.md) — the full page tour, capability probing, and browse badges - [Remote models](remote.md) — registering a hosted or remote endpoint under a logical id - [The engines](engines.md) — which binding your weights need, and platform support - [Your first chat](../getting-started/first-chat.md) — install a model and talk to it - [Environment variables](../reference/environment.md) — `THEYGENT_INFERENCE_PLANE_MODEL_DIR` and friends --- Source: https://docs.ertir.dev/latest/models/remote/ # Remote models Not every model has to run on your machine. You can register **any OpenAI-compatible server or hosted API** under a logical id and call it exactly like a local model — the same `llm` node, the same chat surface, the same agents. This page covers registering one, the credential store that keeps your API key on your machine, the quirks worth knowing, and how to mix local and remote models in a single agent. Remote models use the **`openai-compatible`** binding. Unlike the managed engines, Ertir never starts, stops, or supervises a remote endpoint — it simply forwards your calls to the base URL you give it. For the binding vocabulary, see [Models and engines](../concepts/models-and-engines.md). !!! note "The honest boundary" A remote model means your prompts go to whatever endpoint you register — a hosted provider, or a server on your own network. That is your choice and your hop: Ertir forwards the call and relays the answer, and never inserts itself as a required middleman. Local models keep everything on your machine; remote models go exactly where you point them. ## 1. Store the API key as a local credential Most hosted APIs need an API key. Rather than pasting it into the model registration, you store it once as a **local credential** and reference it by name. 1. Open **Settings** from the sidebar, and find **Endpoints & credentials → Local credentials**. 2. Add a credential: a **Name** (must be environment-variable shaped, e.g. `OPENAI_API_KEY`) and its **Value** (your key). Credentials are **write-only** — the list shows the name and "•••••• set", and the value can never be read back through the UI. They are stored on your inference plane, on this machine, and are **never sent to the control plane**. A model registration refers to a credential by the scheme `secret://NAME` (for example `secret://OPENAI_API_KEY`). !!! tip "You can also use an environment variable" When a registration references `secret://NAME`, the inference plane resolves it by looking first in the local credential store, then at a process environment variable of the same `NAME`. So a key already exported into the inference plane's environment works without adding it to the store. ## 2. Register the remote model 1. In **Registries**, click **Add model → Hosted / local**. 2. Enter a **Logical id** (e.g. `writer` or `triage-fast`) — the name your agents will use. 3. Leave **Binding** on **openai-compatible** (the default for this tab). 4. In **Model**, enter the **provider's** model name — whatever id that endpoint expects in its `model` field. 5. Enter the **Base URL**, including the API path (see the quirk below). 6. Pick a **Credential** — a stored `secret://NAME`, "— no credential —", or a custom ref. 7. Save. The equivalent registration over the management API (the inference plane on `http://localhost:8081`, camelCase wire format): ```bash curl -X PUT http://localhost:8081/admin/models/writer \ -H 'Content-Type: application/json' \ -d '{ "binding": "openai-compatible", "baseUrl": "https://api.your-provider.example/v1", "model": "provider-model-id", "credentialRef": "secret://OPENAI_API_KEY", "modality": "chat" }' ``` For a non-chat upstream (an embeddings or speech endpoint, say), set `modality` accordingly. A reachable endpoint is a pure declaration — Ertir doesn't probe it — so the modality you declare is the one it's routed as. If the endpoint needs no authentication (a local server, for instance), leave the credential empty; Ertir sends a placeholder key that unauthenticated servers ignore. ## Quirks worth knowing ### The base URL must include the API prefix The base URL is the **root the endpoint's `/chat/completions`, `/embeddings`, etc. hang off** — for hosted OpenAI-compatible APIs that almost always ends in `/v1`. If you drop the prefix, calls fail with a `modality_not_supported` error whose message says the path looks wrong. When in doubt, use the same base URL the provider documents for its OpenAI-compatible endpoint, trailing `/v1` included. ### Some hosts reject parameters others accept Hosted models don't all take the same parameters — one may reject `temperature`, another may not support a particular sampling knob. Ertir doesn't second-guess this: it relays the request as you built it, and if the provider rejects a parameter, **the provider's own error message and status come straight back to you**, before the answer starts streaming. If a call fails with an "unexpected parameter" style error, remove that parameter from the model's or node's params. A failure that happens partway through a stream ends the stream with a structured error rather than a torn connection. ### A reply cut off mid-stream can read as finished If a remote server stops partway through a streamed reply, its stream just ends. The layer Ertir dispatches through then closes the stream as though the reply had finished. Your agent gets a shorter answer with nothing to say it was cut off. Local models are always checked for this, because every engine Ertir manages ends a finished reply with a **finish reason**. A remote server is checked only when you say it sends one too: a server that never does would have every reply fail. To turn the check on, click the model in **Registries** and tick **Report a reply cut off mid-stream** (shown for chat models), then save. Over the management API, it's `"requireFinishReason": true` on the registration: ```bash curl -X PUT http://localhost:8081/admin/models/writer \ -H 'Content-Type: application/json' \ -d '{ "binding": "openai-compatible", "baseUrl": "https://api.your-provider.example/v1", "model": "provider-model-id", "credentialRef": "secret://OPENAI_API_KEY", "requireFinishReason": true }' ``` With it on, a streamed reply that stops before the server's finish reason ends with an [`engine_dropped_request`](../reference/troubleshooting.md#engine_dropped_request-502-or-an-error-at-the-end-of-a-stream) error. What arrived before the cut is still relayed, so nothing reads as a finished answer. Streams that end normally are unchanged. Turn it on for a server you know sends a finish reason at the end of every streamed reply. OpenAI-compatible servers normally do, and **Anthropic's OpenAI-compatible API** was checked: plain replies, tool calls, and replies stopped by a token limit all end with one. If you're not sure about a server, leave it off. If every streamed reply from the model then fails with `engine_dropped_request` right after the full answer, the server doesn't send one, so turn it off again. A request the server drops before it starts replying is named either way. ### Remote models have no lifecycle There is nothing to warm, evict, or wait for. A remote id never counts against your resident-engine ceiling, its **State** stays effectively cold, and **Warm** / **Evict** are no-ops. Probing loads nothing: for a llama.cpp server you started yourself (`llama-server`), the probe reads its real context size and chat template, the same way it does for a local llama.cpp model, marked **approx** because Ertir didn't choose its flags. Any other endpoint offers no way to ask, so it advertises only the modality you declared and its other capabilities read as unknown (**approx**), not as unsupported. ## Mixing local and remote in one agent
Run Local + Remote AI Models in One Agent (7:01) — the routed hybrid below, built and run: the local model answers the easy request, the hosted one takes the hard question.
Because both local and remote models are just logical ids, a single agent can use both — routing simple work to a fast local model and hard work to a stronger hosted one. The usual shape is a small local classifier that decides, feeding a [router node](../building/nodes/router.md) that sends each request down the matching branch: ```mermaid graph LR IN["input"] --> C["llm: local triage
(classifies the request)"] C --> R{"router
select: $in.in.route"} R -->|local| L["llm: local model"] R -->|remote| H["llm: hosted model"] L --> OUT["output"] H --> OUT ``` Each `llm` node names its own logical id, so one node runs on a local engine and another calls a hosted API — in the same run. This "hybrid" pattern keeps cheap, private traffic on your machine and reserves the paid hosted call for the requests that need it. To build it, see the [router node](../building/nodes/router.md) and [Referencing inputs](../building/input-references.md) for the `$in..` selector the router reads. Nothing about the agent changes when you swap a local id for a remote one later — re-register the id and every agent that uses it routes to the new endpoint on its next run. See [Swapping a model without touching agents](../concepts/models-and-engines.md#swapping-a-model-without-touching-agents). ## Related pages - [Registries](index.md) — the models page and its Installed table - [Installing local models](installing.md) — the local counterpart to this page - [Models and engines](../concepts/models-and-engines.md) — bindings, sources, and logical ids - [The llm node](../building/nodes/llm.md) — using a model id in an agent - [Router node](../building/nodes/router.md) — the branching node for hybrid local/remote agents --- Source: https://docs.ertir.dev/latest/models/engines/ # The engines An **engine** is the program that actually runs a local model. Ertir manages three of them — **llama.cpp**, **MLX**, and **vLLM** — plus a fourth binding, `openai-compatible`, for endpoints it merely reaches. This page explains what each engine is for, where it runs, how the managed ones start and share memory, and which modalities (chat, vision, embeddings, speech, images) each covers. If you only need to *use* models, you don't have to think about engines much — a catalog install picks the right one for you. Read on when you're choosing hardware, registering weights by hand, or wondering why a first call is slow. For the vocabulary of bindings, sources, and modalities, see [Models and engines](../concepts/models-and-engines.md). ## The four bindings Every registered model has a **binding** that says what runs it. There are exactly four values, and their meaning never changes: | Binding | What it runs | Managed? | |---|---|---| | `llamacpp` | A llama.cpp server on GGUF weights. Runs on any platform where the binary is installed. | Yes — spawned and supervised. | | `mlx` | The MLX family of servers, on Apple Silicon. | Yes. | | `vllm` | A vLLM server on an NVIDIA/CUDA GPU host. | Yes (experimental). | | `openai-compatible` | Any OpenAI-compatible server or hosted API at a base URL you supply. | No — reached, never managed. | The first three are **managed**: Ertir starts the server on first use, health-checks it, and stops it when idle. `openai-compatible` is **reached** — nothing is spawned or stopped; see [Remote models](remote.md). ## llama.cpp **llama.cpp** runs models in the **GGUF** format and is the broadest, most portable local engine — it runs anywhere its `llama-server` binary is installed (macOS, Linux, with or without a GPU). It's the default choice when you want a model to run on the same machine regardless of platform. GGUF models come in **quantizations** (labels like `Q4_K_M`), and the catalog lists each quant as a separate variant with its own fit badge. Under the `llamacpp` binding, Ertir covers several modalities by spawning the right server for the registered modality: - **Chat** and **embeddings** — the `llama-server` binary (embeddings use mean pooling). - **Vision** — the same `llama-server` with a multimodal projector; Ertir finds the projector file next to the weights or falls back to auto-detection. - **Speech-to-text** — served by **whisper.cpp**'s `whisper-server`, registered under `binding: llamacpp` with `modality: audio.transcription`. With `ffmpeg` installed it also transcodes browser microphone recordings (webm/opus); without it, only WAV works. Whisper needs its own native `ggml-*.bin` weights — a chat-style GGUF won't work, and an ambiguous weights path raises a clear error telling you which file to point at. - **Image generation** — a bundled wrapper around stable-diffusion.cpp's `sd-cli` (see the note under [Installing the engine binaries](#installing-the-engine-binaries)). ### llama.cpp launch settings A llama.cpp chat, vision, or embeddings model starts with settings sized to fit a laptop rather than the model's maximum: | Setting (`params` key) | `llama-server` flag | Default | |---|---|---| | `ctxSize` | `-c` | Chat and vision: the model's trained context, capped at 32,768 tokens. Embeddings: the trained context, capped at 8,192 (2,048 when the file doesn't say). | | `parallel` | `-np` | `1` — one request at a time, so the whole context serves it. | | `batchSize` | `-b` | Embeddings: the context size. Otherwise llama-server's own default. | | `ubatchSize` | `-ub` | Embeddings: the context size, so any input that fits the context fits one batch. Otherwise llama-server's own default. | Set any of them in the model's registration `params` to override, for example `"params": {"ctxSize": 65536}` for a longer context, or `{"parallel": 4, "ctxSize": 65536}` to serve four requests at once (the context is shared by the slots). They configure the engine when it starts — they are never sent with a request — and a value that isn't a positive integer, or one set on a non-llama.cpp model, is refused when you register it. The context a running model actually has shows as **max context** in its capabilities. !!! tip "Out of GPU memory" If a large model fails with `Compute error` or runs out of GPU memory, lower its `ctxSize`: the context's cache is allocated up front, on top of the weights. ## MLX **MLX** is Apple's array framework, and Ertir's MLX engines run **only on Apple Silicon** (M-series Macs). On that hardware they're fast and memory-efficient because the CPU and GPU share one pool of unified memory. When you're on an Apple Silicon Mac, MLX is usually the best local option for chat and vision; the same model repo often ships both a GGUF (for llama.cpp) and an MLX variant, and the catalog shows whichever your ready engines can run. Under the `mlx` binding, the modalities map to distinct server programs: - **Chat** — `mlx_lm.server`. - **Vision** — `mlx_vlm.server` (a separate program). - **Text-to-speech** — `mlx_audio.server`. - **Image generation** — a wrapper around `mflux-generate` (see the binaries note below). MLX does **not** serve embeddings or speech-to-text — use llama.cpp (embeddings) and whisper.cpp (transcription) for those, on the same machine. ### MLX launch settings `mlx_lm.server` keeps the processed state of recent prompts and reuses the one whose start matches a new request, so an agent's next step processes only what's new since its last call. On a 35B model that's the difference between about 0.9 seconds and 2–3 seconds before the first token of each step. | Setting (`params` key) | Default | Effect | |---|---|---| | `promptCacheSize` | `10` | How many prompt states the engine keeps for reuse. Raise it when one model serves several agents or concurrent runs; each state takes memory (about 0.1 GB for a 1,500-token prompt on a 35B model). | Like the other launch settings it applies when the engine starts, is never sent with a request, and is refused at registration (`422 invalid_binding`) unless it's a positive integer on an MLX chat model. ### MoE models bigger than memory (expert streaming) A **mixture-of-experts** (MoE) model splits most of its weights into many small "experts" and uses only a few of them for each token — Qwen3.6-35B-A3B, for example, picks 8 of its 256 experts per layer. With **expert streaming**, Ertir keeps the rest of the model in memory and reads each layer's chosen experts from disk as they're needed, holding the recently used ones in a memory cache. A model whose full weights don't fit in your Mac's memory can still run; the answers are exactly the ones the fully loaded model gives, only slower. It's off unless you turn it on for a model, because a model that fits in memory is always faster fully loaded. Three ways to turn it on: - **When installing**: the catalog marks mixture-of-experts models **MoE**, and for an MLX variant that won't fit, the install dialog offers **Stream experts from disk**, already ticked. See [Installing local models](installing.md). - **Later**: in **Registries**, open the model and tick **Stream experts from disk** in its settings; **Expert cache (GB)** sets the cache size. Untick it to load the model fully again. - **Through the API**, with these registration `params`: | Setting (`params` key) | Default | Effect | |---|---|---| | `streamExperts` | `false` | `true` serves the model with its experts streamed from disk. | | `expertCacheGb` | Half your machine's memory, less the part of the model that stays loaded | How much memory the expert cache may use. Requires `streamExperts: true`. | For example, registering a model through the inference plane's API: ```bash curl -X PUT http://127.0.0.1:8081/admin/models/big-moe \ -H 'Content-Type: application/json' \ -d '{"binding": "mlx", "source": "hf", "model": "mlx-community/Qwen3.6-35B-A3B-4bit", "params": {"streamExperts": true, "expertCacheGb": 8}}' ``` Like the llama.cpp launch settings, these configure the engine when it starts and are never sent with a request. A value that isn't `true`/`false` or a positive number, `expertCacheGb` without `streamExperts`, or either setting on anything but an MLX chat model is refused when you register it (`422 invalid_binding`). What to expect, measured with `Qwen3.6-35B-A3B-4bit` (20 GB) on a 36 GB Apple Silicon Mac, generating 128 tokens: | How it runs | Memory used | Speed | |---|---|---| | Fully loaded (streaming off) | 19.7 GB | 53 tokens/s | | Streaming, every expert in the cache | 20 GB | 34 tokens/s | | Streaming, 4 GB expert cache, experts read from the SSD | 5.7 GB | 9 tokens/s | | Streaming, smallest cache, every expert read from the SSD | 1.8 GB | 4 tokens/s | Speed depends mostly on how many of the experts a reply needs are already in the cache, then on your SSD. !!! warning "Limits of expert streaming" - **Only for models that don't fit.** A model that fits in memory is always faster fully loaded. - **MoE models in MLX format only**, on Apple Silicon, for chat. A model the feature can't stream — one without experts, one whose experts are a kind it doesn't read, or a checkpoint that wasn't converted for MLX — is refused when its engine starts, and the model's engine log says why. - **Long prompts are slow from a cold disk.** Processing a long prompt touches nearly every expert, so the first reply to a long prompt can take several seconds per 2,000 tokens before the cache warms up. - **Parallel requests** use more distinct experts at once, so the cache helps less. - LoRA adapters aren't supported with streaming. ## vLLM **vLLM** targets **NVIDIA/CUDA GPU hosts** and is built for high-throughput serving. In this build it is **experimental and unverified**: the `vllm` binding exists and targets chat, but it has not been confirmed on the maintainers' hardware, and it is deliberately left out of the automatic engine install. !!! warning "Treat vLLM as experimental" Only the `chat` modality is wired for vLLM, and none of it is verified in this codebase. If you have a CUDA GPU, you can try it — install vLLM yourself and register a model under the `vllm` binding — but validate it on your own hardware before relying on it. llama.cpp and MLX are the proven local paths. ## Modality support matrix Which engine covers which modality: | Modality | llama.cpp | MLX | vLLM | |---|---|---|---| | **Chat** (`chat`) | ✅ | ✅ | ⚠️ experimental | | **Vision** (`vision`) | ✅ | ✅ | — | | **Embeddings** (`embeddings`) | ✅ | — | — | | **Speech-to-text** (`audio.transcription`) | ✅ (whisper.cpp) | — | — | | **Text-to-speech** (`audio.speech`) | — | ✅ | — | | **Image generation** (`images.generation`) | ✅ (sd-cli) | ✅ (mflux) | — | A registered model whose exact (engine, modality) combination has no server is refused with a clear error rather than being served wrong. `openai-compatible` endpoints aren't in this table — they serve whatever the upstream serves, and you declare the modality when you register them. !!! note "Image generation needs extra setup" The image-generation engines (stable-diffusion.cpp's `sd-cli` and `mflux`) are the newest and least-exercised paths, and their binaries are **not** installed by the standard engine install. You must build/install them yourself and point Ertir at them (`THEYGENT_SDCPP_BIN`, `THEYGENT_MFLUX_BIN`) before an image-generation model will run. See [Image generation in chat](../chat/vision-and-images.md). ## How managed engines run You never start or stop a managed engine by hand. Ertir handles the whole lifecycle, and understanding it explains a few things you'll notice. **Lazy start.** An engine spawns on the **first call** to one of its logical ids, on a free local port, and Ertir waits for it to become healthy before forwarding your request. This is why the very first call to a model is slower than the ones after it — the model is loading. Large models take longer; startup waits allow a few minutes for llama.cpp and MLX, and up to ten for vLLM. Once loaded, the engine stays **warm** for subsequent calls. **A resident ceiling.** Only a limited number of engines stay loaded at once — **two by default** (`THEYGENT_MAX_RESIDENT`). When a new engine is needed and the ceiling is full, Ertir **evicts** one to make room, choosing the lowest-priority, least-recently-used engine. An engine with a request in flight is **never** evicted; if nothing can be freed, the new call is refused with a `no_capacity` error rather than thrashing. **Abandoned calls are cancelled.** If the caller hangs up before an answer arrives — its client timed out, or the connection closed — Ertir closes the request to the engine right away instead of holding it open for an answer nobody will read. That holds for every call that returns one answer rather than a stream: chat answers, embeddings, transcriptions, speech, and generated images. Engines that watch for a closed connection stop working on the request at once, so the engine is free for the next call — llama.cpp stops generating a chat answer, and the bundled image generator stops an abandoned render and skips one still waiting its turn, which matters most because a render can take minutes. Either way, a model you asked to evict no longer waits for the abandoned answer before it unloads. A request to a hosted endpoint is closed the same way. **Idle reaping.** An engine left idle longer than its idle timeout (**900 seconds** by default) is torn down automatically by a background sweep, freeing its memory. Mark a model **Keep warm** to exempt it from reaping. **Manual control.** From a model's row in [Registries](index.md), **Warm** preloads an engine (so the next call skips the cold start) and **Evict** frees it now. Both are safe no-ops for reachable endpoints and models that aren't loaded. **Missing binaries fail cleanly.** If a model's engine binary isn't installed on the machine, the call returns `engine_unavailable` — a clear, up-front error — never a mysterious crash on first inference. **A failed engine is replaced.** If an engine reports that it can no longer compute (llama.cpp's `Compute error`, usually after it ran out of GPU memory) or its process dies, Ertir stops using it: the call fails with `engine_failed` (503, safe to retry), the engine is shut down, and the next call starts a fresh one. The model's row in [Registries](index.md) shows a red **failed** badge, with the reason on hover, until the replacement starts. **A dropped request is named.** If an engine closes the connection before it finishes an answer, the call fails with `engine_dropped_request`, and the engine keeps serving other requests. This covers a reply cut off partway through a stream, which would otherwise read as a finished, shorter answer. The inference plane doesn't retry a request to a local engine; the caller's own retry policy decides. See [troubleshooting](../reference/troubleshooting.md#engine_dropped_request-502-or-an-error-at-the-end-of-a-stream) for the known MLX tool-call cause. **Engine logs.** Each managed engine writes its own output — model loading, warnings, errors — to a log file under the inference plane's state directory (`~/.theygent/inference/logs/` by default), one file per model and modality, kept across restarts and rotated at 8 MB. Open it from the **Logs** button on the model's row, or with `GET /admin/models/{id}/logs`. ```mermaid stateDiagram-v2 [*] --> Cold Cold --> Loading: first call to the id Loading --> Resident: healthy Resident --> Cold: idle timeout / evicted for room Resident --> Resident: more calls (stays warm) ``` ## Installing the engine binaries The engines are separate programs. On macOS, one target installs the managed set: ```bash make engines ``` === "Apple Silicon Mac" `make engines` installs the full set: `llama.cpp`, `whisper.cpp`, and `ffmpeg` via Homebrew, plus the MLX chat, vision, and text-to-speech servers via `uv tool install`. This gives you local chat, vision, embeddings, speech-to-text, and text-to-speech. === "Linux / other" `make engines` prints guidance instead of installing. Install `llama-server` (llama.cpp), `whisper-server` (whisper.cpp), and `ffmpeg` with your package manager. The MLX servers are Apple-Silicon-only and won't run here. === "NVIDIA GPU host" Install vLLM yourself with `pip install vllm`. It is deliberately not part of `make engines`. Remember it is experimental in this build. Ertir finds each binary by checking an explicit environment override first, then your `PATH`, then (for Python-module servers) `python -m …`. To point at a binary in a non-standard location, set its override — `THEYGENT_LLAMACPP_BIN`, `THEYGENT_MLX_BIN`, `THEYGENT_MLX_VLM_BIN`, `THEYGENT_MLX_AUDIO_BIN`, `THEYGENT_WHISPERCPP_BIN`, `THEYGENT_VLLM_BIN`, `THEYGENT_SDCPP_BIN`, or `THEYGENT_MFLUX_BIN`. The full list is in the [environment reference](../reference/environment.md). Ertir launches and supervises these binaries but never builds or downloads them — that part is on you (or on `make engines`). ## Checking what's ready Which engines and modalities your machine can actually run is reported by the inference plane's readiness endpoint. Every (engine, modality) **slot** shows separately — a chat slot as the bare engine name (`llamacpp`, `mlx`), a non-chat slot as `engine:modality` (`mlx:vision`, `llamacpp:audio.transcription`) — and a not-ready slot carries a reason with an install hint: ```bash curl http://localhost:8081/readyz ``` The service stays "ready" as long as **any** engine is available, so a Mac without CUDA still serves llama.cpp and MLX happily. The catalog's engine filters mirror this — it only offers models for slots that are ready. ## Related pages - [Models and engines](../concepts/models-and-engines.md) — bindings, sources, modalities, and logical ids - [Registries](index.md) — the models page, Warm/Evict, and resident count - [Installing local models](installing.md) — installing weights for these engines - [Remote models](remote.md) — the `openai-compatible` binding - [Installation](../getting-started/installation.md) — running `make engines` - [Environment variables](../reference/environment.md) — `THEYGENT_MAX_RESIDENT`, binary overrides, and more --- Source: https://docs.ertir.dev/latest/rag/ # RAG sources A **RAG source** is a named collection of documents your agents can search — your product docs, a contract folder, an internal wiki. You fill it once (upload files or crawl a site), Ertir chunks and embeds the text against a model *you* choose, and stores the vectors in your own Postgres. From then on, any agent can retrieve from it through the [rag node](../building/nodes/rag.md): as a fixed pipeline step, or as a tool the model calls on demand. RAG — retrieval-augmented generation — exists because a document collection never fits a model's context window. Instead of pasting documents into prompts, the agent asks a search question, gets back the handful of most relevant passages, and answers from those. Everything stays in your trust domain: the documents, the vectors, and the embedding calls (which go to your inference plane, not a third party). You manage sources on the **RAG** page in the sidebar. ## Before you start: an embedding model Retrieval needs an **embedding model** — a small model that turns text into vectors so that similar meanings land near each other. Chat models can't do this; it is its own modality, served by your inference plane like any other model: - **Local (recommended):** register an embedding GGUF with llama.cpp — for example `nomic-embed-text-v1.5` (768 dimensions, ~84 MB) — with `modality: embeddings`. See [Installing local models](../models/installing.md) and the [model management API](../reference/api.md). - **Hosted:** any `openai-compatible` binding whose upstream serves `/v1/embeddings` works the same way. Two things to know: 1. **The model is pinned per source.** Vectors from different embedding models are not comparable, so each source locks its model at creation. Changing it later means creating a new source (or re-ingesting into one). The source's create form only offers this choice once, on purpose. 2. **It's a logical id, like every model in Ertir.** The source stores the id (say `nomic-embed`), and your inference plane decides what serves it — swap the engine behind the id any time. !!! tip "Pooling" Most embedding GGUFs work out of the box (`mean` pooling). Models that pool on the last token can set `params: {"pooling": "last"}` on their registration. ## Creating a source Click **New source** on the RAG page. Every source has a name, a kind, and its embedding model. ### Upload sources An **upload** source is a bucket you drop files into. Two ways in, same result: - the **Upload files** button (multi-select file picker), or - **drag & drop** files anywhere onto the source's row — it highlights while you hover. Supported formats: PDF, DOCX, PPTX, XLSX, Markdown, plain text, HTML, CSV, JSON, XML, and EPUB, plus source code and config files (Python, TypeScript/JavaScript, Go, Rust, Java, Swift, C/C++, shell, SQL, YAML, TOML, and more), which index as plain text, up to 50 MiB per file. An admin can raise that limit to 1 GiB under **Settings → RAG**. Unsupported files are skipped with a notice, and one bad file never stops the rest of a batch. Text-based PDFs extract well; scanned/image-only PDFs are not OCR'd yet. ### Crawl sources A **crawl** source ingests a website — point it at a docs root and Ertir walks the pages: | Field | What it does | |---|---| | **Root URL** | Where the crawl starts. The crawl stays on the same origin **and under the root's path** — pointing at `https://example.com/docs` never wanders into the rest of the site. A root that names a page (`…/guide/intro.html`) covers its folder (`…/guide/`). If the root itself redirects (to `https://`, or from `example.com` to `www.example.com`), the crawl follows the site to its new address. | | **Max pages** | The crawl budget (default 200): the exact number of pages ingested at most. The crawl stops when it runs out, whatever is left. With `1`, only the root page is ingested. | | **Render JavaScript** | Off by default. Turn it on for script-rendered sites; it uses a headless browser, which needs a one-time `playwright install chromium` on the machine running the control plane. Most docs sites don't need it. | The crawler respects `robots.txt`, fetches politely (a few pages at a time), and strips navigation/boilerplate so only each page's main content is ingested. Only HTML pages are ingested: images and other files are skipped, a link that redirects outside the crawl's scope is dropped, and directory-listing sort links (`?C=N;O=D`) are not followed. Several crawls can run at once without mixing their pages. Click **Crawl** to start (creating the source in the UI starts the first crawl automatically; over the API, call [`POST /rag/sources/{id}:ingest`](../reference/api.md) after creating it) and **Re-crawl** any time the site changes — unchanged pages are detected by content hash and skipped, so a re-crawl only re-embeds what actually changed. ## Watching an ingest Ingestion runs in the background. A progress card appears bottom-right with live counters — pages fetched, documents embedded, chunks stored — and survives navigating away or reloading the page. You can **Cancel** from the card; the source row shows the same status (`ingesting` → `ready`, `failed`, or `cancelled`). Statuses are honest: - **ready** — the source has searchable content. If a few pages failed along the way, the source is still `ready` and the error note says what went wrong. - **failed** — nothing usable was ingested (site unreachable, embedding model down, …), or a restart interrupted the job. - A failed re-ingest **never destroys what you already had**: new content replaces old only after it has embedded successfully, so a transient outage leaves the previous content serving. - The error note always describes the **latest** ingest: once a later ingest succeeds, an earlier failure is cleared. Text is cut into chunks of about 450 tokens (the `rag.chunk_max_tokens` setting). Symbol-heavy text — tables of contents, config listings, code — is budgeted by its words and punctuation, so it gets shorter chunks. If the embedding server still rejects a chunk as too large for it, only that chunk is split until it fits; the rest of the document is unaffected. Expand a source row (click its name) to see every document with its status, chunk count, and any per-document error. ## Testing retrieval Every source row has a **Query** button — an inline search box that runs *exactly* the retrieval a rag node would run, so you can check what an agent would see before wiring anything: - Each match shows its score, the document it came from, its heading path (e.g. `Install > macOS`), and the passage text. - **sim** is the semantic (cosine) similarity when the vector leg matched. Search is **hybrid**: meaning-based similarity is fused with keyword full-text search, so paraphrases *and* exact identifiers both rank. The keyword half prefers passages containing every word of the query; when none does (a word that only appears in a heading, say), it ranks passages by how many of the words they contain. If the results look wrong, that's a signal to fix the source (crawl scope, missing documents) — not the agent. ## Using a source in an agent Drop a **rag** node in the editor and pick the source in its inspector. Two wirings: - **As a capability** — drag the node's violet `use` handle into an llm's `tools` port. The model decides when to search and what to ask; it receives the matching passages (with their source URLs) as a tool result and answers from them. This is the usual shape for a "chat with my docs" agent. - **As a step** — wire it inline with data edges; the in-port value (or a `$in` template) becomes the query and the matches flow downstream. See the [rag node reference](../building/nodes/rag.md) for ports, configuration, and the result shape. Agents reference a source by its stable id, so re-crawling or uploading more documents **never changes a published agent's version** — the agent always searches the source's current content. ## Managing sources - The list has the same **search box and filter chips** (kind, status) as the MCP page. - Click a source's name to see its documents, oldest first, with each one's status, chunk count, and any error. A large source loads 100 at a time; **Load more** fetches the next ones. - **Rename** and crawl-config changes are a `PATCH` away (the API) — the id, and therefore every agent reference, is unaffected. - **Delete** removes the source with all its documents and vectors (a confirmation dialog stands in the way). Agents that still reference it will refuse to run with a clear `rag_source_not_found` error. ## Related pages - [The rag node](../building/nodes/rag.md) — ports, config, capability vs step - [Installing local models](../models/installing.md) — getting an embedding model - [API reference](../reference/api.md) — the `/rag/*` endpoints --- Source: https://docs.ertir.dev/latest/mcp/ # MCP servers An **MCP server** gives your agents tools that live outside Ertir — a filesystem browser, a database client, a company API, a hosted search service. MCP (the Model Context Protocol) is the open standard those servers speak. You set a server up once on the **MCP** page in the sidebar, and from then on any agent can call its tools through the [mcp_tool node](../building/nodes/mcp.md), either at a fixed step in the graph or autonomously, whenever the model decides to. The MCP page covers the whole story: **browse and install** published servers from public hubs, **add your own** four different ways (a local process, a remote endpoint, an OpenAPI spec, a GraphQL endpoint), **sign in** to OAuth-protected servers from the browser, and manage everything in one list. Servers run in **your trust domain** — the same posture as your inference engines. Local servers are subprocesses on your machine, remote servers are reached directly from your control plane, and credentials are stored encrypted on your side. Nothing transits anyone else's servers on the way. ## One server list Every server lands in the same list, whether you defined it by hand or installed it from a hub. Each row shows the server's **name**, a **transport badge** — `stdio` (local subprocess), `http` / `sse` (remote), `openapi` / `graphql` (generated in-process) — a **connected/idle** badge, a lock icon when the server holds an encrypted secret, and, for hub installs, an **origin badge** naming the registry and version it came from. The actions per row: | Action | What it does | |---|---| | **Tools** | Connect if needed and list the tools the server reports, with names and descriptions. | | **Connect** | Only on OAuth servers: run the browser sign-in (see [below](#authorize-with-oauth-sign-in)). | | **Warm** | Start and connect the server now, instead of on first use. | | **Close** | Disconnect. The definition is kept; the server reconnects lazily on the next call. | | **Rotate secret** | On connections with a pasted credential: write a new secret in place (write-only — the old value is never shown). No agent changes. OAuth rows re-authorize with **Connect** instead. | | **Delete** | Remove the server entirely (a confirmation dialog stands in the way). | A search box and transport/status filter chips help when the list grows. Below the list, the **tool tester** runs a single tool through a throwaway one-node graph so you can check a tool works before wiring it into an agent. Agents reference every kind of server the same way: an `mcp_tool` node picks a server (by name) or a connection, and the node's **tool** field is a searchable picker over the tools that server actually exposes. See the [mcp_tool node reference](../building/nodes/mcp.md). ## Installing from a hub **Browse hubs** opens a catalog over MCP registries — public directories where server authors publish how their server is launched or reached. Two are built in: | Hub | What it is | |---|---| | **Official MCP Registry** | The community registry at `registry.modelcontextprotocol.io`. | | **GitHub MCP Registry** | GitHub's registry of MCP servers. | You can add your own with the `THEYGENT_MCP_REGISTRIES` environment variable — a JSON array of `{"id", "label", "url"}` objects. Any registry that speaks the standard subregistry API works, and a registry you host doubles as an **allowlist**: in an air-gapped or policy-controlled deployment, point the browse surface at your own mirror instead of the public hubs. A malformed value is a loud startup error, never a silently dropped registry. Pick a hub, search, and read the entries: each shows its description, version, transport and package badges, and a star count where the publisher provides one. A **deprecated** entry carries a visible warning with the publisher's message — prefer whatever replacement it names. !!! note "Cold searches can take a while" The official registry's substring search can take fifteen seconds or more on a cold query. Ertir waits generously rather than timing out — a slow first search is the hub warming up, not the hub being down. Repeat searches are cached and fast. ### Installing an entry Expanding an entry shows every launchable form it publishes: a **package** spawned locally over stdio (launched with `npx`, `uvx`, `docker`, or `dnx`, depending on the package type), or a **remote** reached over streamable-HTTP or SSE. Pick one and press **Install**: - The form is generated from the server's **own declared inputs** — required fields are marked, declared choices become dropdowns, and secret-marked values are entered as passwords. - The **connection name** is how agents will reference the server. - **Secrets are stored encrypted server-side and never shown again.** A hub install with secrets is exactly a [connection](../building/nodes/tools.md#authentication-with-connections-and-secrets) with an encrypted secret — rotating the value later is a connection edit, and no agent version changes. - A remote entry that supports OAuth offers **"Sign in with the provider instead"** — install first, then press **Connect** on the new row to authorize in the browser. The installed server appears in the list with its origin badge, and the catalog marks the entry as installed so you don't install it twice. ## Adding a server by hand **Add server** offers four kinds. All four create a connection: the non-secret settings are stored plainly, anything secret is encrypted and write-only. ### Stdio — a local subprocess A server that runs as a process on your machine, in your trust domain. | Field | Notes | |---|---| | **Name** | What agents reference, e.g. `filesystem`. | | **Command** | The executable to launch, e.g. `npx`. | | **Args** | Whitespace-separated arguments. | | **Env** | Non-secret `NAME=value` lines, passed into the subprocess. | | **Secret env** | Secret `NAME=value` lines — stored encrypted, injected into the subprocess at launch, never shown again. | | **Working dir** | Optional; the directory the process starts in. | ### Remote — an HTTP or SSE endpoint A server someone else hosts, reached over streamable-HTTP or SSE. Give it a name, the URL, any non-secret headers, and an auth type: | Auth | You supply | What Ertir sends | |---|---|---| | **None** | — | plain requests | | **Bearer token** | the token | `Authorization: Bearer ` | | **API key header** | a header name + the key | the key in that header | | **Basic** | username + password | `Authorization: Basic base64(user:password)` | | **Header map** | whole headers, one `NAME=value` per line | every header verbatim | | **OAuth2 client credentials** | token URL, client id, optional scope, client secret | fetches a token, sends it as a bearer | | **OAuth sign-in** | nothing to paste | you authorize in the browser after creating — see [below](#authorize-with-oauth-sign-in) | Whatever the type, the credential itself is stored encrypted and turned into request headers server-side — it never appears in the stored config or in any agent document. ### OpenAPI — generate tools from a spec If you already have a REST API, you don't have to write an MCP server for it. Paste an OpenAPI spec (JSON) or give its URL, set the base URL and upstream auth, and **every operation in the spec becomes a callable tool**, named by its `operationId`. The server is hosted in-process — no subprocess, no socket; only the upstream API calls leave, carrying the auth that was resolved server-side. Press **Preview** first: it shows exactly the tool list an agent would see, and **Create** is enabled only once the spec derives cleanly. !!! warning "A relative `servers` entry needs an explicit base URL" Many specs declare a relative server URL (say `/api/v3`), which only means something relative to wherever the spec lives. When you give the spec by URL, Ertir resolves the relative path against the spec's own origin. A **pasted** spec has no origin — give an explicit **Base URL** or the derived tools will have nowhere to call. ### GraphQL — generate tools from an endpoint Give a GraphQL endpoint and its auth, and the server derives exactly **two tools**: `introspect_schema` (returns the schema) and `run_query` (runs a query the model writes). Queries are parsed and validated locally against the introspected schema before anything goes upstream; mutations are rejected unless you turn on **Allow mutations**, and subscriptions always are. Two generic tools rather than one per field is deliberate — the model reads the schema and writes real queries, including nested selections a per-field tool couldn't express. **Preview** confirms the endpoint introspects before you create. ## Authorize with OAuth (sign in) Many hosted MCP servers use OAuth: there is no token to paste, because the server wants *you* to sign in and consent. Ertir handles this without any manual configuration: 1. Create the connection with the **OAuth sign-in** auth type (Remote form), or check **"Sign in with the provider instead"** when installing from a hub. 2. Press **Connect** on the server's row. A browser tab opens the provider's consent page. 3. Approve. The tab tells you it's done; back on the MCP page, the row shows connected. Ertir stores the tokens **encrypted** in your own database and refreshes them automatically — you should only see the consent page again if the provider revokes access or the grant fully expires. Authorizing (or re-authorizing) never changes any agent's version: the tokens live behind the connection's stable secret reference. Unattended runs never hang waiting for a login. If a server needs re-authorization during a triggered or scheduled run, the tool call **fails immediately** with a message telling you to reconnect the server on the MCP page — the node binds its `err` port and the run carries on, per the normal [tool error contract](../building/nodes/tools.md#the-success-error-contract). A browser flow only ever starts because you pressed Connect. ## Using a server in an agent Drop a **Tool** node in the [editor](../building/editor.md), choose **MCP** in the Kind picker, and pick the server — a name-keyed server or a connection, whichever the list shows. The **tool** field is a searchable dropdown over the tools that server actually reports (it degrades to free text if the server can't be reached, so authoring is never blocked). Wire the node as a fixed **step**, or wire its violet `use` handle into an llm's `tools` port to make it a **capability** the model calls on its own — see the [mcp_tool node reference](../building/nodes/mcp.md) for ports, config, and the error contract. ## Troubleshooting - **A hub entry is marked deprecated.** The publisher has retired it. The warning shows their message, which usually names a successor — install that instead. An already-installed deprecated server keeps working, but don't build new agents on it. - **An OpenAPI server derives tools but every call fails.** Almost always a relative `servers` entry with no base URL — edit the connection (or recreate it) with an explicit base URL. Preview shows the upstream URL it will use; check it's absolute. - **The hub search feels stuck.** Cold queries against the official registry can take ~15 seconds. Wait it out once; the page is cached after that. A genuinely unreachable registry comes back as a clear error, not a spinner. - **A run reports "authorize this connection".** The server's OAuth grant needs renewing — open the MCP page and press **Connect** on its row. Runs never open a browser themselves. - **You need to rotate a secret from a hub install.** It's an ordinary connection: press **Rotate secret** on its row and write the new value. The old one is never displayed (write-only), and rotating changes no agent version. ## Related pages - [Example agents](examples.md) — one runnable agent per server kind, plus capability mode - [Apify through the MCP hub](apify.md) — hosted scrapers and crawlers as tools, and when a crawler node fits better - [The mcp_tool node](../building/nodes/mcp.md) — ports, config, step vs capability - [Tool nodes](../building/nodes/tools.md) — connections and the tool error contract - [API reference](../reference/api.md) — the `/admin/mcp/*` and `/connections/{id}/mcp*` endpoints - [Environment variables](../reference/environment.md) — `THEYGENT_MCP_REGISTRIES`, `THEYGENT_MCP_CALL_TIMEOUT_S` --- Source: https://docs.ertir.dev/latest/mcp/examples/ # Example agents One complete, runnable agent per kind of MCP server — a local **stdio** subprocess, a **remote** HTTP server, a server **generated from an OpenAPI spec**, a server **generated from a GraphQL endpoint**, and a bonus example where the **model calls the tool on its own** (capability mode). Every example was run for real against public services before it landed here. Each example is the body for [`POST /graphs/runs`](../reference/api.md) — the same graph you can build visually in the [editor](../building/editor.md) (a Tool node with Kind = MCP, wired between input and output). Run one with: ```bash curl -X POST http://localhost:8080/graphs/runs \ -H 'Content-Type: application/json' \ --data-binary @agent.json ``` Every example references its server by **connection id** (`con_…`). Create the server on the MCP page first (each example says how), then copy the id from the connection — the API returns it on creation, and `GET /connections` lists it. !!! note "Why the id and not the name?" Agents reference a connection by its stable id, so renaming the connection — or rotating its secret — never changes any agent. The id is the one thing you substitute in these examples. ## Stdio — read a file through a local subprocess **Setup:** *Add server → Stdio* with command `npx` and args `-y @modelcontextprotocol/server-filesystem /path/you/allow`. The server is a subprocess on your machine; it can only see the directory you pass. The agent takes a file path as input and returns the file's contents. `$in.in` is the run input (the [input reference grammar](../building/input-references.md)). ```json { "ir": { "schemaVersion": "1.0", "id": "agt_mcp_stdio_files", "name": "mcp-stdio-file-reader", "version": "0.1.0", "models": {}, "tools": {}, "nodes": [ {"id": "n_in", "type": "input", "kind": "boundary", "config": {}, "ports": {"in": [], "out": [{"id": "out"}]}}, {"id": "n_tool", "type": "mcp_tool", "kind": "activity", "config": {"connection": "", "tool": "read_text_file", "args": {"path": "$in.in"}}, "ports": {"in": [{"id": "in"}], "out": [{"id": "out"}, {"id": "err", "type": "error"}]}}, {"id": "n_out", "type": "output", "kind": "boundary", "config": {}, "ports": {"in": [{"id": "in"}], "out": []}} ], "edges": [ {"id": "e1", "source": "n_in", "sourceHandle": "out", "target": "n_tool", "targetHandle": "in"}, {"id": "e2", "source": "n_tool", "sourceHandle": "out", "target": "n_out", "targetHandle": "in"} ] }, "input": "/path/you/allow/notes.txt", "stream": false } ``` ## Remote — ask a hosted server over streamable-HTTP **Setup:** *Add server → Remote* with URL `https://mcp.deepwiki.com/mcp` and auth **None**. DeepWiki is a free public MCP server that answers questions about GitHub repositories — a good zero-setup way to try a remote server. (A server that needs auth is the same recipe plus an auth type; an OAuth server additionally gets a **Connect** press.) Input is a repository name; the agent returns the documentation topics DeepWiki knows for it. Same graph as above — only the connection and tool change: ```json {"id": "n_tool", "type": "mcp_tool", "kind": "activity", "config": {"connection": "", "tool": "read_wiki_structure", "args": {"repoName": "$in.in"}}, "ports": {"in": [{"id": "in"}], "out": [{"id": "out"}, {"id": "err", "type": "error"}]}} ``` with `"input": "modelcontextprotocol/python-sdk"`. ## OpenAPI — current weather from a generated server **Setup:** *Add server → OpenAPI*. Spec URL `https://raw.githubusercontent.com/open-meteo/open-meteo/main/openapi/forecast.yml`, base URL `https://api.open-meteo.com`, auth **None**, then **Preview** → **Create**. The spec's one operation carries no `operationId`, so the derived tool is named from its summary: `Open_Meteo_Weather_Forecast_API`. The input is an object, drilled per field — `$in.in.latitude` reads the `latitude` key of the run input. The `current` argument is a plain literal (anything that isn't a `$in` reference passes through unchanged): ```json {"id": "n_tool", "type": "mcp_tool", "kind": "activity", "config": {"connection": "", "tool": "Open_Meteo_Weather_Forecast_API", "args": {"latitude": "$in.in.latitude", "longitude": "$in.in.longitude", "current": ["temperature_2m", "wind_speed_10m"]}}, "ports": {"in": [{"id": "in"}], "out": [{"id": "out"}, {"id": "err", "type": "error"}]}} ``` with `"input": {"latitude": 52.52, "longitude": 13.405}` — the current weather in Berlin comes back as JSON. ## GraphQL — a fixed query with variables from the input **Setup:** *Add server → GraphQL* with endpoint `https://countries.trevorblades.com/graphql` (a free public API) and auth **None**. The generated server exposes `introspect_schema` and `run_query`. Argument values are **whole references or literals** — there is no string interpolation inside `args`. So the query text is a literal, and the whole `variables` object arrives as the run input via one reference: ```json {"id": "n_tool", "type": "mcp_tool", "kind": "activity", "config": {"connection": "", "tool": "run_query", "args": {"query": "query($code: ID!) { country(code: $code) { name capital currency } }", "variables": "$in.in"}}, "ports": {"in": [{"id": "in"}], "out": [{"id": "out"}, {"id": "err", "type": "error"}]}} ``` with `"input": {"code": "DE"}` → `{"country": {"name": "Germany", "capital": "Berlin", "currency": "EUR"}}`. ## Capability mode — the model calls the tool itself The step-mode examples above run the tool at a fixed place in the graph. In **capability mode** the tool hangs off the llm's `tools` port instead, and the model decides when to call it and with what arguments — here it turns "Paris" into coordinates on its own. Reuse the OpenAPI weather connection from above; `default` must be a chat model registered on your inference plane. ```json { "ir": { "schemaVersion": "1.0", "id": "agt_mcp_capability_weather", "name": "mcp-capability-weather-assistant", "version": "0.1.0", "models": {"default": {"binding": "mlx", "model": "", "params": {"temperature": 0.0, "maxTokens": 2048}}}, "tools": {}, "nodes": [ {"id": "n_in", "type": "input", "kind": "boundary", "config": {}, "ports": {"in": [], "out": [{"id": "out", "type": "any"}]}}, {"id": "n_llm", "type": "llm", "kind": "activity", "config": {"model": "default", "messages": [{"role": "user", "content": "$in"}], "maxToolIterations": 4}, "ports": {"in": [{"id": "in", "type": "any"}, {"id": "tools", "type": "any", "required": false, "role": "tool"}], "out": [{"id": "ok", "type": "any"}, {"id": "err", "type": "error"}]}}, {"id": "n_weather", "type": "mcp_tool", "kind": "activity", "config": {"connection": "", "tool": "Open_Meteo_Weather_Forecast_API", "description": "Get current weather for a latitude/longitude. Pass current=[\"temperature_2m\",\"wind_speed_10m\"]."}, "ports": {"in": [{"id": "in", "type": "any", "required": false}], "out": [{"id": "use", "type": "any", "role": "tool"}]}}, {"id": "n_out", "type": "output", "kind": "boundary", "config": {}, "ports": {"in": [{"id": "in", "type": "any"}], "out": []}} ], "edges": [ {"id": "e1", "source": "n_in", "sourceHandle": "out", "target": "n_llm", "targetHandle": "in", "channel": "data"}, {"id": "e2", "source": "n_llm", "sourceHandle": "ok", "target": "n_out", "targetHandle": "in", "channel": "data"}, {"id": "e_cap", "source": "n_weather", "sourceHandle": "use", "target": "n_llm", "targetHandle": "tools", "channel": "tool"} ] }, "input": "What is the current temperature and wind speed in Paris? Use the weather tool.", "stream": false } ``` In the editor this is the violet `use` handle wired into the llm's `tools` port. Note the capability node's `in` port is declared `"required": false` — nothing feeds it; the model supplies the arguments. ## Related pages - [MCP servers](index.md) — setting up every kind of server - [The mcp_tool node](../building/nodes/mcp.md) — ports, config, step vs capability - [Input references](../building/input-references.md) — the `$in` grammar used in `args` --- Source: https://docs.ertir.dev/latest/mcp/apify/ # Apify through the MCP hub [Apify](https://apify.com/store) runs a store of hosted scrapers, crawlers and automations, which it calls **Actors**: web search that returns pages as Markdown, Google Maps places, website crawls, social media profiles and thousands more. Apify publishes an MCP server, so you can give an agent Actors as tools today with the [MCP hub](index.md) alone. This page covers that path. For agents on smaller local models, or whenever you want to fix what each call may cost, the [crawler node](../building/nodes/crawler.md) is the better fit — see [When to use a crawler node instead](#when-to-use-a-crawler-node-instead). ## What you need - An **Apify account** and its **API token** (Apify Console → Settings → API & Integrations). Actor runs are billed to the account that owns the token. Apify's free plan includes a monthly usage allowance. - A **model that calls tools reliably**. Actor tools carry every input field the Actor accepts, which is a lot to read: see [the costs](#what-it-costs-in-context-and-in-money). ## Install the server The server is `com.apify/apify-mcp-server` in the Official MCP Registry. It is a remote server at `https://mcp.apify.com/`, reached over streamable HTTP. 1. Open the **MCP** page and press **Browse hubs**. 2. Pick **Official MCP Registry** and search for `apify`. 3. Expand `com.apify/apify-mcp-server` and pick its **Remote server (streamable-http)** form. 4. Authenticate in one of two ways: - **Token.** Fill the `Authorization` field with `Bearer ` followed by your API token, for example `Bearer apify_api_…`. The value is stored encrypted and never shown again. - **Sign in.** Check **Sign in with the provider instead**, install, then press **Connect** on the new row and approve access on Apify's page. See [Authorize with OAuth](index.md#authorize-with-oauth-sign-in). 5. Give the connection a name, such as `apify`, and press **Install**. You can also add it by hand: **Add server** → **Remote**, URL `https://mcp.apify.com/`, auth **Bearer token** with your API token. ## Choose which Actors become tools The server's URL takes a `tools` parameter: a comma-separated list of Actor names, tool categories, or single tool names. Each Actor you list becomes one tool. ```text https://mcp.apify.com/?tools=apify/rag-web-browser,compass/crawler-google-places ``` To change the list, open the server's **Settings** on the MCP page and edit its **URL**, then **Close** the server so it reconnects with the new list. Without `tools`, the server serves its defaults. They include the generic `search-actors`, `fetch-actor-details` and `call-actor` tools, documentation search, and a couple of web-browsing Actors. Apify notes that the defaults may change, so list your tools explicitly for agents you depend on. `call-actor` lets the model run *any* Actor it finds — leave it out unless that is what you want. !!! note "Browsing works without a token" With no token, the server answers only its discovery tools — `search-actors`, `fetch-actor-details` and the documentation tools — and only when `tools` lists nothing else. That is enough to explore the store from an agent. Running an Actor always needs a token. ## Wire Actors into an agent Each Actor tool is an ordinary MCP tool: 1. Select an llm node, and in **Tools the model can call** press **Add MCP tools**. 2. Pick the Apify connection, tick the Actor tools, and press **Add**. Each lands as an `mcp_tool` node wired into the llm's `tools` port. 3. Give the llm a prompt that says when to search, and run it. The model decides when to call an Actor and fills its arguments. The run's tool result is what Apify's server returns for the call. See the [mcp_tool node](../building/nodes/mcp.md) for ports, step mode and the error contract. ## What it costs, in context and in money An Actor tool offers the model **every input field** the Actor declares — proxy settings, result counts, paid add-ons, tuning knobs. That has three consequences worth knowing before you rely on this path: - **It takes a lot of context.** Three popular Actors offered this way add about 13,500 tokens to every model call. A model with a small context window has little room left for the conversation, and the first call on a local engine is slow until the engine has cached the tool definitions. - **The model sets the spend.** Pay-per-event Actors charge per result and per add-on. A model offered every field can ask for 50 results, or switch on a paid add-on, where 5 results and none would have done. Apify bills the token's account for whatever the model asks for. - **Results come back whole.** A crawl can return far more text than a local model's context holds, and the tool result is passed to the model as it is. On Apple Silicon with MLX there is one more: some models write a list argument (a list of search terms or URLs) as a bare string, and the engine drops the request instead of answering. Actors with list inputs are the ones that trigger it. ## When to use a crawler node instead A [crawler node](../building/nodes/crawler.md) runs the same Actors without the MCP server, and moves each of those decisions to whoever builds the agent: | | MCP hub | Crawler node | |---|---|---| | What the model sees | every input field | only the fields you expose, usually one or two | | Result counts, add-ons, proxies | the model chooses | fixed on the node | | Spend per call | whatever the model asks for | capped by the node and by a platform-wide ceiling | | Result size | whole | trimmed to a character budget you set | | List inputs on MLX | can drop the request | offered as a single value and wrapped for the Actor | | Long crawls | limited by the server's call timeout | any length on a [durable run](../running/durable.md) | Use the MCP hub when a capable model should explore the store and pick Actors on its own. Use a crawler node when you know which Actor the agent needs. ## Privacy The model's tool arguments — search terms, URLs, place names — go to Apify, and the Actor's results come back into your run. Apify's servers do the crawling. Your token stays encrypted in the control plane's secret store and is sent only to the URL you configured. See [Privacy and data flow](../concepts/privacy.md). ## Related pages - [MCP servers](index.md) — the hub, connections and OAuth sign-in - [The mcp_tool node](../building/nodes/mcp.md) — step and capability wiring - [The crawler node](../building/nodes/crawler.md) — Actors as nodes with fixed inputs and spend caps --- Source: https://docs.ertir.dev/latest/coding-agents/ # Use Ertir from Coding Agents Ertir ships an **MCP server** that hands your coding agent the platform as tools. Add it to Claude Code, Codex, OpenCode or any other MCP client, and the coding agent can: - **Offload work to your local models.** Summarize a long file, draft a commit message or release notes, classify or extract data, or get a second opinion. The work runs on the model you pick. With a local MLX or llama.cpp model, the content stays on your machine and costs nothing per token. - **Run your Ertir agents.** Anything you have built and published, such as a triage router, a PII firewall or a review loop, becomes one tool call away. - **Build new agents for you.** Describe a repeatable job and the coding agent writes the agent, checks it, test-runs it, and publishes it. It then appears on your **Agents** page like any agent you built in the editor. - **Search your documents and code.** The coding agent can index a project's docs, specs or source into a [retrieval source](../rag/index.md), then search it with the same hybrid retrieval your agents use. !!! note "Two directions of MCP" The [MCP](../mcp/index.md) section is about Ertir *using* MCP servers inside agents. This page is the other direction: **Ertir as an MCP server** that other tools use. ## Before you start - **Ertir is running.** Use `make up` or Docker Compose (see [Running Ertir](../getting-started/running.md)). The MCP server talks to the control plane on `:8080` and the inference plane on `:8081`. - **At least one chat model is registered** (see [Installing local models](../models/installing.md)). Add an embeddings model too if you want the coding agent to index and search documents. - **`uv` is on your `PATH`.** The coding tool starts the MCP server from your Ertir checkout with `uv run`. - **A personal API key.** In Ertir, open the profile menu at the bottom of the sidebar and choose **API keys → Create**. Copy the `tyk_…` key; it is shown only once. The key acts with your role, and creating agents needs **editor** or above (see [Users, roles & sign-in](../running/users.md#personal-api-keys)). ## Print your setup commands From your Ertir checkout: ```bash make mcp-setup ``` It prints a ready-to-paste setup for each tool below, with your checkout's path already filled in. Replace `` with your key. ## Claude Code ```bash claude mcp add theygent --scope user \ -e 'THEYGENT_API_KEY=tyk_…your key' \ -- uv run --project /path/to/Ertir --package theygent-mcp theygent-mcp ``` `--scope user` makes Ertir available in every project. Leave it out to add it only to the current project. Check the connection with `claude mcp list`, or run `/mcp` inside a session. Then ask in plain words, for example *"use Ertir's local model to summarize this log"*. Claude picks the tools itself. Agent runs on a local model can take a few minutes. If a long run is cut off, raise Claude Code's MCP tool timeout, for example `export MCP_TOOL_TIMEOUT=600000` (milliseconds) before starting `claude`. ## Codex ```bash codex mcp add theygent --env 'THEYGENT_API_KEY=tyk_…your key' \ -- uv run --project /path/to/Ertir --package theygent-mcp theygent-mcp ``` Codex stops a tool call after 60 seconds by default, which is too short for agent runs on a local model. Give it room in `~/.codex/config.toml`: ```toml [mcp_servers.theygent] command = "uv" args = ["run", "--project", "/path/to/Ertir", "--package", "theygent-mcp", "theygent-mcp"] tool_timeout_sec = 600 env = { THEYGENT_API_KEY = "tyk_…your key" } ``` ## OpenCode Add Ertir to `opencode.json`, either in the project or in `~/.config/opencode/opencode.json`: ```json { "$schema": "https://opencode.ai/config.json", "mcp": { "theygent": { "type": "local", "command": ["uv", "run", "--project", "/path/to/Ertir", "--package", "theygent-mcp", "theygent-mcp"], "enabled": true, "environment": { "THEYGENT_API_KEY": "tyk_…your key" } } } } ``` Check it with `opencode mcp list`. It should show `theygent` as connected. ## Other MCP clients Any client that can launch a stdio MCP server works with the same command and environment. For a client that only speaks HTTP, serve streamable HTTP instead: ```bash THEYGENT_API_KEY='tyk_…your key' \ uv run --package theygent-mcp theygent-mcp --http --port 8083 ``` Then point the client at `http://127.0.0.1:8083/mcp`. The HTTP mode binds to `127.0.0.1` by default. Anyone who can reach that port acts with your API key, so don't expose it beyond your machine. ## Settings The MCP server reads its settings from the environment the coding tool starts it with: | Variable | Default | What it does | |---|---|---| | `THEYGENT_API_KEY` | — | Your personal API key. Required by every agent, run, sample and retrieval tool. The model tools work without it. | | `THEYGENT_CONTROL_PLANE_URL` | `http://127.0.0.1:8080` | Where the control plane is. | | `THEYGENT_INFERENCE_PLANE_URL` | `http://127.0.0.1:8081/v1` | The inference plane's OpenAI-compatible root, ending in `/v1`. | | `THEYGENT_MCP_DEFAULT_MODEL` | first chat model | The model `ask_model` uses when the coding agent doesn't name one. | | `THEYGENT_MCP_RUN_TIMEOUT_S` | `600` | How long agent runs and model calls may take, in seconds. | Pass extra variables the same way as the key: `-e` for Claude Code, `--env` for Codex, or `environment` for OpenCode. ## What stays where Model calls (`ask_model`) go straight from the MCP server to the inference plane on your machine. Prompts never pass through the control plane. Files given to `ask_model` as `files` are read by the MCP server and sent only to the model you chose. The coding agent and its provider see the answer, not the file. Agent runs and retrieval searches go through the control plane, as they do from the interface, and show up on the **Runs** page with their full trace. The coding tool sees only what the tools return. See [Privacy & data flow](../concepts/privacy.md). ## Next - [Tools & recipes](tools.md) lists every tool, gives prompts that work well, and has a snippet that teaches your coding agent when to reach for Ertir. --- Source: https://docs.ertir.dev/latest/coding-agents/tools/ # Tools & Recipes Once [Ertir is connected](index.md), your coding agent sees the tools below. You rarely name them. Ask for the outcome and the agent picks the tools itself. Wherever a tool takes an agent or a retrieval source, it accepts the id or the exact name you gave it. ## Models | Tool | What it does | |---|---| | `list_models` | The models registered in Ertir: logical id, engine, modality (chat, vision, embeddings…), and which chat model `ask_model` uses by default. | | `ask_model` | Sends one prompt to a model and returns its answer. `files` attaches local text files: the MCP server reads them and sends them only to the model, so a long log or file never passes through the coding agent's own context or its provider. Only the answer comes back. Up to 256 KiB of files per call. `think: false` turns a reasoning model's thinking off for a faster answer. | ## Agents and runs | Tool | What it does | |---|---| | `list_agents` | Your published agents, newest first. | | `get_agent` | One agent's versions and full agent document. | | `run_agent` | Runs a published agent, waits for the result, and returns its output and run id. Pin a `version`, or continue a chat `session_id`. | | `run_graph` | Runs an agent document **without publishing it**. This is how the coding agent tests an agent it is building. | `run_agent` and `run_graph` start ordinary runs, not [durable runs](../running/durable.md). An agent that needs a human approval, a subgraph, a loop or a map runs durably, so start it from the interface or the API instead. | `get_run` | A run's status, output or error, by run id, including runs started elsewhere. | ## Building agents | Tool | What it does | |---|---| | `agent_authoring_guide` | How to write an agent: the document format, how values flow, patterns, a full example, and every node type with its ports and defaults. Pass `node_types` to get the full config schema of specific types. | | `validate_agent` | Checks a document against the same rules publishing applies, without running it, and names each problem. | | `create_agent` | Publishes a new agent. It appears on your **Agents** page and can be opened in the editor. | | `publish_agent_version` | Publishes a new version of an existing agent. Earlier versions stay runnable. | | `list_samples` / `install_samples` | The ready-made [sample agents](../samples/index.md), and installing them with your models in their slots. | Agents the coding agent builds are ordinary Ertir agents. They follow [versioning](../concepts/versioning.md), show up in the editor, can get [triggers](../running/triggers.md), and their runs are traced on the **Runs** page. ## Retrieval | Tool | What it does | |---|---| | `list_rag_sources` | Your [retrieval sources](../rag/index.md) and their status. | | `create_rag_source` | Creates an `upload` source, or a `crawl` source for a website. Needs an embeddings model from `list_models`. | | `add_rag_files` | Indexes local files, whole directories, or glob patterns such as `docs/**/*.md`. Relative paths resolve against the project the coding tool runs in. Hidden, dependency, and build directories are skipped. Each file is named by its path in the project, so two `index.ts` files stay two documents, and adding a file again replaces its earlier copy. One call indexes up to 500 files. Files stream from disk, and a file over Ertir's upload limit (50 MiB by default, raised under **Settings → RAG**) is skipped with the limit in its result. | | `add_rag_text` | Adds a document from text, such as notes, a generated summary, or an API response. | | `ingest_rag_source` | Starts or re-runs a crawl source's crawl. | | `get_rag_source` | Ingest progress and each document's status. | | `search_rag` | Hybrid semantic and keyword search over a source. Returns the best passages with their documents and scores. | Source code and config files index as plain text, next to the document formats listed on the [RAG page](../rag/index.md#upload-sources). ## Recipes Prompts that work well with Claude Code, Codex or OpenCode once Ertir is connected: **Offload a summary to a local model** > Have Ertir's local model read `logs/build.log` and summarize the failures in five bullets. > Pass the log as a file, don't read it yourself. Use think=false. **Draft with a private model** > Write the release notes for the commits since the last tag. Draft them with Ertir's local > model, then tidy the draft yourself. **Index the docs, then answer from them** > Create an Ertir retrieval source called `project-docs` with the `local-embed` model and add > `docs/` and `README.md` to it. When it's ready, search it for how authentication is configured. **Build a reusable agent** > Build me an Ertir agent called `commit-titler` that turns a change description into a > conventional-commit title using my local model. Test it on two examples, then publish it. **Use an agent you already have** > Run my Ertir agent `pii-firewall` on `fixtures/customers.csv` before you paste any of it here. ## Teach your coding agent when to use Ertir The MCP server tells the coding agent what it offers. A line in your project instructions turns that into a habit. Add something like this to `CLAUDE.md` (Claude Code) or `AGENTS.md` (Codex, OpenCode): ```markdown ## Ertir Ertir's MCP tools are available. For summaries, first drafts and extraction over large or private files, call `ask_model` with the paths in `files` (think=false for quick tasks) instead of reading the files yourself. Search the `project-docs` retrieval source before guessing how something works here. Check `list_agents` for an existing agent before writing a one-off script for a repeatable task. ``` ## When something goes wrong Every tool error says what to do next: | Message says | Fix | |---|---| | `THEYGENT_API_KEY is not set` | Add your key to the server's environment, then restart the coding tool. | | `rejected the API key (401)` | The key was revoked or mistyped. Create a new one under **API keys**. | | `refused this action (403)` | Your role can't do this. Publishing agents needs **editor** or above. | | `is not reachable at …` | Ertir isn't running, or isn't at that address. Start it with `make up`, or set the plane URL variables. | | `No chat model is registered` | Install a model from **Registries → Add model**. | | `did not answer in time` | A run took longer than `THEYGENT_MCP_RUN_TIMEOUT_S`. Raise it, and the coding tool's own tool timeout. | Errors from Ertir itself pass through with their code, for example `agent_exists` when publishing an id that is taken (use `publish_agent_version`), or `unsupported_document` for a file type retrieval can't read. --- Source: https://docs.ertir.dev/latest/running/ # Runs A **run** is one execution of an agent or a model — the record of a single request from start to finish. Every chat message, every agent invocation, and every trigger fire creates a run. The **Runs** page is where you find them all, watch them stream live, read their output, and diagnose failures. For the conceptual model of runs and how they relate to sessions and memory, see [Runs & sessions](../concepts/runs-and-sessions.md). This page is the operational reference: the Runs page, the run lifecycle, the rules that decide a run's output, and what happens when a run is cancelled or fails. ## Where runs come from Runs are never created by hand — they are the by-product of doing something: - **Chats** — every turn in [New Chat](../chat/index.md) or an agent's Chat panel is a run. - **Agent invocations** — running a published agent from the [Bench](bench.md), or over the API. - **Triggers** — a [webhook, cron schedule, or token invoke](triggers.md) firing a published agent unattended. ## The Runs page Open **Runs** from the left sidebar. It lists every run, **newest first**, and **auto-refreshes every 3 seconds** (the refresh pauses while the browser tab is hidden). The list loads older runs as you scroll — there is no "load more" button. Each row shows six columns: | Column | What it shows | |---|---| | Run | The run id (shown in full) | | Status | A colour-coded status chip | | Model | The logical model id, for a plain model run | | Graph | For a saved-agent run, `@` | | Session | The [session](../concepts/runs-and-sessions.md) the run belongs to, if any | | Created | When the run started (hover for the exact time) | The status chips double as **filters**: click `completed` to show only completed runs, click it again to clear. The search box filters by run id, model, graph, or session. If you have no runs yet, the page reads *"Runs come from chats, agent invocations, and triggers — start a chat."* ## Run statuses and lifecycle A run moves through a small set of statuses: | Status | Meaning | |---|---| | `created` | The run row exists; work is about to begin. | | `streaming` | The run is executing — tokens or step results are flowing. | | `waiting` | A [durable run](durable.md) is paused at a [human approval](human-in-the-loop.md) node, awaiting input. | | `completed` | The run finished and its output is persisted. | | `failed` | The run ended with an error. | ```mermaid stateDiagram-v2 [*] --> created created --> streaming created --> failed streaming --> waiting waiting --> streaming streaming --> completed streaming --> failed completed --> [*] failed --> [*] ``` The `waiting` state only appears in durable mode — it is how a run pauses for a person to approve or supply input, and it can wait indefinitely, even across a restart. When you resume it, the run flips back to `streaming`. See [Human-in-the-loop](human-in-the-loop.md). ## The run detail page Click any run to open its detail. The header shows a breadcrumb back to the list plus the current status. Below it: - A **metadata card** — Model, Graph (`id @ version`), Session (a link), Created, Updated, and the agent's **Content hash** (the immutable fingerprint of the exact graph version that ran — see [Agent versioning](../concepts/versioning.md)). - The **output** — while the run is live it streams with a pulsing cursor and a *"streaming…"* label; once finished, the page shows the persisted output. A reasoning model's thinking renders in a collapsible **Thinking** block above the answer. - A **Trace** — a zoomable waterfall of every step, its timing, and (subject to your capture policy) its inputs and outputs. This is covered in full on [Observability](observability.md). - For a **waiting** run, a **Resume** panel appears so you can supply the awaited input — see [Human-in-the-loop](human-in-the-loop.md). !!! note "Runs are permanent" Runs cannot be deleted — they are your execution history. Deleting a session [detaches](../concepts/runs-and-sessions.md) its runs (they lose the session link) rather than removing them. ## Run output rules The output you see on a completed run is decided by a few firm rules: - **The output is the value of the graph's `output` node**, coerced to a string (a non-string value is JSON-serialised). For a plain model run it is simply the model's answer. See the [input & output nodes](../building/nodes/input-output.md). - **At most one `output` node may execute per run.** Multiple `output` nodes in a graph are allowed, but if two of them go live in the same run, the run fails loudly (*"a second output node executed — the run output would be ambiguous"*) rather than silently picking one. Route exclusive branches with a [router](../building/nodes/router.md) or [guardrail](../building/nodes/guardrail.md) so only one output node is reachable. - **An `output` (or `router`) node consumes exactly one input port.** These are single-value consumers, so if such a node is wired with more than one input port the run fails loudly at execution time (`template_error`) — the run reports *which* value would be ambiguous rather than silently picking one. Wire exactly one input port, or compose multiple upstreams in a value-producing node (LLM/tool/MCP tool) instead. ### Honest empty output Sometimes a run completes but produces no answer. Rather than reporting a green success with a blank result, the run stays `completed` but carries an explanatory **note** in its `error` field, which the interface styles amber (not red). You will see reasons such as: - *"hit the token limit (finish_reason=length) … raise maxTokens"* — the model spent its whole budget (often a reasoning model thinking); raise `maxTokens` on the [LLM node](../building/nodes/llm.md). - *"output empty: upstream error on node 'x': …"* — a node upstream of the output errored and its success path was skipped. - *"the graph declares no output node"* / *"no output node executed this run"* — the taken branch reached no output node. ## Cancelling a run - In a chat, the **Send** button becomes **Stop generating** while a run streams. In the [Bench](bench.md), a separate **Stop** button appears next to the **Run** / **Run durably** button while a run streams. Either way, clicking it aborts the run and keeps the partial answer, marked *"stopped"*. - **Leaving a chat mid-stream** aborts the request; the server cancels the run on disconnect and terminalises it to `failed` with *"interrupted: client disconnected mid-stream"*. ## Errors Errors are reported honestly and never as a broken success: - **Before streaming starts**, a problem surfaces as a clean HTTP error with a machine-readable code — you never get a `200` followed by a broken stream. - **Mid-stream**, a failure emits a final status frame marking the run `failed`, then closes the stream. - **Inference errors are relayed faithfully.** If the model can't be found you get `model_not_found`; if the engine is busy or down, `engine_unavailable`; if the inference plane is unreachable at all, `inference_unreachable`. - **Graph-shaped errors** carry a specific code: `router_error` (a [router](../building/nodes/router.md) selected a handle that doesn't exist), `template_error` (an unresolved [input reference](../building/input-references.md)), or `transform_error` (a bad [transform](../building/nodes/transform.md) expression). !!! note "A tool error is not a run failure" When a [tool](../building/nodes/tools.md) or [MCP tool](../building/nodes/mcp.md) node raises, the error message is bound to that node's error output and **the run continues** — the node's bar shows red in the trace while the run stays green. A tool error is structured output, not a crash. ### After a restart If the control plane restarts while a run is mid-flight, that run is swept to `failed` with the reason *"interrupted: control-plane restarted while run was in-flight"* — no zombie run that lies about being live. **`waiting` runs and durable runs are excluded** from this sweep: a paused run is not a zombie, and a durable run [resumes from its last completed step](durable.md) instead. ## Related pages - [Durable runs](durable.md) — crash-safe execution and the durable-only node types. - [Human-in-the-loop](human-in-the-loop.md) — pausing a run for approval and resuming it. - [Observability](observability.md) — the trace waterfall and per-node I/O. - [Runs & sessions](../concepts/runs-and-sessions.md) — the underlying concepts. --- Source: https://docs.ertir.dev/latest/running/durable/ # Durable runs A **durable run** checkpoints each step as it completes, so if the process crashes — or the machine reboots — the run picks up from the last completed step instead of being lost. Durable execution is also what lets a run **pause for a person** at a [human approval](human-in-the-loop.md) node and wait, potentially for days, before continuing. Most agents don't need this. The everyday interactive path — chatting, running an agent from the Bench — is fast and fully sufficient. Reach for durable mode when a run is long-lived, unattended, or needs to survive interruption. ## What durability gives you - **Crash-safe resume.** A completed step is journaled. If a crash interrupts the run, the durable runtime replays the completed steps from the journal (it does **not** re-execute them) and continues from where it stopped. - **Indefinite pauses.** A run can pause at a [human node](human-in-the-loop.md), persist as `waiting`, and resume whenever the input arrives — even across a restart. - **The advanced node types.** [Subgraphs, loops, and maps](../building/nodes/orchestration.md) and human approval only run durably (see below). ### The execution guarantee, stated honestly Durability is **exactly-once for completed (journaled) steps**, and **at-least-once for the step that was in flight at the moment of a crash**. That in-flight step re-executes on resume. Read-shaped work — calling a model, an HTTP tool, or an MCP tool — is safe to repeat. If you build an agent whose step has a non-idempotent side effect (say, charging a card), be aware that a crash at exactly the wrong moment can repeat it. ## Which nodes require durable mode Four node types **only** run on the durable runtime. On the interactive path they are rejected up front with a `durable_required` error: | Node | Why it needs durability | |---|---| | [`human`](human-in-the-loop.md) | Pauses the run indefinitely awaiting input — an in-process wait can't survive a restart. | | [`subgraph`](../building/nodes/orchestration.md) | Runs another published agent as a resumable child workflow. | | [`loop`](../building/nodes/orchestration.md) | Repeats a body agent; completed iterations must not re-run after a crash. | | [`map`](../building/nodes/orchestration.md) | Fans out one child run per element; a crash resumes only the incomplete branches. | If an agent contains any of these, the Bench shows a single **Run durably** button (labelled *"durable-only — runs on the durable runtime"*) instead of the normal run button. ## Turning durable mode on Durability is **off by default**. There are two ways to enable it, and they are equivalent — one runtime, two ways to host it. === "On your own machine" Set the environment variable in your `.env` and restart: ```bash THEYGENT_DURABLE=1 ``` (`true`, `yes`, and `on` also work, case-insensitively.) The control plane then runs the durable runtime **in-process** — there is no separate program to launch. This is the right choice for desktop and single-machine use. === "Server / air-gapped deployment" Run the standalone **durable worker** as its own process, pointed at the same Postgres: ```bash uv run --package theygent-worker theygent-worker ``` The worker is **not** started by `make up` — you start it yourself. It is the right choice when you want unattended runs to keep going independently of the API process, or to scale that work across machines. Either way, the durable runtime keeps its bookkeeping in a **separate `dbos` schema** inside the same Postgres you already use, and migrates it automatically. There is no extra migration step. !!! note "Interactive runs stay non-durable in both modes" Enabling durable mode does **not** change your chats or interactive agent runs. The streaming surfaces — New Chat, the Bench's normal run, and the direct run/invoke endpoints — always execute on the fast in-process path, whether or not durable mode is on. Only unattended fires and the durable-run endpoint use the durable runtime. ## The worker process When it starts, the standalone worker: 1. Connects the durable runtime to your shared Postgres. 2. Rehydrates the [MCP server](../building/nodes/mcp.md) registry, so `mcp_tool` steps can spawn their servers in the worker's trust domain. 3. Reconciles [cron schedules](triggers.md) to the enabled schedule triggers. 4. Blocks, consuming the durable work queue and recovering any workflows a crash left pending. It reads `DATABASE_URL` (required) and `THEYGENT_INFERENCE_PLANE_URL` (default `http://127.0.0.1:8081/v1`) — the same variables the control plane uses. See [Environment variables](../reference/environment.md). ## Running an agent durably ### From the interface Open a published agent's **Run** modal from the [Agents page](../building/saving-agents.md). The Run button is a split control: - Its main segment runs the agent normally (streaming). - Its caret menu offers **Run durably** — *"checkpoints each step so it resumes after a crash (no token streaming)."* A durable run is enqueued and then polled — it does not stream tokens; you watch its progress in the trace waterfall and read the persisted output when it finishes. If the server isn't in durable mode, you get an amber note: *"The control-plane isn't running in durable mode — start it with `THEYGENT_DURABLE=1` to run durably."* ### Over the API `POST /agents/{id}/durable-runs` runs a published agent durably. It is fire-and-poll: it returns **`202`** with the run id, and you poll [`GET /runs/{id}`](index.md) for progress and output. ```bash curl -X POST http://localhost:8080/agents/agent.my-agent/durable-runs \ -H "Content-Type: application/json" \ -d '{"input": "process the overnight queue"}' ``` The body is the same shape as a normal saved-agent run — `input`, optional `version` or `content_hash` to pin a version, optional `session_id`. The response: ```json { "run_id": "…" } ``` A few behaviours worth knowing: - **It requires durable mode.** Without it, you get `400 durable_required`. - **The version is pinned at enqueue.** If you don't pin one explicitly, the endpoint pins the content hash it just validated, so a version published between your request and the worker picking it up can't swap the graph out from under you. - **The run row appears promptly.** The endpoint waits up to ~2 seconds for the run row to exist before returning, so an immediate `GET /runs/{id}` doesn't 404. ## Related pages - [Human-in-the-loop](human-in-the-loop.md) — the human approval flow, which requires durable mode. - [Subgraph, loop & map nodes](../building/nodes/orchestration.md) — the other durable-only node types. - [Triggers & webhooks](triggers.md) — unattended fires become crash-resumable in durable mode. - [Environment variables](../reference/environment.md) — `THEYGENT_DURABLE` and the worker's variables. --- Source: https://docs.ertir.dev/latest/running/human-in-the-loop/ # Human-in-the-loop Some agents shouldn't run to completion on their own. A **human approval** node pauses a run at a chosen point, waits for a person to review and supply input, and only then continues. This is how you build approval gates, manual review steps, and "ask me before you act" checkpoints. Human approval is built on [durable runs](durable.md): a paused run persists as `waiting` and can wait indefinitely — minutes or days — surviving restarts until someone answers. ## Before you start The human node **only runs in durable mode**. If durable mode is off, the interactive path rejects it with `durable_required`, and `POST /runs/{id}/resume` returns `400`. Turn it on first — see [Durable runs](durable.md). ## The flow end to end ```mermaid sequenceDiagram participant You participant CP as Control plane participant W as Durable runtime You->>CP: Run the agent durably CP->>W: Start the workflow W-->>CP: status = waiting (at the human node) Note over W: Paused — checkpoint persisted You->>CP: POST /runs/{id}/resume {input} CP->>W: Deliver the input W-->>CP: status = streaming → completed ``` ### 1. Build: add a human node On the canvas, drop a **human approval** node into the path where you want the pause, wiring the value to review into its input and its output onward to the next step. See the [human node reference](../building/nodes/human.md) for ports and wiring. Its configuration controls what the pause expects: | Field | Type | Default | Description | |---|---|---|---| | `prompt` | string | — | An advisory description of what input you expect, shown to the reviewer. | | `inputSchema` | object | — | Advisory shape of the expected input; on resume its **required** keys are enforced. | | `timeout` | number \| null | `null` | Seconds to wait. `null` means wait forever. | | `onTimeout` | `fail` \| `default` | `fail` | On timeout, either fail the run or bind `default` and continue. | | `default` | any | — | The value used when `onTimeout` is `default`. | Publish the agent (see [Drafts & publishing](../building/saving-agents.md)) so it can be run durably. ### 2. Run it durably Run the published agent on the durable runtime — from the Bench's **Run durably** button, via `POST /agents/{id}/durable-runs`, or from a [trigger](triggers.md) in durable mode. See [Durable runs](durable.md) for each path. When execution reaches the human node, the run **pauses**: its status becomes `waiting` and it records which node it is waiting at (`awaiting_node`). It is now excluded from the restart sweep — it will sit patiently until you answer. ### 3. See that it's waiting On the [Runs page](index.md) the run shows a `waiting` chip. Open it and the detail page shows a **Resume panel**: *"Paused — awaiting input at ``"*. ### 4. Resume it #### From the interface In the Resume panel, choose **Text** or **JSON** input mode, type the value, and click **Resume**. JSON is parsed on the spot and validated loudly; an empty JSON box means a `null` input. The panel appears both on the run detail page and inside the Bench modal for a run started there. A run may pause **more than once** — an agent can have several human gates. After each resume it flips back to `streaming` and continues until the next gate or the end. #### Over the API `POST /runs/{id}/resume` delivers the awaited input: ```bash curl -X POST http://localhost:8080/runs/RUN_ID/resume \ -H "Content-Type: application/json" \ -d '{"input": {"approved": true, "note": "looks good"}}' ``` The `input` can be any JSON value — a string, an object, whatever the node expects. On success the endpoint returns **`202`**: ```json { "runId": "…", "status": "resuming", "awaitingNode": "…" } ``` The delivered value flows out of the human node's success output, and the run resumes. The resume endpoint's other responses tell you exactly what went wrong: | Status | Code | Meaning | |---|---|---| | 400 | `durable_required` | The deployment isn't in durable mode. | | 404 | `run_not_found` | No such run. | | 409 | `run_not_waiting` | The run isn't paused (it already finished, failed, or is still running). | | 409 | `awaiting_node_missing` | The run is waiting but records no target node. | | 422 | `resume_schema_mismatch` | The node's `inputSchema` requires keys your input is missing. | | 410 | `workflow_gone` | The durable workflow no longer exists; the run is marked `failed`. | !!! note "Double-clicking Approve is safe" Each human node is delivered on its own dedicated channel. A duplicate resume — a double-clicked **Resume**, a client retry racing the status flip — is buffered inertly against the already-answered gate and can never accidentally satisfy the run's *next* human gate. ## What happens on timeout If the node has a `timeout` and no one answers in time, the outcome follows `onTimeout`: - `fail` (the default) — the run fails with a clear reason naming the node and the timeout. - `default` — the node's configured `default` value is bound and the run continues as if that were the answer. ## Related pages - [Human node reference](../building/nodes/human.md) — ports, configuration, and worked example. - [Durable runs](durable.md) — the runtime that makes waiting possible. - [Runs](index.md) — statuses, the `waiting` state, and the run detail page. --- Source: https://docs.ertir.dev/latest/running/triggers/ # Triggers & webhooks A **trigger** gives a published agent an **unattended entry point** — a way to run without you sitting at the interface. There are three: - **Token invoke** — a standing HTTP endpoint you call with a bearer token. - **Webhooks** — an inbound URL a third-party system POSTs to, authenticated by a signature. - **Cron schedules** — a recurring timetable that fires the agent on its own. All three run the **same** graph on the **same** run path as an interactive run; they just start it differently. ## Prerequisites - **The agent must be published.** A trigger always fires a published, versioned agent — never a pasted graph or a draft. See [Drafts & publishing](../building/saving-agents.md). - **It runs a pinned version.** A webhook or schedule pins **exactly one** of `version` or `content_hash`, so an unattended deploy always runs an *immutable* artifact and never silently drifts to "latest". Pinning zero or both is a `400 invalid_trigger`, and the pin must resolve when you create the trigger (a dangling pin 404s then, not at fire time). See [Agent versioning](../concepts/versioning.md). !!! note "Where triggers are managed" Webhook and schedule triggers are created and managed over the control-plane API (the endpoints below). The interface shows a **read-only** list of an agent's triggers on the `input` node's inspector panel — there is no in-app create/edit screen for them yet. The curl examples on this page are the way to set them up. All examples target the control plane at `http://localhost:8080`. Managing triggers (`POST /triggers` and friends) requires an editor-or-above bearer in the `Authorization` header — see [Users, roles & sign-in](users.md). ## Token invoke The simplest unattended entry point: a single endpoint, `POST /agents/{id}/invoke`, gated by a bearer token. Nothing to register — it accepts two kinds of credential: - **A personal API key** (`tyk_…`) — the recommended, per-user way; mint one from your profile menu. See [API keys](users.md#personal-api-keys). - **The shared `THEYGENT_INVOKE_TOKEN`** — one per-deploy secret set in the control plane's environment (see [Environment variables](../reference/environment.md)): ```bash THEYGENT_INVOKE_TOKEN=a-long-random-secret ``` !!! warning "Deny by default" With `THEYGENT_INVOKE_TOKEN` **unset** and no API key presented, the invoke endpoint is **closed** — every call returns `401`. This is deliberate: an open unattended surface is never the default. Interactive sign-in tokens (`tys_…`) deliberately do not open it either. **Call it** with either credential in the `Authorization` header: ```bash curl -X POST http://localhost:8080/agents/agent.my-agent/invoke \ -H "Authorization: Bearer a-long-random-secret" \ -H "Content-Type: application/json" \ -d '{"input": "summarize today'\''s incidents"}' ``` The body accepts `input`, an optional `version` or `content_hash` pin, and an optional `session_id`. Unlike an interactive run, invoke defaults to **non-streaming**, so it returns the finished result: ```json { "runId": "…", "status": "completed", "output": "…" } ``` A missing or wrong token returns `401 unauthorized`. Every invoke is correlatable afterwards via [`GET /runs/{id}`](index.md). ## Webhooks A webhook is an inbound URL a third-party system (a CI pipeline, an issue tracker, a form backend) POSTs to. The request is authenticated not by the invoke token but by a **per-webhook signature**, because the caller is an external system rather than a credentialed user. ### 1. Create the webhook trigger `POST /triggers` with `kind: "webhook"`. The config requires a `secret` — the shared signing key both sides use: ```bash curl -X POST http://localhost:8080/triggers \ -H "Content-Type: application/json" \ -d '{ "agent_id": "agent.my-agent", "kind": "webhook", "version": "1.0.0", "config": { "secret": "shared-signing-secret" } }' ``` This returns `201` with the trigger, including its **id** (which you need for the inbound URL). The `secret` is redacted to `"***"` in every response — store your copy of it safely; it is never readable back. ### 2. Sign and send the payload The caller POSTs the raw JSON payload to `POST /hooks/{trigger_id}` with header **`X-TheYgent-Signature`** set to the **HMAC-SHA256** of the **raw request body**, keyed by the `config.secret`. Both `sha256=` (a common convention) and a bare `` digest are accepted, and the comparison is constant-time. ```bash SECRET='shared-signing-secret' BODY='{"repo":"acme/app","branch":"main"}' SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | sed 's/^.* //') curl -X POST http://localhost:8080/hooks/TRIGGER_ID \ -H "Content-Type: application/json" \ -H "X-TheYgent-Signature: sha256=$SIG" \ -d "$BODY" ``` !!! warning "Sign the exact bytes you send" The signature must be computed over the **raw body exactly as transmitted** — the same bytes, no reformatting. Signing a pretty-printed copy while sending a minified one (or vice versa) will fail verification. ### 3. The payload becomes the input The parsed JSON body **is** the agent's run input. A multi-input agent drills into it with [`$in.in.`](../building/input-references.md) — in the example above, `$in.in.repo` and `$in.in.branch`. On success the endpoint returns **`202`** with the run result: ```json { "runId": "…", "status": "completed", "output": "…" } ``` Webhook errors are specific: | Status | Code | Meaning | |---|---|---| | 401 | `invalid_signature` | The signature is missing or doesn't match. | | 404 | `trigger_not_found` | No webhook trigger with that id. | | 409 | `trigger_disabled` | The trigger exists but is disabled. | | 400 | `invalid_payload` | The body isn't valid JSON. | ## Cron schedules A schedule fires the agent on a recurring timetable, with no external caller at all. ### Create the schedule trigger `POST /triggers` with `kind: "schedule"`. The config requires a `cron` expression, and may include an `input` value passed to the agent on every fire: ```bash curl -X POST http://localhost:8080/triggers \ -H "Content-Type: application/json" \ -d '{ "agent_id": "agent.my-agent", "kind": "schedule", "version": "1.0.0", "config": { "cron": "0 9 * * *", "input": "daily digest" } }' ``` ### Cron syntax and timezone The `cron` field is a standard five-field cron expression: ```text ┌───────────── minute (0–59) │ ┌─────────── hour (0–23) │ │ ┌───────── day of month (1–31) │ │ │ ┌─────── month (1–12) │ │ │ │ ┌───── day of week (0–6, Sunday = 0) │ │ │ │ │ 0 9 * * * → every day at 09:00 */15 * * * * → every 15 minutes 0 0 * * 1 → every Monday at midnight 30 8 1 * * → 08:30 on the 1st of every month ``` The expression is validated when you create the trigger — a malformed cron is a `400 invalid_trigger` up front, never a silently never-firing schedule. !!! note "Schedules are evaluated in UTC" Cron times are interpreted in **UTC**, not your local timezone. `0 9 * * *` fires at 09:00 UTC. Convert your intended local time to UTC when you write the expression. ### How firing behaves - **No backfill.** Due-ness is computed from the last fire, so a long downtime doesn't replay every window you missed — the schedule simply resumes on its next due instant. - **No retry.** A fire that fails is recorded honestly on its run row, and the schedule advances to the next window. There is no automatic re-attempt. - **Restart-safe.** Enabled schedules are re-read from the database continuously, so a restart doesn't drop them and doesn't double-fire within a window. !!! warning "Run the scheduler on one instance (non-durable mode)" In the default (non-durable) setup, the schedule dispatcher must run on **exactly one** control-plane instance — two would each fire the same window. In [durable mode](durable.md) this constraint is lifted: durable schedules de-duplicate across instances, so two workers firing the same instant run the agent once. ## Managing triggers | Action | Endpoint | Notes | |---|---|---| | List | `GET /triggers` | Newest first; the `secret` is always redacted to `"***"`. | | Get one | `GET /triggers/{id}` | — | | Edit | `PATCH /triggers/{id}` | Only `enabled` and `config` are editable — **kind and the version pin are immutable**. A config edit is re-validated (a webhook can't drop its secret; a schedule can't take a bad cron). Re-pinning to a new version means creating a new trigger. | | Delete | `DELETE /triggers/{id}` | Returns `204`. Historical runs keep their trigger lineage as a breadcrumb even after the trigger is gone. | Disable a trigger without deleting it by patching `enabled` to `false`: ```bash curl -X PATCH http://localhost:8080/triggers/TRIGGER_ID \ -H "Content-Type: application/json" \ -d '{"enabled": false}' ``` ## Related pages - [Drafts & publishing](../building/saving-agents.md) — a trigger fires a published, pinned agent. - [Durable runs](durable.md) — make unattended fires crash-resumable. - [Referencing inputs](../building/input-references.md) — how a webhook payload maps into the graph. - [Runs](index.md) — every fire produces a run you can inspect. - [Users, roles & sign-in](users.md) — API keys, the per-user credential for the invoke endpoint. - [Environment variables](../reference/environment.md) — `THEYGENT_INVOKE_TOKEN` and the rest. --- Source: https://docs.ertir.dev/latest/running/users/ # Users, roles & sign-in Ertir's control plane has an **identity layer**: every person gets an account with one of three roles, every call to the API carries a bearer token, and admins can wire up single sign-on through any OIDC or OAuth2 provider. This page covers the first-run wizard, what each role can do, managing accounts, SSO, and the personal API keys that let your own scripts and frontends call Ertir. All examples target the control plane at `http://localhost:8080`. ## First run: create the admin A fresh install has **zero accounts, and the API is closed** — every management endpoint returns `401` until the first account exists. The only surfaces that answer before setup are the sign-in machinery itself (`/auth/status`, `/auth/setup`, `/auth/login`, and the SSO browser redirects), the health probes (`/healthz`, `/readyz`), and the unattended entry points that carry their own credentials ([webhooks](triggers.md#webhooks) with their HMAC signature, and `POST /agents/{id}/invoke` behind its own gate — see [API keys](#personal-api-keys) below). The first time you open the interface at `http://localhost:5174`, it shows a **first-run wizard**: pick a username and password, and you land in the app signed in. The account it creates is always an **admin** — someone has to own the install. Under the hood the wizard makes one call: ```bash curl -X POST http://localhost:8080/auth/setup \ -H "Content-Type: application/json" \ -d '{"username": "ada", "password": "correct-horse-battery", "display_name": "Ada"}' ``` It returns `201` with `{"token": "tys_…", "user": {…}}` — you are already signed in. Once **any** account exists, setup is permanently closed: further calls return `409 setup_already_complete`. ## Signing in Local accounts sign in with a username and password: - **Usernames are lowercase handles** — letters and digits, with `.` `_` `-` allowed after the first character. Sign-in is case-insensitive (`Ada` and `ada` are the same account); the display name is free text. - **Passwords are at least 8 characters** and are stored only as a memory-hard hash — never recoverable, by anyone. `POST /auth/login` returns `{token, user}`. The token is an opaque `tys_…` bearer that you present on every subsequent call as `Authorization: Bearer tys_…`; the server stores only its hash. A wrong password and an unknown username get one indistinguishable answer — `401 invalid_credentials` — while a disabled account is its own honest `403 account_disabled`. !!! note "Session lifetime" A sign-in lasts **168 hours (7 days)** by default. Admins change this with the `auth.session_ttl_hours` platform setting on the Settings page; the new value applies to sessions minted after the change — existing sessions keep the expiry they were issued with. **Sign out** (in the profile menu) revokes the presented session immediately. ## The three roles | What you can do | viewer | editor | admin | |---|---|---|---| | Chat with models and published agents; run published agents; your own chat sessions; read the agents list; inspect your runs and traces | ✓ | ✓ | ✓ | | Manage your own profile: display name, theme, password, personal API keys | ✓ | ✓ | ✓ | | Build agents: the editor, drafts, publishing, inline test runs | — | ✓ | ✓ | | Configure the building blocks: connections, MCP servers, RAG sources, triggers, the Bench | — | ✓ | ✓ | | Own the install: platform settings, user management, sign-in providers, import/export | — | — | ✓ | Roles are ordered — an editor can do everything a viewer can, an admin everything an editor can. The floors are enforced **server-side, per endpoint**: a call below the floor returns `403 forbidden` no matter what any client shows. The interface's navigation is role-aware as a courtesy — **Runs** and the **Configuration** group appear from editor up, and the **Settings** entry is admin-only — but hiding is UX, not the enforcement. ### Chat sessions are per-account Your conversations are yours. A viewer or editor sees only their **own** chat sessions (plus any ownerless sessions recorded before the identity layer existed); another user's session id answers `404`, exactly like a session that doesn't exist. Admins operate the install and can see and manage every session. ## Managing users (admin) Admins manage accounts under **Settings → Users** in the interface, or over the API: | Action | Endpoint | Notes | |---|---|---| | List | `GET /users` | The install's whole account list. | | Create | `POST /users` | `username`, `role` (default `viewer`), optional `password`, `display_name`, `email`. A taken handle is `409 username_taken`. | | Edit | `PATCH /users/{id}` | Change `display_name`, `email`, `role`, or `disabled`. | | Reset password | `POST /users/{id}/password` | Sets a new password **and revokes every live session** of that account — the reset exists because the old credential can no longer be trusted. | | Delete | `DELETE /users/{id}` | Returns `204`. The account's sessions and API keys die with it; runs and chat history keep their ownership breadcrumb — history outlives the account. | Two behaviors worth calling out: - **Disabling is immediate.** Setting `disabled: true` revokes every live session on the spot, and API keys owned by a disabled account stop resolving. A later sign-in attempt gets `403 account_disabled`. - **Creating an account without a password** makes an **SSO-only** account: the user cannot password-login and is expected to arrive through a sign-in provider. Give it an `email` and the first provider sign-in with a matching verified email claims it — see [pre-provisioning](#who-gets-in-auto-provision-domains-and-pre-provisioning). !!! warning "The last admin is protected" The last active admin can never be demoted, disabled, or deleted — any attempt returns `409 last_admin`. Promote a second admin first. This is deliberate: an Ertir install with no admin has no in-band way to recover itself. ### Self-service Every user, regardless of role, manages their own account from the **profile menu** at the bottom of the sidebar: display name, interface theme, password, API keys, and sign out. Changing your own password requires your current password and signs out every *other* session of your account (the one making the change survives). Over the API this is `GET/PATCH /auth/me`, `POST /auth/me/password`, and `POST /auth/logout`. ## Personal API keys An API key is a long-lived programmatic bearer — the recommended way for **your own frontends, scripts, and CI jobs** to call the control plane. Any user can mint one from the profile menu (**API keys → Create**), or over the API: ```bash curl -X POST http://localhost:8080/auth/api-keys \ -H "Authorization: Bearer tys_your-session-token" \ -H "Content-Type: application/json" \ -d '{"name": "deploy-script"}' ``` The `201` response contains the raw `tyk_…` token **exactly once** — it is stored hashed and can never be shown again. Afterwards, only the key's first characters identify it in the list. The rules that keep keys safe: - **A key never widens.** Its role is capped at your role when minted (requesting more is a `400`), *and* narrowed to your current role at use — demote an account and every key it owns instantly narrows with it, no rotation needed. - **Revocation is immediate.** `DELETE /auth/api-keys/{id}` (or the interface's Revoke button) kills the key on the spot. A key opens **both** the regular API surface and the unattended invoke endpoint. Calling a regular endpoint: ```bash curl http://localhost:8080/agents \ -H "Authorization: Bearer tyk_your-api-key" ``` Calling a published agent's [unattended invoke endpoint](triggers.md#token-invoke): ```bash curl -X POST http://localhost:8080/agents/agent.my-agent/invoke \ -H "Authorization: Bearer tyk_your-api-key" \ -H "Content-Type: application/json" \ -d '{"input": "summarize today'\''s incidents"}' ``` !!! note "What opens `/invoke`" `POST /agents/{id}/invoke` accepts an API key **or** the per-deploy `THEYGENT_INVOKE_TOKEN` (see [Triggers](triggers.md#token-invoke)). Interactive `tys_…` session tokens deliberately do **not** open it — it is the unattended surface. With neither credential configured nor any key presented, the endpoint stays closed (`401`). !!! note "Gateway keys and service keys are different" The [AI Gateway](../ai-gateway/index.md) has keys of its own: a developer's **gateway key** (`tyg_…`, from **AI Gateway → Setup & keys**) identifies them to a gateway, and a **service key** (`tygs_…`, minted by an admin) lets a gateway reach the control plane. Neither opens the management API — that takes a personal API key or a session. ## Single sign-on Admins configure sign-in providers under **Settings → Sign-in**. Each enabled provider becomes a button on the login screen; the whole flow is authorization code + PKCE, resolved server-side — the control plane exchanges the code and reads the provider's userinfo endpoint, then bounces the browser back to the interface with a fresh session token. A provider comes in two shapes: - **OIDC** — set `issuer_url` and the control plane discovers the endpoints (`/.well-known/openid-configuration`). - **Plain OAuth2** — for providers without a discovery document, set the three endpoints explicitly: `authorization_endpoint`, `token_endpoint`, `userinfo_endpoint`. Common to both: `client_id`, a **client secret** (write-only — encrypted at rest, never echoed back; sending a new one rotates it in place), `scopes` (default `openid profile email`), and the provisioning policy below. !!! warning "Set a persistent secret key first" The client secret is encrypted with `THEYGENT_SECRET_KEY`. If that variable is unset, the control plane falls back to an **ephemeral key that does not survive a restart** — so after the next restart the stored secret can no longer be decrypted and sign-in fails until you re-enter it. Set a stable [`THEYGENT_SECRET_KEY`](../reference/environment.md) *before* configuring a provider (generate one with `python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"`). ### The redirect URL When you register the app with your provider, the redirect / callback URL to enter is: ```text http://localhost:8080/auth/oidc/callback ``` — that is, `{control-plane}/auth/oidc/callback`. If your control plane is reached at a different address (a reverse proxy, a non-default bind), change it with the `auth.oidc_redirect_url` platform setting or pin it with the `THEYGENT_OIDC_REDIRECT_URL` [environment variable](../reference/environment.md). ### Worked example: an OIDC provider (Google-style) 1. In the provider's console, create an OAuth client, register the redirect URL above, and note the client id and secret. 2. Create the provider (interface: **Settings → Sign-in → Add provider**; API below): ```bash curl -X POST http://localhost:8080/auth/providers \ -H "Authorization: Bearer tys_your-admin-session" \ -H "Content-Type: application/json" \ -d '{ "name": "Google", "slug": "google", "config": { "issuer_url": "https://accounts.google.com", "client_id": "1234567890-abc.apps.googleusercontent.com", "scopes": "openid profile email", "auto_provision": true, "default_role": "viewer", "allowed_domains": ["you@example.com"] }, "client_secret": "GOCSPX-your-client-secret" }' ``` A **Google** button appears on the login screen. A half-configured provider (neither an `issuer_url` nor all three explicit endpoints) is rejected up front with `400 invalid_provider`, and a taken slug is `409 provider_exists`. ### Plain OAuth2 example: GitHub GitHub has no OIDC discovery document, so configure the endpoints explicitly: ```json { "authorization_endpoint": "https://github.com/login/oauth/authorize", "token_endpoint": "https://github.com/login/oauth/access_token", "userinfo_endpoint": "https://api.github.com/user", "client_id": "your-github-client-id", "scopes": "read:user user:email" } ``` !!! note "Plain-OAuth2 identity is what the provider returns" Identity comes from the provider's userinfo response — GitHub sends `id` and `login` rather than OIDC's `sub`. If the account's email is not exposed there, the created account has no email: a domain allowlist will then **deny** the sign-in (that is the policy working as intended), and email-based pre-provisioning cannot match. ### Who gets in: auto-provision, domains, and pre-provisioning Three per-provider knobs decide what happens on a first sign-in: - **`auto_provision`** (default on) — create an account automatically for anyone the provider vouches for, at **`default_role`** (`viewer`, `editor`, or `admin`; default `viewer`). With auto-provision off, unknown users are turned away with an `account_not_provisioned` message. - **`allowed_domains`** — a sign-in allowlist. Each entry is either a **full email** (an exact-address allow) or a **bare domain** (a whole-organization allow). `["you@example.com"]` admits only that one address; `["example.com"]` admits everyone at `…@example.com`. A sign-in whose email is missing or matches nothing is refused (`domain_not_allowed`); no list means no restriction. For a personal install, list your own email so a domain-wide account can't sign in as you. - **Pre-provisioning by email** — create the account yourself first (`POST /users` with an `email` and **no password**), and the first provider sign-in with a matching verified email links to it, keeping the role you chose. Only password-*less* accounts can be claimed this way — an account with a password is never linkable through a provider merely asserting an email. After the first sign-in the provider identity is **linked** to the account: later sign-ins land on the same account even if the email changes, and a disabled account stays refused regardless of what the provider says. If the provider supplies a profile picture, it becomes the account's avatar (shown in the sidebar and profile) and refreshes on each sign-in; local password accounts show their initials instead. Deleting a provider removes the sign-in button and its stored secret; accounts created through it survive (they just need another way in — a password set by an admin, or another provider). ## Related pages - [Triggers & webhooks](triggers.md) — the unattended entry points API keys open. - [Environment variables](../reference/environment.md) — `THEYGENT_OIDC_REDIRECT_URL`, `THEYGENT_INVOKE_TOKEN`. - [Import & export](../import-export/index.md) — admin-only, like everything that moves the whole install. - [Runs & sessions](../concepts/runs-and-sessions.md) — the history that outlives accounts. --- Source: https://docs.ertir.dev/latest/running/observability/ # Observability Every run is traced. Ertir records a timing tree for each execution into its own database and renders it as a zoomable **run waterfall** — one bar per graph node, with timings, token usage, and a click-through into each node's input and output. Nothing is sent to an external tracing service unless you opt in. This page explains how to read the waterfall, what gets captured and how to change it, and how to export traces to your own tools. ## Where the waterfall appears The same waterfall shows up in three places, so you rarely have to go looking for it: - On the **Runs** detail page — open any run from the [Runs list](index.md) to see its full trace below the output. - Inside the **agent bench** modal — every run you launch from [the Bench](bench.md) shows its waterfall as it executes. - Folded under each answer in [chat](../chat/index.md) — expand the collapsed **Trace** block on a run-backed turn (it loads only when you open it). ## Anatomy of a trace A trace is a tree of **spans** — timed segments of work. Three levels nest inside each other: - A **run-root** span covers the whole run. - One **node** span per graph node that executed (the LLM call, a tool, a router, and so on). A plain prompt run — one with no graph — is traced as a single synthetic node. - Child **phase** spans record the fine-grained steps inside a node. The phases that actually appear are: - `model.generate` — one per model generation, under an LLM node. - `tool.` — one per tool call an LLM makes autonomously inside a tool-calling loop; its bar is annotated with a check or cross for success or failure. - `queue.wait` — the gap between enqueue and pickup, on [durable runs](durable.md) only. Phase spans carry timing only — never payloads. ```mermaid graph TD run["run (root)"] --> llm["llm node"] run --> tool["tool node"] run --> out["output node"] llm --> gen["model.generate"] llm --> tc["tool.search"] ``` Loop and map iterations each get their own span, named `#` for the *i*-th branch. Nodes on a branch that was never taken (a dead router or guardrail branch) appear as thin grey **skipped** spans, in place, so the shape of the graph stays legible. Each span carries a status: `ok`, `err`, `skipped`, or `running` (still in flight). ## Reading the bars Bars share one time axis: a bar's position is when it started and its width is how long it took, so a glance shows you what ran when and where the time went. The tree nests under collapse chevrons, so you can fold a busy node away. Colors encode status: | Color | Meaning | |---|---| | Pulsing amber | In flight right now | | Violet | An instrumented phase (`model.generate`, `tool.*`, `queue.wait`) | | Green | A node that finished `ok` | | Red | A node that finished `err` | | Grey | A skipped node (a dead branch) | Hatched bands between bars mark **gaps** — time when nothing was on the timeline. Hovering one tells you what kind of gap it was: queue wait, an engine or MCP server spawning, or a transition between nodes. Rows carry inline annotations where they help: the duration, a check or cross for a tool phase, and — on a `model.generate` row — the model id and the generation rate in tokens per second. **Zooming.** Roll the mouse wheel over the timeline to zoom in, from 1× up to 32×, anchored at the cursor so you can drill into a busy stretch without losing your place. The current factor shows in the header (for example `2.0×`); a **Fit** button resets to the whole run. ## Inspecting a span Click any bar to open its inspector — a side pane when the window is wide, stacked below when it is narrow. It shows a compact stat line for that span: status, how long it `took`, the `model`, `ttft` (time to the first answer token), tokens in and out, the generation `rate`, the `finish` reason, the I/O byte sizes, and — when the work ran off the main process — which `worker` handled it. A [crawler](../building/nodes/crawler.md) call adds the `items` it returned and, when the provider reported it, its `cost` in USD. Below the stats are up to four payload sections for the node: **Input**, **Reasoning**, **Tool calls**, and **Output**. These are fetched lazily when you open the row, and whether they show anything depends on the capture policy (below). Selecting the run-root row instead shows the whole-run token totals. The **Reasoning** section (rendered in violet, between Input and Output) holds a reasoning model's thinking for that node. Thinking is kept out of the run's output and out of session memory — it is recorded here for inspection only, never folded into the answer. The **Tool calls** section (rendered in amber) appears on a model node that called tools on its own. It lists each call the model made with its arguments and — the useful part — the **result** the tool returned, which otherwise lives only in the model's private conversation. Selecting a single `tool.…` phase row narrows the section to just that one call's result. Like reasoning, this is recorded for inspection only; it never becomes a wire between nodes. ## Node input and output capture The waterfall's timing and status are always recorded. The **payloads** — the actual input and output values on each node's ports — are governed by a capture policy, because raw payloads can be large and can contain sensitive data. ### Capture levels There are three ordered levels: | Level | What is stored | |---|---| | `full` | The per-port input and output payloads, plus their byte sizes. | | `metadata` | Byte sizes only — no payloads. | | `off` | Nothing — not even byte sizes reach the span. | `off` is a hard stop: at `off` nothing about the payload is written locally and nothing rides an [OTLP export](#exporting-traces-to-external-tools). The inspector's Input/Output sections then show a plain reason instead of content — for example "I/O capture is disabled for this agent" or "Sizes only — full I/O capture is off for this agent". These states are never errors; the timeline stays fully readable. ### What is captured by default The effective level for a run is the **minimum** of three things: a deployment ceiling, a topology default (used when an agent has set no policy of its own), and any per-agent policy. - **The deployment ceiling** is the environment variable `THEYGENT_IO_CAPTURE` (`off`, `metadata`, or `full`). It defaults to **`full`** — the sensible default for a local daily driver — and no agent can capture above it. - **The topology default** is `THEYGENT_TOPOLOGY` (`local` or `hosted`), default `local`. A `local` topology defaults to `full`; a `hosted` topology defaults to `metadata`, so raw payloads never land in a hosted database by default (an agent can still opt back up to `full`, capped by the ceiling). So out of the box, on a local install, full input/output capture is on and the inspector shows everything. ### Changing capture for one agent You can set a capture policy per published agent. There is no screen for this today — it is an API call to the control plane. Read the current policy: ```bash curl http://localhost:8080/agents/agent.triage/io-policy \ -H "Authorization: Bearer dev-local" ``` Set it — for example, to store sizes only for one agent: ```bash curl -X PUT http://localhost:8080/agents/agent.triage/io-policy \ -H "Authorization: Bearer dev-local" \ -H "Content-Type: application/json" \ -d '{"io_capture": "metadata"}' ``` The response reports both what you asked for and what actually happens, so you can see when a deployment limit is overriding you: ```json { "agent_id": "agent.triage", "io_capture": "metadata", "effective": "metadata", "capped": false, "ceiling": "full", "topology_default": "full", "has_explicit_policy": true } ``` `effective` is the level that will apply; `capped` is `true` when the deployment ceiling or topology has pinned you below what you requested. !!! note "Changing capture does not change the agent" Setting a capture policy does **not** mint a new agent version — the policy lives beside the agent, not inside its hashed graph, so the [content hash](../concepts/versioning.md) is unchanged. The level is resolved once per run, not per node. An inline graph run with no published agent uses the topology default. ### How long payloads are kept Captured payloads are kept until you choose otherwise. To expire them, give an agent its own retention in its capture policy, `io_retention_seconds` (a positive number of seconds; for example `{"io_capture": "full", "io_retention_seconds": 604800}` keeps a week). Or set **Settings → Telemetry → io retention days**, which covers every agent without a retention of its own (0, the default, keeps everything). Once an hour the control plane deletes payloads older than that. The run, its timing and its spans stay. Only the inputs and outputs go. ### The size cap Individual payloads are capped so a single huge value can't bloat the store. The cap is `THEYGENT_IO_CAPTURE_MAX_BYTES`, default `262144` (256 KiB). An over-cap value is not dropped — it is stored as a preview plus its true byte size, and the inspector shows an amber note: "Payload truncated to the capture cap (the byte counts are the true sizes)." ## Token usage on model spans When a model reports usage, it lands on that model's `model.generate` phase span as three attributes: `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and `gen_ai.usage.total_tokens`. Per-node and per-run totals shown in the inspector are summed over these — so a multi-turn tool-calling loop accumulates all its turns onto the one LLM node, and the run-root row shows the whole-run total. Two things worth knowing: - **There is no cost figure on spans.** A USD cost appears only in [the Bench](bench.md), and only when the gateway prices the call (hosted APIs that report a cost). Local engines have no price, so no cost is shown. - **Usage also feeds the quota gate.** A [quota node](../building/nodes/ratelimit-quota.md) meters spend by reading `gen_ai.usage.total_tokens` off the agent's spans over a window. Turning payload capture off does not affect it — spans always flow; only the payloads are gated. ## The live trace While a run is streaming, the waterfall is live. It polls the persisted trace roughly once a second and overlays a live event stream, so in-flight bars keep growing (pulsing amber) and settle to green or red as each step closes. In the agent bench, hovering a node row also flashes the matching node on the read-only mini canvas beside the waterfall. Under the hood this is two endpoints: - `GET /runs/{run_id}/trace` — the persisted span tree, `{runId, status, spans}`, with no payloads. - `GET /runs/{run_id}/trace/stream` — a live Server-Sent Events overlay emitting `span.open` and `span.close` events and a final `done`, with a `: keepalive` comment every 15 seconds of silence. A run paused at a human gate keeps the stream open. The click-through payloads come from `GET /runs/{run_id}/nodes/{node_id}/io`, which returns the node's `inputs`, `outputs`, byte counts, `capture_level`, and — when payloads are gated — a `reason`. ## Exporting traces to external tools By default traces stay entirely local — the in-app waterfall reads Ertir's own store and there is no external tracing backend to run. If you already use an OpenTelemetry-compatible collector, you can opt in to export. Set the standard OTLP endpoint and export turns on: ```bash export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 ``` With the endpoint unset, export is off. When it is set, spans are pushed over OTLP/HTTP as they close. The OpenTelemetry SDK is an optional dependency — if the endpoint is set but the SDK isn't installed, Ertir logs a clear warning and simply leaves export off; a run never fails because of tracing. What crosses, and what does not: - **Only span scalars are exported** — timing, status, model id, token usage, byte sizes, worker attribution. The captured input/output **payloads never leave the machine** via OTLP; they stay in the local store. - Exported spans carry `service.name = theygent-control-plane` plus `theygent.run_id`, `theygent.node_id`, `theygent.status`, `theygent.executor_id`, and `theygent.worker_host`. The ids are deterministic, so the exported trace nests exactly like the in-app waterfall — including when a durable run resumes on a different worker after a crash. - To scrub an attribute on the exported copy, list its names in `THEYGENT_OTEL_REDACT_ATTRS` (comma-separated). Those values are replaced with `[redacted]` on the way out while the local span keeps them intact. ## When the trace is empty If observability is off or a run recorded no spans, the waterfall shows a plain note — "Trace unavailable — observability may be off, or no spans were recorded for this run." — rather than hammering the server. Telemetry is also fail-safe by design: a capture or write failure degrades with a warning and never fails the run it was observing. ## Works well with - [Runs](index.md) — the list and detail views the waterfall lives in. - [The Bench](bench.md) — where you compare runs and read token usage and cost. - [Runs and sessions](../concepts/runs-and-sessions.md) — the run lifecycle and where output comes from. - [Rate limit & quota nodes](../building/nodes/ratelimit-quota.md) — the quota gate that meters the same token usage. - [Environment variables](../reference/environment.md) — the full list of `THEYGENT_*` and OTLP settings. --- Source: https://docs.ertir.dev/latest/running/bench/ # The Bench The Bench is where you test and measure things before you wire them into an agent. Point it at a model to see how fast it answers, how it handles a prompt, and what it costs; point it at a published agent to run it with a real input and watch every step. It is the same tooling for a quick "does this work" check and a careful side-by-side comparison. (For trying a graph *while you are still building it*, the editor has its own [Test panel](../building/editor.md#testing-on-the-canvas) — no publish needed.) There are two benches — one for models, one for agents — plus a small tester for MCP tools. All three share the same measurement and trace machinery. ## What you can bench - **Models** — any registered model, across its modalities: chat and vision, embeddings, speech-to-text, and text-to-speech. Model testing goes **straight to your inference plane's data endpoints** — the raw prompt, image, or audio never passes through the control plane. Only the measured metrics are recorded afterwards. - **Agents** — any [published agent](../building/saving-agents.md), run through the normal run path so you exercise the whole graph exactly as production would. - **MCP tools** — a single registered tool, run standalone to check its shape and output. ## Benching a model Open the [Registries](../models/index.md) page and press the **Bench** button on any model row (titled "Test & benchmark this model"). A modal opens titled `Bench · `. At the top, the model's capabilities are probed and shown as badges — its modalities, its maximum context, and an "approximate" marker when the probe is a static hint rather than a confirmed value. A **Warm** button preloads the engine so your first timing isn't skewed by the model loading; a warm that fails surfaces as an error rather than a mysterious slow run. The panel you get below is chosen by the model's declared modalities: | Modality | Panel | What you do | |---|---|---| | `chat` | Chat | A real multi-turn conversation with an optional system prompt. Vision rides here — image attach turns on when the model advertises vision. | | `embeddings` | Embeddings | Embed one input, or two to see their cosine similarity. | | `audio.transcription` | Speech-to-text | Record from the mic or attach an audio file; the transcript comes back as the reply. | | `audio.speech` | Text-to-speech | Type text; the reply is a playable audio bubble. | Image-generation models have no bench panel of their own — you exercise them from [New Chat](../chat/vision-and-images.md) instead. ### Tuning parameters Each panel has a parameter form that is **schema-driven and capability-narrowed**: a parameter that needs a capability the model doesn't advertise is hidden, not greyed out, and every field has a `?` with help text. The chat form covers `temperature`, `top_p`, `max_tokens` (placeholder `2048`), `stop`, `presence_penalty`, `frequency_penalty`, `seed`, a **Reasoning** toggle, `reasoning_effort` (low/medium/high), `tool_choice`, and `response_format`. Embeddings, speech-to-text, and text-to-speech each have their own narrower set. These are the same specs the [editor](../building/editor.md) uses to tune a node — so what you dial in on the Bench is exactly what you can set on a node or binding later. ### Reading the metrics Every exchange is measured and the numbers show as badges. Which ones you see depend on the modality: | Metric | Label | Meaning | |---|---|---| | `ttftMs` | TTFT | Time to the first visible answer token (thinking does not count — inline reasoning can't fake an early TTFT). | | `totalMs` | Total | End-to-end time. | | `tokensPerSec` | Throughput | Tokens per second over the post-TTFT generation window. | | `promptTokens` / `completionTokens` | — | Token counts (completion falls back to a chunk count, clearly approximate, if the gateway reports no usage). | | `cost` | Cost | USD, shown only when the gateway prices the call — absent for local engines. | | `rtf` | Real-time factor | Speech-to-text: audio seconds ÷ processing seconds. | | `ttfbMs` / `charsPerSec` | TTFB / Throughput | Text-to-speech: time to the first audio byte, and characters per second. | | `latencyMs` | — | Image generation. | ### Saving, comparing, and presets Three actions turn a one-off test into something reusable: - **Save result** records this exchange's metrics to the control plane's bench store (`POST /bench/runs`). The server computes a `params_digest` (so two runs that differ by one parameter are distinct results) and an `output_digest` (a content fingerprint used for comparison **without** storing the raw output). The raw output is kept only if you opt in, and then only as a **local reference** on your own machine — never uploaded as a blob. - **Compare** appears once you've saved two or more results in the modal. Pick an A and a B and you get per-metric deltas (B − A) and whether the outputs matched (digest equality — no payloads compared). Deltas are colored by direction: for throughput and real-time factor, higher is better; for latency and the rest, lower is better; zero is neutral. The compare list holds only results you recorded in the current modal session. - **Save as preset** stores the current parameters as a named, modality-scoped snippet (`POST /bench/presets`). A preset is literal parameter values only — the name is never stored inside a graph, so it can't drift an agent's content hash. You can apply a preset to an agent later (below). All of a model bench's raw traffic — `/v1/chat/completions`, `/v1/embeddings`, `/v1/audio/transcriptions`, `/v1/audio/speech`, `/v1/images/generations` — goes directly to your inference plane. The control plane only ever sees the metrics and digests you save. ## Benching an agent Open the Agents page (the grid of your [published agents](../building/saving-agents.md)) and press **Run** on an agent card — disabled until the agent has at least one published version. The agent bench modal opens. (The **Bench** button appears on *model* rows in [Registries](../models/index.md); an agent card uses **Run**.) Pick a **Version** (it defaults to and follows the latest until you pin one), then give it an **Input**. The control follows the pinned version's input-node `modality` — a text box, a validated JSON editor, a microphone, an image attach, or a file picker — exactly as on the canvas and in chat; see [what the modality changes](../building/nodes/input-output.md#what-the-modality-changes). JSON is parsed loudly, so a malformed object tells you rather than failing silently, and a modality that needs an attachment keeps **Run** disabled until you stage one. A **JSON** option stays in the mode dropdown regardless, for the multi-input agents you address with `$in.in.` (see [referencing inputs](../building/input-references.md)). ### Run vs Run durably The **Run** control is a split button: - **Run** launches a normal streaming run (`POST /agents/{id}/runs`). Reasoning arrives on its own channel and the answer streams token by token, exactly like chat. - **Run durably** launches a checkpointed run on the [durable runtime](durable.md) (`POST /agents/{id}/durable-runs`). It returns a run id and the modal polls for the result — there is no token streaming, because a durable run journals results rather than streaming them. Use this to get crash-resumability. Agents that contain `human`, `subgraph`, `loop`, or `map` nodes are **durable-only** — they can run only on the durable runtime, so the modal shows a single **Run durably** button for them. !!! warning "Durable mode must be enabled" If the control plane isn't running in durable mode, a durable run returns a `durable_required` error and the modal shows: "The control-plane isn't running in durable mode — start it with `THEYGENT_DURABLE=1` to run durably." See [durable runs](durable.md). ### Reading an agent bench result Once the stream ends, the canonical output is re-read from the persisted run row (a graph that ends at a tool or router emits no tokens — the stored output is what counts). The answer renders like a chat reply: a collapsible thinking block above a markdown bubble — or, for an `audio` or `image` output boundary, a player or the image itself rather than the artifact reference behind it. A **Stop** button aborts a streaming run mid-flight, and leaving the modal aborts it too. Below the answer is the live [run waterfall](observability.md) alongside a read-only mini canvas of the pinned graph — hovering a node row in the waterfall flashes the matching node on the canvas, so you can see where the time went at a glance. If a durable run pauses at a human gate it shows a resume panel right there; the run buttons stay usable and the paused run remains visible under [Runs](index.md). For agents that aren't durable-only, a **Chat** section lets you hold a real conversation with the agent — turns share session memory and record as a session under Recents, just like [chat](../chat/index.md). It reads the same boundary, so a voice agent gets the microphone composer here too. ### Applying a preset to an agent If you saved parameter presets from a model bench, the agent bench offers **Apply a preset**: pick a saved preset and one of the agent's model bindings, and Ertir writes the preset's literal parameters into that binding and publishes a **new agent version** through the registry. Each preset's modality is shown beside its name so you can tell which binding it belongs on; the preset name never enters the graph — only its values do, so the new version's [content hash](../concepts/versioning.md) reflects real content. ## Testing an MCP tool The [MCP](../building/nodes/mcp.md) page has a **tool tester** for exercising one registered tool in isolation. Pick a connected server, name a tool, and send it either an uploaded image (under a configurable argument name) or free-form JSON arguments. It composes a throwaway `input → mcp_tool → output` graph and runs it through the normal run path — no special execution route — so what you see is exactly what an agent would get. Structured detections (for example bounding boxes from a vision tool) draw over the image; raw output shows below. ## Where results are stored Saved model-bench results, presets, and any test suites live in the control plane's Postgres database (the bench tables), alongside your runs and agents. What is stored is deliberately lightweight: - **Metrics and digests**, not raw outputs. The `output_digest` lets comparisons check whether two outputs matched without keeping the text. - **Raw output only on opt-in**, and then only as a **local reference** in your own trust domain — Ertir never uploads the generated text, image, or audio. - Bench records carry the coordinates you'd expect for later analysis: `target_kind` (`model` or `agent`), `modality`, `logical_id`, `binding`, the `params` and their digest, the `metrics`, the `output_digest`, and a `run_id`. Saved results are readable through the API (`GET /bench/runs`, filterable by `logical_id`, `agent_id`, `suite_id`, or `case_id`) and comparable in-modal. Suites of golden cases and assertions exist in the API for future use, but there is no page for them yet — recording results, in-modal compare, and presets are what the interface surfaces today. ## Works well with - [Installing local models](../models/installing.md) — get a model into your registry so it has a Bench button. - [Remote models](../models/remote.md) — register a hosted, OpenAI-compatible model to bench (and to see a cost figure). - [Observability](observability.md) — the run waterfall the agent bench embeds. - [Durable runs](durable.md) — what "Run durably" does and why some agents require it. - [Drafts & publishing](../building/saving-agents.md) — creating the versions the agent bench runs and the presets it applies. --- Source: https://docs.ertir.dev/latest/ai-gateway/ # AI Gateway The AI Gateway sits between **Claude Code** — the CLI, IDE extensions, and the desktop app — and the Anthropic API. It records what each exchange cost, what it would have cost if subagents and cold-cache turns had run on a cheaper model, and why each request was or wasn't a candidate — and, once you choose to, it **routes** them there: to a cheaper Anthropic model, or to [models you run yourself](local-models.md). You see where your team's model spend goes, per day, model, developer and session, before you move anything. !!! note "Observe first, then route" In **observer** mode every request reaches Anthropic exactly as Claude Code sent it; the gateway reports what routing *would* save. When those figures convince you, an admin switches to **router** mode, and the saving becomes real: a subagent asking for Opus 5.5 runs on Sonnet 5.5, for example. Only the model changes — never the prompt — and a model that refuses a request hands it back to the one Claude Code asked for before Claude Code notices. The one setting the setup adds that changes Claude Code's own behavior, [tool search](setup.md#why-each-setting), turns back on what a custom gateway address otherwise turns off. ## What you get - **Spend and what routing would save — or saved** — per day, request class (main conversation, subagents, compaction, side requests), model, client, developer and session. See [Reports & policy](reports.md). - **Session timelines** — every request of a Claude Code session in order, with what the policy would have done and the reasons, in plain words. - **Setup warnings** — for example, Claude Code sending every tool's definition with every request because tool search is off, which can multiply input tokens several times. - **Subscription seats** — for developers on a claude.ai plan, how much of the plan's five-hour and weekly allowance their traffic used. - **Local models** — a share of the work, per kind of request, answered by models on your inference plane at no API spend, each measured on your own traffic first. See [Local models](local-models.md). ## What it sees, and what leaves it The gateway relays prompts, code, model responses and credentials, so it runs **where you run it** — your machine or your cluster — in the same trust domain as Claude Code. It never writes a prompt or a response anywhere, and never stores a credential except an organization key you give it. What it sends to the control plane is **metadata only**: timings, model names, token counts and costs, the routing decision and its reasons, Claude Code's session and agent ids, and the kind of each request. Never prompt or response text. Everyone sees their own records in full; admins see everyone's; other roles see organization totals without individual detail. ## Personal or team | | Personal | Team | |---|---|---| | Who it serves | You, on your own machine | Every developer who points Claude Code at it | | Listens on | Loopback only (`127.0.0.1`) | Any address you choose | | Identifies callers by | Nobody — it is yours | Each developer's **gateway key** (`tyg_…`) | | Billing | Your own Claude login or API key | Each developer's own credential, or the organization's key | | Records | Kept locally; read them with the `report` command, or attach it to the control plane | Sent to the control plane: the **AI Gateway** page | [Running the gateway](running.md) covers both, on a machine, in Docker Compose, and in Kubernetes. ## Observer, router and passthrough An admin chooses the gateway's **mode** on the AI Gateway **Settings** tab: - **Observer** records every exchange — its cost and what routing would have done — without changing it. - **Router** records every exchange and applies the routing decisions: routed requests run on the target model — a cheaper Anthropic model with the rest of the request exactly as Claude Code sent it, or a [local model](local-models.md). A target that refuses a request sends it back to the requested model, unseen, and that conversation stays there until its cache goes cold. Switching back to observer stops routing within one heartbeat (30 seconds). - **Passthrough** forwards traffic without recording anything. A gateway attached to the control plane forwards in passthrough until an admin saves a configuration: your organization's traffic is recorded only once someone chose to record it. A personal gateway on its own records from the start. ## Try it on your own machine A personal gateway needs no keys and no setup beyond Claude Code's settings: 1. **Start Ertir** with `make up` ([Running Ertir](../getting-started/running.md)). The gateway listens on `127.0.0.1:8082`. 2. **Point Claude Code at it**, in the shell you start Claude Code from: ```bash export ANTHROPIC_BASE_URL='http://localhost:8082' export CLAUDE_CODE_GATEWAY_HINT_HEADERS='1' export ENABLE_TOOL_SEARCH='true' ``` 3. **Work as usual.** Your own Claude login or API key goes through the gateway unchanged. 4. **Read the report** — what your traffic cost, and what routing would save: ```bash uv run --package theygent-ai-gateway theygent-ai-gateway report --since 24h ``` 5. **To see it on the AI Gateway page, and to route**, attach the gateway to the control plane with a service key ([Running the gateway](running.md#attach-it-to-the-control-plane)). Then save a configuration on **AI Gateway → Settings**: observer mode to record, router mode to route. ## Getting started with a team 1. **Run a gateway** — [Running the gateway](running.md). 2. **Turn recording on** — an admin opens **AI Gateway → Settings** and saves the configuration with **Observer** selected. 3. **Set up Claude Code** — each developer opens **AI Gateway → Setup & keys**, creates a gateway key, and copies the setup for their terminal, IDE or desktop app — [Set up Claude Code](setup.md). 4. **Work as usual** — then watch **Overview**, **Report** and **Sessions**. 5. **Route** — when the Report's saving is one you want, an admin selects **Router** on the Settings tab (with a subagent target such as `claude-sonnet-5-5`). The Report then shows what routing *saved*, against what the same traffic would have cost on the models Claude Code asked for. ## The AI Gateway page | Tab | Who | What it shows | |---|---|---| | **Overview** | everyone | The mode, the configuration version, each gateway instance's health (online, running the latest configuration, upstream reachable, records waiting to export) | | **Report** | everyone | Spend, what it would have cost with routing, the saving, spend by day, breakdowns, seats, setup warnings | | **Sessions** | everyone | Claude Code sessions, most recently active first, and each one's timeline of requests and decisions | | **Setup & keys** | everyone | How you are billed, your gateway keys, and copy-ready setups per client | | **Catalog** | admins | Kinds of side requests Claude Code sends (titles, suggestions, checks) and what routing may do with each | | **Settings** | admins | The configuration and its history, the organization's Anthropic key, developers' billing, service keys | ## Related pages - [Set up Claude Code](setup.md) — the settings, per client, and why each one is there. - [Reports & policy](reports.md) — reading the figures, and how the policy decides. - [Local models](local-models.md) — sending work to models you run, measured first. - [Running the gateway](running.md) — run modes, the organization's key, errors. - [Environment variables](../reference/environment.md#ai-gateway) — the gateway's settings. --- Source: https://docs.ertir.dev/latest/ai-gateway/setup/ # Set up Claude Code Pointing Claude Code at the AI Gateway is a handful of environment settings. The **Setup & keys** tab on the AI Gateway page builds them for you — with your key filled in, right after you create it — so this page explains what they are and why each one is there. ## Before you start - **The gateway's address.** By default `http://localhost:8082`. If your team runs a shared gateway, use its address; the Setup & keys tab lets you set the address its snippets use. - **A gateway key** (team gateways only). Create one on **Setup & keys → New key**. It is shown **once**, in the dialog that creates it, together with setups that already contain it. A gateway key identifies you to the gateway and opens nothing else in Ertir; revoke it there when a machine is retired. - **How you are billed.** An admin sets it per developer (**Settings → Developers**): | Billing | Your traffic reaches Anthropic with | Your gateway key travels as | |---|---|---| | **Your own Claude login or API key** | your claude.ai subscription or your own API key, as today | a header of its own (`X-TheYgent-Gateway-Key`) | | **The organization's key** | the organization's Anthropic key, which the gateway adds in place of yours | your only credential (`ANTHROPIC_AUTH_TOKEN`) | The Setup & keys tab shows which one applies to you and builds the matching setup. ## Terminal and IDEs For the CLI — and IDE extensions launched from a terminal that has these set — export them in your shell profile. Billed to your own credential: ```bash export ANTHROPIC_BASE_URL='http://localhost:8082' export ANTHROPIC_CUSTOM_HEADERS='X-TheYgent-Gateway-Key: tyg_…your gateway key' export CLAUDE_CODE_GATEWAY_HINT_HEADERS='1' export ENABLE_TOOL_SEARCH='true' ``` Billed to the organization's key, the gateway key *is* your credential: ```bash export ANTHROPIC_BASE_URL='http://localhost:8082' export ANTHROPIC_AUTH_TOKEN='tyg_…your gateway key' export CLAUDE_CODE_GATEWAY_HINT_HEADERS='1' export ENABLE_TOOL_SEARCH='true' ``` On a **personal** gateway there is no key: set `ANTHROPIC_BASE_URL` and the last two lines. ## `settings.json` The same settings work as the `env` block of `~/.claude/settings.json`, which every Claude Code session reads, however it was started: ```json { "env": { "ANTHROPIC_BASE_URL": "http://localhost:8082", "ANTHROPIC_CUSTOM_HEADERS": "X-TheYgent-Gateway-Key: tyg_…your gateway key", "CLAUDE_CODE_GATEWAY_HINT_HEADERS": "1", "ENABLE_TOOL_SEARCH": "true" } } ``` ## The desktop app The desktop app ignores `ANTHROPIC_BASE_URL` and `settings.json`; it reaches a gateway only through its own third-party inference settings, which carry an API key and never a claude.ai login. So it works through the gateway **for developers billed to the organization's key**: | Setting | Value | |---|---| | Inference provider | Gateway | | Gateway base URL | the gateway's address | | API key | your gateway key | | Tool search | On | Developers on their own claude.ai login keep using the CLI and IDEs through the gateway. ## For admins: managed settings The **Managed settings** view on Setup & keys gives the `env` block for everyone at once — the gateway's address and the two flags, **never a key**; each developer adds their own. Distribute it the way you distribute Claude Code's managed settings today. ## Why each setting - **`ANTHROPIC_BASE_URL`** sends Claude Code's model traffic to the gateway instead of straight to Anthropic. - **`CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`** makes Claude Code label each request: the main conversation, a subagent, compaction, a side request, and the session and agent it belongs to. Without the labels the gateway cannot tell those apart, so no request is a routing candidate. The Report warns when they are missing. - **`CLAUDE_CODE_AUTO_COMPACT_WINDOW`** (in a shell) or **`autoCompactWindow`** (in a settings file) appears only when the gateway serves main conversations on a local model. Claude Code compacts by the context window of the model it asked for, 1 million tokens for current models; this tells it to compact while the conversation still fits the local model. See [Local models](local-models.md#5-for-the-main-conversation-update-claude-codes-setup). - **`ENABLE_TOOL_SEARCH=true`** keeps MCP tool search on. Claude Code turns it **off for any custom base URL**, and then every request carries the full definition of every connected tool. In one measurement with many connectors loaded, a one-line prompt went from about 28,000 to 184,000 input tokens — 6.5 times as many. The Report flags this as *tool search is off*. ## Claude Code's own cost display Claude Code prices every token at the model it **asked for**. In observer mode that is also the model that answered, so Claude Code's figures and the gateway's agree (at list prices; the gateway also applies any contracted rates an admin entered). In **router** mode, routed requests run on a cheaper model, but Claude Code still files and prices them under the model it asked for. Its own cost figures overstate spend: the session cost, the usage summary, a status line that shows cost, and the SDK's `total_cost_usd`. For example, one Explore subagent task cost $0.16 on Opus 5.5 and $0.08 on Sonnet 5.5, with near-identical token counts. Route that subagent to Sonnet and Claude Code still shows about $0.16 for it. On the AI Gateway **Report**, its spend is about $0.08, and $0.16 appears as *without routing*. The token counts still agree. Claude Code's per-model usage matches the Report grouped by **model requested**, and **served model** shows where the tokens actually went. The Report is the figure to trust for cost. On a subscription, dollar figures are the API-equivalent value of the traffic, not a bill; the Report shows plan usage separately. ## Check that it works 1. Run any Claude Code prompt. 2. On **AI Gateway → Sessions**, your session appears with its requests, once the gateway is in observer mode (records reach the page within a few seconds). 3. If Claude Code reports an error from the gateway instead, the message names what is missing — see [errors the gateway returns](running.md#errors-the-gateway-returns). ## Related pages - [AI Gateway](index.md) — what the gateway does and what leaves it. - [Reports & policy](reports.md) — what the figures mean. - [Running the gateway](running.md) — starting one, and its errors. --- Source: https://docs.ertir.dev/latest/ai-gateway/reports/ # Reports & policy What the AI Gateway page's figures mean, how the routing policy decides what moves, and how admins tune it. In observer mode the gateway only records each decision; in router mode it also applies it — changing nothing but the model. ## The Report tab The report covers the last week unless you pick another range. Admins start on everyone's traffic; everyone else on their own (and can switch to organization totals). | Tile | Means | |---|---| | **Spend** | What the traffic cost, at list prices plus any contracted rates an admin entered. Once the gateway routes, its hint shows what the same traffic would have cost *without routing* — on the models Claude Code asked for | | **With routing** | What it costs with the policy's decisions applied: a request the gateway routed costs what it cost, one it only would have routed costs the target's price, everything else as it ran | | **Saving** | Without routing minus with routing: what routing saved (router mode) or would save (observer mode), and its share | | **Calls** | API calls, how many were routed, how many would have been, and how many could have been answered by a local model instead | Below them: spend by day (the full bar is what each day cost as requested, the filled part with routing; the gap is the saving), and a breakdown you can group by request class, client, the model Claude Code requested, the model that served it, the routing decision or its reason, the model a shadow run ran on — and, where you may see individuals, by developer or session. Grouping by requested model matches Claude Code's own usage summary, which files every token under the model it asked for. **Shadow runs.** A saving is an estimate: it assumes the target would have written as much as the model Claude Code asked for. Shadow runs check it. After a request that routing would have sent elsewhere, the gateway sends a copy to that target; after a request in a local dial's *trial* (see Settings below), it sends a copy to the local model. Claude Code never sees the copy, and it never delays Claude Code's answer. The **Shadow runs** table shows, per model, how many copies ran, how many of their answers routing could have used, and how much the target wrote compared with the model Claude Code asked for. For agent turns (subagents and the main conversation) it also shows how often the target picked the same next action as the real reply: the same tool first, or the answer where the reply answered. That is the measure to watch before trusting a local model with agent work. It also shows what the copies cost, which is kept apart from the spend above. The Sessions timeline shows the shadow run under each request that had one, a trial named as such. **Subscription seats.** For developers on a claude.ai plan, the dollar figures are the API-equivalent value of their traffic, not a bill. The seats table shows each developer's latest reading of the plan's five-hour and weekly allowance and how much of it was used in the range. **Setup warnings** appear above the figures when the traffic shows a problem, each with its fix: | Warning | Fix | |---|---| | Tool search is off — every tool's definition went with every request | Set `ENABLE_TOOL_SEARCH=true` (desktop app: tool search on) | | Claude Code is not labelling its requests | Set `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` | | Unknown request classes | A newer Claude Code than this gateway knows: update Ertir | | No price known for a model | An admin adds its rates under **Settings → Contracted prices** | ## The Sessions tab Sessions are listed most recently active first — yours, or everyone's for admins. Open one for its timeline: each request in order with its offset from the session start, its class (main, subagent, compaction, auxiliary), the agent type for subagents, the model requested and the one that served it, tokens, cost, errors, and the decision — *stays*; *routed →* a target, with what that saved against the requested model (router mode); or *would route →* a target, with what that would have cost (observer mode). Every reason is a short label; hover it for the explanation. A few of them: | Reason | Means | |---|---| | subagent spawn | A subagent starts with an empty cache, so moving it costs nothing extra | | cold cache | The main conversation's cache is cold (its start, after compaction, or expired) | | mid-conversation | Inside one prompt's tool loop: a conversation never switches mid-turn | | warm cache | Moving would throw away a warm cache and pay to rebuild it | | follows cache | The conversation keeps the model its cache is on until the cache goes cold | | switch pays | Staying would cost at least the configured margin times rebuilding the cache elsewhere | | no target set | The configuration names no target for this kind of request | | target can't take it | The target model would reject this request's shape | | target not cheaper | The target doesn't cost less than the model Claude Code asked for | | unreviewed kind | A side request whose kind an admin has not reviewed in the Catalog | | local share | The conversation is in the share its kind sends to a [local model](local-models.md) | | no local model fits | In the local share, but no listed local model can take this request (tool calls, images or prompt size) | | outgrew local model | The conversation grew past its local model: it moved to a larger listed one, or to Anthropic when none can take it | | target refused | The target, or the local model, could not take the request: it went to the model Claude Code asked for, unseen | ## How the policy decides Prompt caches belong to one model, so moving a conversation mid-stream throws a warm cache away and pays to rebuild it. The policy therefore only considers a move where the cache is cold anyway: - **A subagent, when it starts.** Its cache is empty either way — the clearest saving. In measurements, the same exploration subagent cost about half as much on Sonnet 5.5 as on Opus 5.5. - **The main conversation, at a cold point** — its start, after compaction, or after its cache expired — and only if the configuration names a main target. - **Then it stays where its cache is.** Every later request — the tool loop, and new prompts while the cache is warm — follows the same decision until the next cold point (*follows cache*), so a conversation never changes model mid-turn, and a routed one isn't pulled back to the requested model either. A warm move happens only where staying would cost at least the **warm switch margin** times rebuilding the cache on the other model (*switch pays*). If Claude Code switches model itself, the policy decides afresh. Some traffic never moves: Claude Code's quota check, side requests that ride the main conversation's cache, auto mode's safety checks, and side requests of a kind nobody has reviewed. Compaction reads the main conversation's cache, so it moves only with the conversation: it runs on the model the conversation is on — the one asked for, a cheaper one, or a local one. And a target must accept the request: Sonnet 5.5 takes Claude Code's requests to Opus 5.5 as they are; Haiku 4.5 rejects them, so a move to it is turned down as *target can't take it*. A target must also cost less, at your contracted prices where you set them. A request that already asks for a cheaper model, such as a subagent on Haiku 4.5, or for a model with no known price, stays as *target not cheaper*. **Estimates.** A "with routing" figure assumes the target model would have used the same number of tokens. Models differ in how much they write, so treat savings as estimates; the actual figures are exact. **Archived days.** Requests older than the retention period (90 days by default) are archived: the Report reads those days from daily totals, so every figure and grouping stays the same. Its footer says from which day that applies. Setup warnings, seat readings and the prices used cover only the days still kept in detail. A Sessions timeline lists only the requests still kept in detail, and its totals still count every request. **Prices.** Figures use Anthropic's published list prices. The Report's footer names the date they were read (*Prices: 2026-10-11*), and Ertir checks for changes every week. Requests in Claude Code's fast mode are priced at fast-mode rates. A model priced by prompt length, such as Haiku 5.5, is priced at its higher rates for a long prompt. Where your organization pays other rates, an admin enters them under **Settings → Contracted prices**. ## Catalog (admins) Claude Code also sends small **side requests** — session titles, suggestions, checks. The Catalog lists each kind it has seen, told apart by the shape of the request, with how often it was seen, the Claude Code versions that sent it, and its status: | Status | Routing may | |---|---| | unreviewed | do nothing — it passes through untouched until someone decides | | observe only | record and report it, never route it | | cheaper model | send it to the configuration's subagent target | | local model | answer it with the configuration's local model | | never route | do nothing — it always stays on the model Claude Code asked for | Kinds the gateway recognises by their shape come with a **suggestion** — the quota check and side requests riding the main conversation's cache: *never route*; the session title: *local model* — which you can apply in one click or override with **Review**. A suggestion is only that: the status is your decision, and saving one creates a new configuration version. In router mode, a *local model* side request goes to a model on your inference plane, the **Side requests** dial's (see [Local models](local-models.md)). Claude Code gets the local model's answer in the form the Anthropic API would have given it, but only after the gateway has checked it: for a session title, JSON with a `title`. A request the local model cannot take goes to the model Claude Code asked for, marked *target refused*, and Claude Code never sees the difference. That covers a request carrying something that doesn't translate (a document, for example), an error, no answer in time, an answer of the wrong shape, or no inference plane running. A local answer costs no API spend; its *without routing* figure is an estimate, counted with the local model's tokens. ## Settings (admins) - **Mode** — observer, router or passthrough (see [AI Gateway](index.md#observer-router-and-passthrough)). Saving refuses a target no Claude Code request could go to — a model the gateway has no data for, or one that rejects Claude Code's requests, such as Haiku 4.5. - **Routing targets** — the Anthropic model a subagent could start on (for example `claude-sonnet-5-5`), the model for the main conversation at cold points (empty: never considered), and the local model for side requests the Catalog allows to run locally (a [logical model id](../concepts/models-and-engines.md)). - **Local models** — the models on your inference plane that may take Claude Code's work, listed small to large, each with its context size, the most it may write in a turn, whether it follows tool calls and takes images, and optionally what it costs to run. Beside each one is what the last week shows: the requests it served, and how often it picked the same next action as Claude. Then one dial per kind of request (side requests, subagents, the main conversation), each a **share** served locally, a **trial** measured locally while Anthropic answers, and the **model** (or "smallest that fits"). How to set them, and their limits: [Local models](local-models.md). - **Warm switch margin** — a conversation whose cache is warm moves only where staying would cost at least this many times rebuilding its cache on the other model (at least 1). Cache reads cost a fraction of cache writes, so in practice warm conversations stay put. - **Shadow runs** — which targets get shadow runs (see the Report tab above). - *Local model* is on by default. It costs no API spend, only the local model's time. (A dial's trial is its own choice and runs whatever this says.) - *Cloud targets* are off by default, because each copy is a real API call. Cloud shadow runs use only the organization's key (on a personal gateway, your own API key), never a subscription seat's login or a developer's own key. They stop for the day at the **budget** (per gateway instance, counted from its last restart), and pause for ten minutes when the target's rate limit runs low. - **Contracted prices** — per-model rates (USD per million tokens: input, output, 5-minute and 1-hour cache writes, cache reads) where yours differ from list price. Every save creates a new, immutable **version**; gateways pick it up at their next heartbeat (every 30 seconds by default). The **History** lists every version; loading an older one starts an edit from it, and saving makes it the newest. If someone else saved meanwhile, the page keeps your edit and asks before replacing theirs — their Catalog reviews are kept either way. The same tab holds the [organization's Anthropic key](running.md#the-organizations-key), **Developers** (who pays for each developer's traffic) and the gateways' **service keys** ([Running the gateway](running.md#attach-it-to-the-control-plane)). ## Related pages - [AI Gateway](index.md) — what the gateway does and what leaves it. - [Set up Claude Code](setup.md) — the settings the warnings refer to. - [Local models](local-models.md) — the local dials, trials and their limits. - [Running the gateway](running.md) — the command-line report for a personal gateway. --- Source: https://docs.ertir.dev/latest/ai-gateway/local-models/ # Local models In router mode, the AI Gateway can answer some of Claude Code's work with models you run on your [inference plane](../concepts/models-and-engines.md), instead of the Anthropic API. A request a local model answers costs no API spend. How much goes local is your choice, per kind of request, and you measure a model on your own traffic before it serves any. !!! note "Measure before you serve" A small model follows Claude Code's agent work poorly. In one measurement, a 4-billion-parameter model picked the same next step as Claude on 1 real exploration turn in 6. A larger model does better, but only your own traffic can say by how much. So every agent share starts at 0%, and a **trial** measures a model on conversations Anthropic still answers. ## How it works Each kind of request has a **dial** on **AI Gateway → Settings → Local models**: | Dial | Default | What goes local | |---|---|---| | Side requests | 100% | The kinds of side request (session titles, for example) an admin reviewed *local model* in the [Catalog](reports.md#catalog-admins) | | Subagents | 0% | A share of subagents, from their first request to their last. You can limit it to some agent types, such as `Explore` | | Main conversation | 0% | A share of conversations, from their start or their last compaction | Each dial has three settings: - **Share.** The percentage of conversations served by a local model. A conversation is in or out of the share the same way every time, and only where its prompt cache is cold anyway: when a subagent starts, when a conversation starts, or after compaction. It then stays on that model for its whole task, so a conversation never changes model mid-turn. A conversation already running when you raise a share moves at its next cold point. - **Trial.** A further percentage of conversations, just above the share. Anthropic still answers them. Afterwards the gateway sends each of their requests to the local model and records whether it picked the **same next action** as the real answer: the same tool first, or answering where the real answer answered. A trial costs no API spend, works in router mode too, and never delays anyone: a request a local model is serving always comes first, and a trial running then is dropped. Raising the share by the trial serves exactly the conversations that were tried. The share and the trial add up to 100% at most. - **Model.** A listed model, or *smallest that fits*: the first model in the list that can take the request. A subagent or main-conversation turn needs a model that follows tool calls. A request with images needs one that takes images. Every model needs the prompt to fit within its context. **A conversation that outgrows its model moves up.** When a conversation's prompt no longer fits its local model, it moves to the next larger listed model that can take it, marked *outgrew local model*, and stays there. Only when no listed model can take it does it go to Anthropic, which reads the whole history without a cache, and stays there until its next cold point. When a local model can't take a request for another reason, Anthropic answers it before Claude Code sees anything. That covers an error, no answer in time, or an answer that isn't usable. The request is marked *target refused*, and the conversation stays with Anthropic until its next cold point. ## Set it up ### 1. Run a model Register the model on the inference plane under a [logical model id](../concepts/models-and-engines.md), such as `coder-medium`. It can be: - a model you [install and run locally](../models/installing.md) (llama.cpp or MLX), or - a model served elsewhere, on your own GPU servers for example, registered as a [remote, OpenAI-compatible model](../models/remote.md): its URL and, if it needs one, its key. For subagents and the main conversation, choose a model that follows tool calls well and has a large context: Claude Code's subagent turns carry about 7,000 to 20,000 tokens, and a main conversation grows well beyond that. The gateway reaches the inference plane at `THEYGENT_INFERENCE_PLANE_URL` (`http://127.0.0.1:8081/v1` by default). A gateway on another machine needs that address. ### 2. List it On **AI Gateway → Settings → Local models**, choose **Add a local model** and fill in: | Field | Set it to | |---|---| | Logical id | The id you registered on the inference plane | | Context (tokens) | The context the model is served with: the gateway never sends a larger prompt | | Max output (tokens) | The most it may write in one subagent or main-conversation turn (16,384 unless you change it). Raise it for a large model that writes whole files | | Tool calls | Ticked if it follows tool calls well enough for agent work | | Images | Ticked if it takes images | | Cost | Optional: what running it costs, in USD per million input and output tokens (your hardware, power or rental, spread over what it serves). Leave both empty and its answers count as costing nothing | List models small to large: *smallest that fits* picks the first that can take a request. A very small model for side requests and a medium one for agent work is a common pair. ### 3. Measure it with a trial Set a trial on the dial you want to grow, for example **Subagents: share 0%, trial 20%**, with **Only these agent types** set to `Explore` to start with the simplest subagents. Save. Then read the evidence: - **Settings → Local models** shows, beside each model, the requests it served in the last week and how often it picked the same next action. - **Report → Shadow runs** shows the same per model: how many copies ran, how many answers were usable, the *same next action* rate, and how much it wrote against the model Claude Code asked for. - **Sessions** shows each tried request with *Local trial on …* under it, and the verdict. Trials are samples. A gateway runs one local trial at a time, and a request that finishes while one is running is not tried. A busy team therefore has a fraction of its turns tried. ### 4. Raise the share When the *same next action* rate is one you trust, move the trial into the share: **share 20%, trial 20%** serves the conversations you measured and starts measuring the next ones. Raise it step by step. The Report then counts what the local model served as routed, at no API spend, and shows it against what the same work would have cost on the model Claude Code asked for. ### 5. For the main conversation: update Claude Code's setup Claude Code decides when to compact a conversation from the context window of the model it asked for, which is 1 million tokens for current models behind a gateway. A local model holds far less, so left alone, a long conversation would outgrow it. Once the **Main conversation** dial has a share, the setups on **Setup & keys** also tell Claude Code to compact sooner: at three quarters of the context of the largest local model a main conversation can reach (for example 196,000 tokens for a 262,144-token model). The compaction then runs on the local model too, and the conversation never has to leave it. The Settings tab shows the value under the dial. Developers pick it up by copying their setup again (it adds `CLAUDE_CODE_AUTO_COMPACT_WINDOW` to the shell setup, and `autoCompactWindow` to `settings.json`), or from the managed settings an admin distributes. Claude Code compacts no earlier than 100,000 tokens, so a model for the main conversation needs about 125,000 tokens of context or more; the Setup & keys and Settings tabs say when the listed ones fall short. ## What always stays with Anthropic Some requests never go to a local model, whatever the dials say: - Claude Code's quota check, auto mode's safety checks, side requests that reuse the main conversation's cache, and side-request kinds nobody has reviewed. - **Compaction of a conversation Anthropic serves.** Compaction reads the conversation's cache, so it runs where the conversation runs: a conversation on a local model is compacted by that model. - **Requests that can't be translated faithfully**: documents such as PDFs, Anthropic's server tools (web search, code execution) and their results, and images for a model that doesn't take them. - **Token counting** and **model listing**. The gateway therefore still needs an Anthropic credential: local models extend it, they don't replace it. ## Limits to know - **Answers have time limits.** A local model's answer must start within 15 seconds, plus the time to read the prompt, or Anthropic answers instead. Reading is counted at 100 tokens a second, or at the model's own speed where the gateway has measured it slower: it measures each model on the answers it serves, and a model reads a long conversation much slower than a short one. A long prompt is therefore waited for at least as long as Claude Code itself waits. An answer that opens with a tool call is held back until the call is whole, so Anthropic can still step in: the model may pause no more than 30 seconds while writing it, and the call must be whole within 5 minutes of the request. A large file written steadily is waited for; a model that stalls is not. Once an answer has gone out, Claude Code gets it to the end. A side request's answer is checked whole, so it must be complete within 30 seconds plus the reading time. - **Claude Code waits about five minutes for an answer to start.** Its stream watchdogs give up on a response that sends nothing for that long, and then retry the request. A local model that needs longer to read a prompt (a long conversation on modest hardware, after a compaction for example) causes that: the engine usually finishes reading meanwhile, so the retry is answered quickly from its cache, at no API spend, but the developer waits. To have Claude Code wait instead, raise `CLAUDE_STREAM_IDLE_TIMEOUT_MS` (up to 30 minutes), set `API_FORCE_IDLE_TIMEOUT` to `0`, and raise `API_TIMEOUT_MS` (see Claude Code's [environment variables](https://code.claude.com/docs/en/env-vars)). A faster engine is the better fix: a model that reads a few thousand tokens a second never comes near it. - **Answers have length limits.** A subagent or main-conversation turn can write up to the model's **max output** (16,384 tokens unless you raise it), and a side request up to 4,096. - **Claude Code compacts by the model it asked for**, unless its setup says otherwise: see [step 5](#5-for-the-main-conversation-update-claude-codes-setup). A developer still on an older setup compacts late, and their long conversations move to larger models, or to Anthropic. - **Prompt sizes are estimated.** The gateway counts about 2.75 bytes of Claude Code's request per token, a little more than real sessions use, so it moves a conversation slightly before its model is full rather than after. - **Local compute is priced only if you price it.** Without a **cost**, the Report counts a local answer as costing nothing, and the saving it shows is larger than your real saving if your models run on hardware you pay for. With one, local answers, trials and what routing would cost in observer mode are all counted at it. - **Concurrency is the engine's.** Each developer's requests reach the model as they come. An engine serving one request at a time makes the others wait, and a request that waits past the time limit goes to Anthropic. For a team, serve the model with an engine that handles many requests at once (vLLM is experimental), or give a llama.cpp model several slots with its `parallel` setting ([engine settings](../models/engines.md)). The slots share the context, so raise `ctxSize` with them. ## Send as much as you can to one large model For a team running a large model on its own servers: 1. Register it on the inference plane as a remote model, and list it with its real context and **Tool calls** ticked. Give the main conversation a model with a large context: 128K tokens at the very least, more is better. 2. Measure it first: a trial on **Subagents**, then on **Main conversation**. 3. Set **Subagents** and **Main conversation** to a 100% share, and review the side-request kinds you want local as *local model* in the Catalog. 4. Run in **Router** mode, and have every developer take the updated setup from **Setup & keys** (or distribute the managed settings), so Claude Code compacts while conversations still fit. Every subagent and main-conversation turn then goes to the local model, except what [stays with Anthropic](#what-always-stays-with-anthropic) and what falls back under the [limits above](#limits-to-know). Watch **Report → Calls**, and the *outgrew local model* and *target refused* reasons on the Sessions tab, to see how much still reaches Anthropic, and why. ## Related pages - [Reports & policy](reports.md) — the figures, the reasons, and the rest of the Settings tab. - [Installing local models](../models/installing.md) and [Remote models](../models/remote.md) — getting a model onto the inference plane. - [Engines](../models/engines.md) — engine settings, such as parallel slots. --- Source: https://docs.ertir.dev/latest/ai-gateway/running/ # Running the gateway The AI Gateway is the fourth Ertir service, on port **8082**. It runs the same way as the others — on a machine with `make up`, in Docker Compose, or in Kubernetes — and in one of two [modes](index.md#personal-or-team): **personal** (yours, on loopback, no keys) or **team** (shared, developers identified by gateway keys). ## On your machine `make up` starts the gateway with the other services, configured by your `.env`. As shipped in `.env.example` it is a **personal gateway**: it listens on `127.0.0.1:8082`, forwards your own Claude login or API key, and records from the start, locally. Read what it recorded with the report command: ```bash uv run --package theygent-ai-gateway theygent-ai-gateway report --since 24h ``` `--by class,client,model` (also `session`, `served-model`, `decision`, `reason`, `shadow-model`) picks the groupings, `--since`/`--until` take `30m`, `24h`, `7d`, `2w` or a date, and `--json` prints JSON. It computes the same totals as the Report tab, with the same setup warnings. ### Attach it to the control plane To see a gateway on the **AI Gateway** page, it reports to the control plane with a **service key**: 1. An admin mints one under **AI Gateway → Settings → Service keys → New service key**. It is shown once, beside the line to paste into `.env`. 2. In `.env`, set: ```bash THEYGENT_CONTROL_PLANE_URL=http://127.0.0.1:8080 THEYGENT_AI_GATEWAY_SERVICE_KEY=tygs_… ``` 3. `make restart`. The gateway appears on the **Overview** within one heartbeat (30 seconds). Attached, a gateway forwards without recording until an admin saves a configuration in observer mode — see [Settings](reports.md#settings-admins). ### Serve other machines (team mode) A personal gateway accepts requests without a key, so it refuses to listen beyond loopback. To serve your team from this machine, attach it as above and add: ```bash THEYGENT_AI_GATEWAY_AUTH_MODE=team THEYGENT_AI_GATEWAY_HOST=0.0.0.0 ``` Every request then needs a developer's gateway key; developers create theirs on **Setup & keys** and point Claude Code at `http://:8082`. If the interface is served from another origin than `http://localhost:5174`, add it to `THEYGENT_AI_GATEWAY_ADMIN_ORIGINS` so the Settings tab can reach the gateway. ## Docker Compose The containerized gateway always runs in team mode, so it starts only with a service key. Bring the stack up, mint a service key as an admin, put it in `.env` as `THEYGENT_AI_GATEWAY_SERVICE_KEY`, then: ```bash docker compose --profile gateway up -d ``` It publishes `:8082`, keeps its state in a Docker volume, and waits for open streams to finish when it stops. The organization's key can come from `.env` (`THEYGENT_AI_GATEWAY_ORG_API_KEY`) or from the Settings tab. ## Kubernetes `kubectl apply -k deploy/k8s` creates the gateway with the rest of the stack, and it waits until its Secret exists. Mint a service key as an admin, then: ```bash kubectl -n theygent create secret generic theygent-ai-gateway \ --from-literal=THEYGENT_AI_GATEWAY_SERVICE_KEY=tygs_… \ --from-literal=THEYGENT_AI_GATEWAY_ORG_API_KEY=sk-ant-… # optional ``` The gateway starts on its own once the Secret is there. It scales with load, each replica keeping its own volume (records not yet delivered, the last known configuration and keys) and a stable name, so it stays one instance on the Overview. Because there can be several replicas, the organization's key comes **from the Secret only**: the Settings tab shows it and says so. After changing the Secret, restart the gateway: `kubectl -n theygent rollout restart statefulset/ai-gateway`. For developers outside the cluster, expose the `ai-gateway` Service with an Ingress or a load balancer; in development, `kubectl -n theygent port-forward svc/ai-gateway 8082:8082`. ## The organization's key Developers billed to the organization reach Anthropic with its key, which the gateway adds in place of theirs. The key belongs to the gateway alone — it never passes through the control plane — and it is never shown again once set: - **From the deployment** — `THEYGENT_AI_GATEWAY_ORG_API_KEY` in `.env`, Compose or the Kubernetes Secret. A key set this way cannot be replaced from the page. - **From the page** — a single gateway (on a machine, or in Compose) also accepts it on **AI Gateway → Settings → Organization's Anthropic key**: your browser sends it straight to the gateway, which keeps it in its own state directory. Until a key is set, requests from org-billed developers are refused (`org_key_not_configured`). A personal gateway has no organization key: everyone uses their own credential. ## Health | Endpoint | Answers | |---|---| | `GET /healthz` | Liveness: `{"status":"ok"}` | | `GET /readyz` | `200` when ready, `503` while starting or — in team mode — until developers' keys are loaded; with the gateway's id, mode, configuration version and control-plane address | ```bash curl http://localhost:8082/readyz ``` ## Errors the gateway returns When the gateway itself refuses a request, Claude Code shows the message, and the response carries an `x-theygent-gateway-error` header naming the reason. Anything else — Anthropic's own errors, rate limits — comes back exactly as Anthropic sent it. | Status | Code | What to do | |---|---|---| | 401 | `gateway_key_missing` | The gateway is in team mode: add your gateway key to the setup ([Set up Claude Code](setup.md)) | | 401 | `gateway_key_invalid` | The key is unknown or revoked: create a new one on Setup & keys | | 503 | `gateway_keys_unavailable` | The gateway has not loaded developers' keys yet — it is starting, or cannot reach the control plane and has none saved. Retry shortly; check its service key | | 401 | `upstream_credential_missing` | You are billed to your own credential and Claude Code sent none: sign in to Claude Code, or set your API key | | 503 | `org_key_not_configured` | You are billed to the organization, and the gateway has no organization key: an admin sets it | | 403 | `endpoint_not_allowed` | The organization's key serves Claude Code's model endpoints only | | 504 | `upstream_timeout` | Anthropic did not answer in time (retryable) | | 502 | `upstream_unreachable` | The gateway could not reach Anthropic: check its network | ## Where it keeps its data A gateway keeps a small state directory — `~/.theygent/ai-gateway` on a machine, a volume in Compose and Kubernetes (`THEYGENT_AI_GATEWAY_STATE_DIR` moves it): the records it has not delivered yet (so nothing is lost while the control plane is down), the last known configuration, developers' keys as one-way digests (never the keys themselves), and a stored organization key. Every file is readable by its owner only. The control plane keeps each request in detail for 90 days by default (**AI Gateway → Settings → Request records**; 0 keeps everything). After that, a whole day at a time moves to a compressed archive in the same database. Reports read archived days from daily totals that keep every grouping, so no figure changes. Only those days' per-request timelines leave the Sessions tab, and the card lists each archived day for download. A day's records arrive as JSON lines, one record each, exactly as the gateway sent it. ## Related pages - [Environment variables](../reference/environment.md#ai-gateway) — every gateway setting. - [Set up Claude Code](setup.md) — the client side. - [Running Ertir](../getting-started/running.md) — the other services. --- Source: https://docs.ertir.dev/latest/import-export/ # Import & export **Settings → Import / Export** moves an install — or any part of it — between machines as a single zip file. Export packs your agents, runs, traces, chats, MCP servers, RAG sources, and model registrations into one download; import unpacks a bundle on another machine, shows you a preview of exactly what it will create, and only writes after you confirm. Two things never travel in a bundle, by design: **secret values** and **model weights** — the import preview tells you what has to be re-entered or re-downloaded on the target. You can also export and import a **single agent** as a plain JSON file, straight from the Agents page — see [Drafts & publishing](../building/saving-agents.md#exporting-and-deleting-agents). ## What's in a bundle The Export card offers seven sections; each maps to a slice of your install: | Section | What travels | |---|---| | **Agents** | Every published agent with **all** its versions (graph + canvas layout), plus your drafts, triggers, and per-agent I/O capture policies. | | **Runs** | The full run history — status, model, parameters, output, timestamps. | | **Traces** | Per-run spans and node I/O, plus the audio/image artifacts your runs produced (packed as raw files in the zip). Selecting Traces automatically includes Runs — a trace without its runs would be unreadable. | | **Chats** | Sessions and their messages, in order. | | **MCP servers** | Connections and name-keyed MCP server registrations — definitions only. | | **RAG sources** | Source definitions (name, kind, embedding model, crawl config) — no documents or vectors. | | **Model registry** | Your model registrations (logical id → binding), the catalog-install provenance needed to re-download weights, and credential **names**. | !!! warning "What is never in a bundle" Bundles carry definitions, never credentials or weights. Specifically excluded, always: - **Secret values of any kind** — connection secrets, webhook signing secrets, and MCP env/header *values* (only the key names travel, so the target knows what to ask you for). - **API credential values** — the inference plane exports credential *names* only (e.g. `HF_TOKEN`), never values. - **Model weights** — registrations re-download their weights on the target, or point at weights you reinstall yourself. - **RAG content** — documents, chunks, and vectors stay behind; sources re-ingest on the target. There is no way to opt secrets or weights into a bundle. An export zip is safe to store or hand over in the sense that it contains no credential material — but it does contain your prompts, run outputs, and chat history, so treat it with the same care as the install itself. ## Exporting 1. Open **Settings** from the sidebar and switch to the **Import / Export** tab. 2. In the **Export** card, tick the sections you want — **All** is pre-selected. Ticking **Traces** pulls **Runs** along automatically (and unticking Runs drops Traces). 3. Click **Export**. Your browser downloads one file, named like `theygent-export-2026-07-18.zip`. If an artifact referenced by a trace no longer exists on disk, the export skips it and tells you in a toast — the zip still completes. !!! note "The zip is assembled in your browser" The browser fetches the control-plane sections from `:8080` and the model registry **directly** from your inference plane on `:8081`, then zips them together locally. Registry state never passes through the control plane — the browser is the only place both halves meet, which is why the export lives in the interface rather than behind a single server endpoint. The curl equivalents fetch the two halves separately. The control plane serves its bundle from `POST /export` (pick any subset of `agents`, `runs`, `traces`, `sessions`, `mcp`, `rag` — an empty or unknown selection is `400 invalid_include`): ```bash curl -X POST http://localhost:8080/export \ -H 'Content-Type: application/json' \ -d '{"include": ["agents", "runs", "traces", "sessions", "mcp", "rag"]}' \ -o control-plane.json ``` The inference plane serves its registry bundle from `GET /admin/export`: ```bash curl http://localhost:8081/admin/export -o models.json ``` ## Importing 1. In the **Import** card, click **Choose file…** and pick a theygent export zip — or a single exported agent `.json`. 2. Read the **preview**: per-section counts plus prominent warnings — how many models will download weights from Hugging Face, how many connections will need credentials re-entered or re-authorized, which MCP servers need env values, which RAG sources need re-crawling or re-uploading, and the credential **names** the source inference plane had (values never travel — re-enter them on the target). **Nothing is written at this point.** 3. Click **Import…** and confirm in the dialog. 4. The report shows what was created, what was skipped, and every warning. Any model weight downloads the import kicked off appear as progress cards in the notification center (bottom-right), exactly like a catalog install. ### Import rules - **Existing entities are kept, never overwritten.** Anything whose id (or name, for MCP servers) already exists on the target is skipped. That makes an import **safe to re-run** — importing the same bundle twice changes nothing the second time. - **Content hashes are recomputed by the server.** An imported agent version is validated and re-hashed exactly like a publish; the hash in the file is never trusted. If the target already has the same agent version with *different* content, that version is reported as a `version_conflict` and skipped — the rest of the bundle still imports. - **One bad entry never aborts the rest.** An invalid agent version, an unparseable row — each is reported in the warnings list while everything else lands. - **Ids and timestamps are preserved**, so runs, traces, and chats on the new machine correlate exactly like the originals. ### After an import Because secrets and weights never travel, some things need your attention on the target: - **Connections** import without their secret — re-enter it (or re-run the OAuth authorization) on the [MCP page](../mcp/index.md) before the connection works. - **Webhook triggers** import **disabled** and without a signing secret. Set a new secret (`PATCH /triggers/{id}`), then enable the trigger. Schedule triggers import as they were. - **MCP servers** import with their env/header values empty — open the server's settings and re-enter them. - **RAG sources** import empty: re-run the crawl, or re-upload the documents. - **Model weights** re-download automatically for catalog-installed models (watch the notification center). A model that was registered against a local file path with no install provenance is registered anyway but warns `weights_unavailable` — it will fail at first use until you put weights at that path or reinstall it. See [Moving models to another machine](../models/index.md#moving-models-to-another-machine). The curl equivalents import the two JSON halves the zip contains (the zip itself has no single server endpoint — the browser unpacks it). A body that isn't a bundle is `400 invalid_bundle`; a bundle from a future Ertir is `400 unsupported_bundle_version`: ```bash curl -X POST http://localhost:8080/import \ -H 'Content-Type: application/json' \ --data-binary @control-plane.json ``` ```bash curl -X POST http://localhost:8081/admin/import \ -H 'Content-Type: application/json' \ --data-binary @models.json ``` Artifacts (the raw files under `artifacts/` in the zip) restore under their original refs with `PUT /artifacts/{ref}` — a ref that already exists on the target is kept, never overwritten (the response says `created: false`): ```bash curl -X PUT http://localhost:8080/artifacts/art_01J9ZS3NDEKTSV4RRFFQ69G5FA \ -H 'Content-Type: audio/webm' \ --data-binary @art_01J9ZS3NDEKTSV4RRFFQ69G5FA ``` ## Related pages - [Drafts & publishing](../building/saving-agents.md) — per-agent Export and Delete on the Agents page - [Registries](../models/index.md) — the model registry the inference half of a bundle covers - [MCP servers](../mcp/index.md) — re-entering connection secrets and env values after an import - [RAG sources](../rag/index.md) — re-ingesting an imported source - [API reference](../reference/api.md) — `/export`, `/import`, `/admin/export`, `/admin/import` - [Troubleshooting](../reference/troubleshooting.md) — the import/export error codes --- Source: https://docs.ertir.dev/latest/reference/api/ # API reference A quick reference to the HTTP endpoints you are most likely to call directly — for scripting, automation, or debugging. Ertir has **three services** with an API, each with its own base URL: - **Control plane** — `http://localhost:8080` — agents, runs, sessions, triggers, connections, artifacts, observability, and the AI gateway's configuration and reports. - **Inference plane** — `http://localhost:8081` — the OpenAI-compatible data plane (`/v1/*`) and model management (`/admin/*`). - **AI gateway** — `http://localhost:8082` — Claude Code's Anthropic-format traffic, plus a small admin surface. The interface uses these same endpoints, so anything you can do in the UI you can do here. This page lists the user-facing surface; it is not the full internal API. !!! note "Wire conventions" Control-plane **request bodies and stored run/session records use `snake_case`** (`session_id`, `content_hash`). The immediate run **result** envelope and streaming (SSE) frames use `camelCase` (`runId`). The **inference plane** (`/v1/*` and `/admin/*`) is `camelCase` throughout. The graph document (`ir`) is always `camelCase`. Keep them straight when you assemble a body. !!! warning "Authentication" Every control-plane endpoint below requires a bearer token — `Authorization: Bearer ` — where the token is either your interactive session or, for scripts and your own frontends, an **API key** minted from your profile. Endpoints are gated by role (viewer / editor / admin); see [Users, roles & sign-in](../running/users.md). Three surfaces differ: `/healthz` + `/readyz` are open, webhook delivery `POST /hooks/{id}` carries an HMAC signature instead, and `POST /agents/{id}/invoke` accepts an API key or the shared [`THEYGENT_INVOKE_TOKEN`](environment.md). The inference plane is a separate, locally-trusted service — still do not expose either plane to an untrusted network. --- ## Control plane ### Health | Method | Path | Purpose | |---|---|---| | GET | `/healthz` | Liveness — is the process up? Returns `{"status":"ok"}`. | | GET | `/readyz` | Readiness — 200 when ready, 503 `not-ready` naming the down dependency (database or inference plane). | ### Agents Published agents are immutable, content-addressed versions. See [Drafts & publishing](../building/saving-agents.md) and [Agent versioning](../concepts/versioning.md). | Method | Path | Purpose | |---|---|---| | POST | `/agents` | Publish a new agent from an IR document. Body `{ir, name?}`. `201`; reusing an existing id → `409 agent_exists`. | | GET | `/agents` | List agents, newest first (`limit` 1–200 default 50, `before` cursor). | | GET | `/agents/{id}` | One agent with its versions, newest first. | | POST | `/agents/{id}/versions` | Publish a new version. Identical content under an existing version is an idempotent `200`; different content under the same version → `409 version_conflict`. | | GET | `/agents/{id}/versions/{version}` | The full stored IR + canvas layout for one version. | | GET / PUT | `/agents/{id}/io-policy` | Read/set the per-agent I/O capture policy. PUT body `{io_capture, io_retention_seconds?, redact_rules?}`; `io_retention_seconds` is a positive number of seconds after which captured payloads are deleted (omit it to follow the platform's `telemetry.io_retention_days`). Setting it never changes the content hash. | | DELETE | `/agents/{id}` | Delete the whole agent: **all** its versions, its triggers (schedules stop firing), and its I/O policy, in one step. `204`; unknown id → `404 agent_not_found`. Past runs, chats, and drafts survive — they keep the agent id and content hash as breadcrumbs. | Individual versions are immutable — there is no endpoint to edit or delete a *single* version. Deleting the agent removes every version at once; there is no delete for runs. ### Samples The shipped example-agent catalog. Installing publishes an ordinary agent (same registry, same gate as `POST /agents`) with your chosen models stamped into its bindings, creating any MCP connections the sample needs. See [Sample agents](../samples/index.md). | Method | Path | Purpose | |---|---|---| | GET | `/samples` | The catalog: title, capabilities, model slots, connections, and per-sample `installed` state. Any signed-in role. | | POST | `/samples/install` | Install by catalog id. Body `{ids, models: {slot: {logical_id, binding}}}`. Editor role. Unfilled slots → `400 sample_models_required`, an illegal binding → `400 sample_model_invalid` (nothing written in either case); unknown id → `404 sample_not_found`. Beyond agents (children first for composition samples), an install creates the sample's connections (MCP servers, or a provider's token connection with no token yet) and RAG sources (reused by name on reinstall) and registers any shipped trigger **disabled**. The response `report` carries one entry per sample — `installed`, `already_installed`, or `error` (that sample rolled back and persisted nothing; the rest of the batch still ran) — with `connections`, `rag_sources`, and `warnings` (degraded-install notes, e.g. seed ingest could not start). Reinstalling after deleting only some of a sample's agents repairs the missing pieces. | ### Drafts The editor's autosave tier: mutable work-in-progress graphs. A draft is **never validated as a graph** (a half-wired document must still save; only "the `ir` is a JSON object" is enforced → `400 invalid_draft`) and never hashed. The canvas layout (`view`) is stored alongside the document. See [Drafts & publishing](../building/saving-agents.md). | Method | Path | Purpose | |---|---|---| | POST | `/drafts` | Create a draft. Body `{ir, agent_id?}` — `agent_id` links the draft to the published agent it edits (optional, immutable after create). `201`. | | PUT | `/drafts/{id}` | Replace the draft's document (the autosave write). Body `{ir}`. | | GET | `/drafts` | List drafts, most recently **edited** first (`limit` 1–200 default 50, `before` cursor, optional `agent_id` filter). Summaries only — no `ir`/`view`. | | GET | `/drafts/{id}` | The full draft, `ir` + `view` — what the editor reloads. | | DELETE | `/drafts/{id}` | Discard a draft. `204`. | Unknown ids are `404 draft_not_found`. Unlike versions, drafts are freely deletable — the editor deletes the working draft itself after a successful publish. ### Runs Statuses: `created → streaming → waiting → completed | failed`. See [Runs & sessions](../concepts/runs-and-sessions.md). | Method | Path | Purpose | |---|---|---| | POST | `/runs` | Plain prompt run. Body `{input, model, params?, stream?, session_id?}`. `model` is a logical id. Streams SSE by default. | | POST | `/graphs/runs` | Run an inline IR graph — no publish needed (this is what the editor's Test panel uses). Body `{ir, input, stream?, session_id?}`. Invalid IR → `400 invalid_ir` (no run row created). | | POST | `/agents/{id}/runs` | Run a published agent by reference. Body `{input, version?, content_hash?, session_id?, stream?}`. | | POST | `/agents/{id}/invoke` | Token-authed, non-interactive sibling of the above. Same body but `stream` defaults **false**. Requires a bearer token. | | POST | `/agents/{id}/durable-runs` | Run a published agent on the durable runtime (the only way to run `human`/`subgraph`/`loop`/`map` agents). Returns `202 {run_id}`; poll `GET /runs/{id}`. Needs [durable mode](../running/durable.md). | | GET | `/runs` | List runs, newest first (`limit`, `before`). | | GET | `/runs/{id}` | Run detail, including `output`, `error`, `content_hash`, `trigger_id`, `awaiting_node`, `completed_at`. | | POST | `/runs/{id}/resume` | Deliver input to a run paused (`waiting`) at a `human` node. Body `{input}`. Returns `202`. | **Start a streaming prompt run:** ```bash curl -N -X POST http://localhost:8080/runs \ -H 'Content-Type: application/json' \ -d '{"input": "Summarize the theory of relativity.", "model": "triage-fast", "stream": true}' ``` The stream is Server-Sent Events: `run` (status frames), `delta` (answer tokens), `reasoning` (model thinking, kept out of the output), terminated by `data: [DONE]`. Set `"stream": false` for a single JSON response `{runId, status, output}`. **Run a published agent (non-streaming) with a session:** ```bash curl -X POST http://localhost:8080/agents/agent.support/runs \ -H 'Content-Type: application/json' \ -d '{"input": {"question": "Where is my order?"}, "session_id": "ses_abc", "stream": false}' ``` **Invoke an agent with the invoke token:** ```bash curl -X POST http://localhost:8080/agents/agent.support/invoke \ -H 'Authorization: Bearer YOUR_INVOKE_TOKEN' \ -H 'Content-Type: application/json' \ -d '{"input": {"question": "Where is my order?"}}' ``` **Resume a waiting run:** ```bash curl -X POST http://localhost:8080/runs/RUN_ID/resume \ -H 'Content-Type: application/json' \ -d '{"input": {"approved": true}}' ``` ### Observability Per-run trace data. See [Observability](../running/observability.md). | Method | Path | Purpose | |---|---|---| | GET | `/runs/{id}/trace` | The persisted span tree (timing, status, attribution) — no payloads. | | GET | `/runs/{id}/trace/stream` | Live SSE overlay: events `span.open`, `span.close`, `done`; `: keepalive` comment every 15s. | | GET | `/runs/{id}/nodes/{node_id}/io` | Lazy per-node input/output payloads, gated by the capture policy (a gated state is a `reason`, never an error). | ### Triggers & webhooks A trigger deploys a published, pinned agent behind an unattended entry point. See [Triggers & webhooks](../running/triggers.md). | Method | Path | Purpose | |---|---|---| | POST | `/triggers` | Create a trigger. Body `{agent_id, kind, version?|content_hash?, config, enabled?}`. Pin exactly one of `version`/`content_hash`. | | GET | `/triggers` | List triggers (webhook secrets redacted to `"***"`). | | GET / PATCH / DELETE | `/triggers/{id}` | Read / update (`enabled`, `config` only — kind and pin are immutable) / delete a trigger. | | POST | `/hooks/{id}` | Fire a `webhook` trigger. The raw JSON body is the run input; requires the `X-TheYgent-Signature` header. Returns `202`. | Trigger `config` keys: `schedule` needs `config.cron` (a valid cron expression) and takes optional `config.input`; `webhook` needs `config.secret` (the inbound signing secret). **Fire a webhook** — the signature is the HMAC-SHA256 of the *raw* request body keyed by the trigger's secret: ```bash BODY='{"question":"Where is my order?"}' SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "YOUR_WEBHOOK_SECRET" | awk '{print $2}') curl -X POST http://localhost:8080/hooks/TRIGGER_ID \ -H "X-TheYgent-Signature: sha256=$SIG" \ -H 'Content-Type: application/json' \ -d "$BODY" ``` A bad or missing signature → `401 invalid_signature`; a disabled trigger → `409 trigger_disabled`. ### Sessions Opt-in conversational memory. See [Runs & sessions](../concepts/runs-and-sessions.md). | Method | Path | Purpose | |---|---|---| | GET | `/sessions` | List sessions, newest activity first. | | GET | `/sessions/{id}` | Session detail with messages in order. | | POST | `/sessions` | Create/ensure a session. Body `{id?, metadata?}`; a missing id is minted server-side. | | POST | `/sessions/{id}/turns` | Append a `{user_content, assistant_content}` pair (both non-blank). | | DELETE | `/sessions/{id}` | Delete the session and its messages; its runs are detached, not deleted. | ### Connections Named tool/MCP credentials. The `secret` is write-only and never returned. See [Tools](../building/nodes/tools.md) and [MCP servers](../mcp/index.md). | Method | Path | Purpose | |---|---|---| | POST | `/connections` | Create. Body `{name, kind, config, secret?, enabled?}`. `kind` is `http_auth` or `mcp_server`. | | GET | `/connections` | List (responses expose `hasSecret` only; secret-ish config keys redacted). | | GET / PATCH / DELETE | `/connections/{id}` | Read / update (a non-empty `secret` rotates in place without changing any content hash) / delete. | ### Crawlers The Crawlers browser's read path: the provider's public store, proxied and cached for five minutes, and an Actor's input schema converted into a `crawler` node config. Editor role. Nothing is created — installing is the editor dropping the returned config on the canvas. `provider` defaults to `apify`, the only one; another value is `400 crawler_provider_unknown`. See [Crawler](../building/nodes/crawler.md). | Method | Path | Purpose | |---|---|---| | GET | `/crawlers/catalog` | Search the store. Query `search?`, `category?`, `sort_by?` (`popularity` · `newest` · `relevance`), `limit?` (≤ 100), `offset?`. Returns `{entries, total, offset, limit}`; each entry carries `actor`, `title`, `author`, `categories`, `pricing {model, priceUsd, unit}`, `users30d`, `runs30d`, `successRate30d`. The store unreachable → `502 crawler_catalog_error`. | | GET | `/crawlers/catalog/actor?actor=…` | One Actor: its `entry`, `readmeSummary`, `defaultRunOptions`, its input `fields` (form-ready, each marked `exposable` when a model could fill it), and `suggested` — a complete node config plus `unfillable` required fields and a `suggestedId`. An id that isn't `username/actor-name` → `400 invalid_actor`; an unknown Actor → `404 crawler_not_found`; no input schema → `404 crawler_schema_unavailable`. | | POST | `/crawlers/catalog/convert` | Re-derive a node config for a choice of exposed fields. Body `{actor, provider?, expose?, scalar?, input?, description?, connection?, limits?}` (camelCase, like the config). Returns the same shape as `suggested`; `parameterSchema` always comes from the Actor's real input schema. | ### RAG sources Retrieval collections agents search through the rag node. See [RAG sources](../rag/index.md). | Method | Path | Purpose | |---|---|---| | POST | `/rag/sources` | Create. Body `{name, kind, embedding_model, config}`. `kind` is `upload` or `crawl`; for `crawl`, `config` carries `root_url` (required), `max_pages?`, `render_js?`. `embedding_model` is a logical id, never an engine name. Creating a `crawl` source does not start the crawl — call `:ingest`. | | GET | `/rag/sources` | List (keyset pagination via `limit` + `before`). | | GET / PATCH / DELETE | `/rag/sources/{id}` | Read (status + live ingest `progress`) / rename or edit crawl config / delete with all documents and vectors. | | GET | `/rag/sources/{id}/documents` | The source's documents, oldest first, with per-document status, chunk counts, and errors. Paged: `limit` (1–500, default 500) and `after`. The response is `{documents, next_after}`; pass `next_after` as `after` for the next page, and `next_after: null` marks the last page. An `after` that names no document of the source is a `400 invalid_rag_cursor`. | | POST | `/rag/sources/{id}/documents?filename=…` | Upload one document as a raw body (the `Content-Type` header is the file's mime). The size cap is the `rag.max_upload_bytes` setting, 50 MiB by default; a larger body is a `413 document_too_large`, refused from its `Content-Length` before the body is read. Returns `202`; poll the source for progress. | | POST | `/rag/sources/{id}:ingest` | Start (or re-run) the crawl. `202`; unchanged pages are hash-skipped. `409` while an ingest is already running. | | POST | `/rag/sources/{id}:cancel` | Cancel the in-flight ingest (a no-op on a settled source). | | POST | `/rag/sources/{id}/query` | Run the same hybrid retrieval a rag node runs. Body `{query, top_k?, min_similarity?}`; returns the match list. | ```bash curl -X POST "http://localhost:8080/rag/sources/rag_01ABC/documents?filename=handbook.pdf" \ -H 'Content-Type: application/pdf' \ --data-binary @handbook.pdf ``` ### Artifacts Audio/image blobs by reference. See [Voice](../chat/voice.md). | Method | Path | Purpose | |---|---|---| | POST | `/artifacts` | Upload raw bytes (the `Content-Type` header is the stored mime). Returns `201 {ref, contentType, bytes}`. | | GET | `/artifacts/{ref}` | Fetch a stored `art_…` blob (only `art_*` ids are served). | | PUT | `/artifacts/{ref}` | Store raw bytes under a ref you supply — the restore path an [import](../import-export/index.md) uses so refs embedded in imported runs keep resolving. The ref must be a full `art_` + 26-character id exactly as `POST /artifacts` mints them (`400 invalid_artifact_ref` otherwise). An existing ref is never overwritten: a fresh write returns `201 {ref, contentType, bytes, created: true}`, a ref that already exists returns `200` with the *stored* artifact's metadata and `created: false`. | ```bash curl -X POST http://localhost:8080/artifacts \ -H 'Content-Type: audio/webm' \ --data-binary @recording.webm ``` ### MCP servers Register external tool servers. See [MCP servers](../mcp/index.md) and [MCP tools](../building/nodes/mcp.md). Name-keyed registrations (plain stdio/remote servers, no secrets — openapi/graphql must be created as connections instead): | Method | Path | Purpose | |---|---|---| | PUT / GET / DELETE | `/admin/mcp/servers/{name}` | Register / read / remove a server config. | | GET | `/admin/mcp/servers` | List registered servers (env **values** are never echoed). | | GET | `/admin/mcp/servers/{name}/tools` | List the server's tools (lazy-connects; unreachable → `503 mcp_unavailable`). | | POST | `/admin/mcp/servers/{name}:warm` / `:close` | Start / stop the server process. | Connection-backed servers (everything the MCP page creates — hub installs, remote servers with auth, OpenAPI/GraphQL servers) get the same operational surface through their connection id: | Method | Path | Purpose | |---|---|---| | GET | `/connections/{id}/mcp/tools` | List the server's tools through the connection (lazy-connects; unreachable → `503 mcp_unavailable`). | | POST | `/connections/{id}/mcp:warm` / `:close` | Connect the server now / tear down the live connection (the connection row is kept). | | POST | `/connections/{id}/mcp-oauth:start` | Begin interactive authorization for an `auth.type: oauth` connection. Returns `{status, authorizationUrl?}` — open the URL in a browser; the provider redirects back to `/mcp/oauth/callback`. | | GET | `/connections/{id}/mcp-oauth` | Authorization status: `{authorized, pending, lastError, connected}`. Tokens are never returned. | ### MCP hubs (catalog) Browse MCP registries and install published servers. Bodies and responses on this surface are `camelCase`. See [MCP servers](../mcp/index.md). | Method | Path | Purpose | |---|---|---| | GET | `/admin/mcp/registries` | The browsable registries: the two built-ins plus any from [`THEYGENT_MCP_REGISTRIES`](environment.md). | | GET | `/admin/mcp/catalog` | One page of a registry's servers (`registry`, `search`, `limit` 1–100, `cursor`); entries carry installed-state stamps. A registry that can't be reached → `502 mcp_registry_error`. | | GET | `/admin/mcp/catalog/entry` | One server (`registry`, `name`, `version` default `latest`) with its install candidates and their declared inputs. | | POST | `/admin/mcp/catalog/install` | Install a candidate as an `mcp_server` connection. Body `{registry, name, version?, candidateId, connectionName, values, useOauth?}`. Secret-flagged values go to the encrypted store server-side. Returns `201` with the connection. | | POST | `/admin/mcp/generated:preview` | The tools an OpenAPI/GraphQL server *would* derive — nothing is created. Body `{kind, spec?|specUrl?, url?, allowMutations?}`. | ### Bench store The Bench records results and presets here; testing itself runs against the inference plane. See [The Bench](../running/bench.md). | Method | Path | Purpose | |---|---|---| | POST / GET | `/bench/runs` | Record / list bench results (filter by `logical_id`, `agent_id`, `suite_id`, `case_id`). | | GET | `/bench/compare?a=&b=` | Per-metric deltas between two saved results. | | POST / GET / DELETE | `/bench/presets` | Save / list / delete named param presets. | | POST / GET | `/bench/suites` | Store / list golden-case suites (API-only; no UI yet). | ### AI gateway What the [AI Gateway](../ai-gateway/index.md) page uses. Everyone manages their own gateway keys and reads their own records; admins manage everything else. Request bodies and answers are `snake_case`. | Method | Path | Purpose | |---|---|---| | GET | `/gateway/me` | Your billing profile (`passthrough` or `org_key`) and live gateway-key count. | | POST / GET | `/gateway/keys` | Create a gateway key (`{name}`; the `tyg_…` token is in this response only) / list yours (`?all=true` for everyone's, admin). | | DELETE | `/gateway/keys/{id}` | Revoke a key; gateways refuse it within one key sync. | | GET | `/gateway/report` | Totals and groupings for a range: `since`, `until`, `by` (`class`, `client`, `session`, `model`, `served-model`, `developer`, `decision`, `reason`, `shadow-model`, `day`), `scope` (`mine` or `org`), plus setup `warnings` and subscription `seats`. | | GET | `/gateway/sessions` · `/gateway/sessions/{id}` | Sessions most recently active first (`since`, `before`, `limit`) · one session's timeline of requests and decisions (`after`, `limit`). | | GET / PUT | `/gateway/config` | The latest configuration (its version is the `ETag`; `404 gateway_config_not_found` before any) / save the next version (admin; `If-Match` refuses a stale edit with `412 config_version_conflict`; `422 invalid_gateway_config`). | | GET | `/gateway/config/versions` · `/gateway/config/versions/{n}` | The history · one version. | | GET / PUT | `/gateway/fingerprints` · `/gateway/fingerprints/{fingerprint}` | The side-request catalog · review one kind (admin; saves a new configuration version). | | GET | `/gateway/instances` | Each gateway's latest heartbeat (offline after 120 s). | | GET / PUT | `/gateway/developers` · `/gateway/developers/{user_id}` | Every account's billing (admin) · set one: `{"credential_profile": "org_key"}` (or `"passthrough"`). | | POST / GET / DELETE | `/gateway/service-keys` · `/gateway/service-keys/{id}` | Gateways' service keys (admin; the `tygs_…` token is in the create response only). | | GET | `/gateway/archive` · `/gateway/archive/{day}` | The archived days, with their request counts and sizes · one day's records as gzip-compressed JSON lines, exactly as the gateway sent them (admin; `404 archive_not_found`, `400 invalid_day`). | ```bash curl "http://localhost:8080/gateway/report?since=2026-10-01T00:00:00Z&by=day,model" \ -H "Authorization: Bearer tyk_your-api-key" ``` ### Import & export The control-plane half of the transfer bundle — the Settings **Import / Export** tab drives these plus the inference plane's pair below. See [Import & export](../import-export/index.md). | Method | Path | Purpose | |---|---|---| | POST | `/export` | Build a bundle. Body `{"include": [...]}` — any non-empty subset of `agents`, `runs`, `traces`, `sessions`, `mcp`, `rag` (`traces` implies `runs`); anything else → `400 invalid_include`. Returns the bundle JSON. **Never contains secret values, secret refs, webhook signing secrets, or MCP env/header values** (key names only). | | POST | `/import` | Apply a bundle (the same shape; sections optional). Id-preserving, idempotent, skip-on-exists; one bad entry never aborts the rest. Returns `{report}` with per-section counts and a flat `warnings` list. Not a bundle → `400 invalid_bundle`; a future `format_version` → `400 unsupported_bundle_version`. | ```bash curl -X POST http://localhost:8080/export \ -H 'Content-Type: application/json' \ -d '{"include": ["agents", "sessions"]}' -o control-plane.json curl -X POST http://localhost:8080/import \ -H 'Content-Type: application/json' \ --data-binary @control-plane.json ``` --- ## Inference plane The data plane is OpenAI-compatible. In every `/v1/*` call, the `model` field is a **logical id** you registered — never an engine name. See [Models & engines](../concepts/models-and-engines.md). ### Data plane (`/v1/*`) | Method | Path | Purpose | |---|---|---| | GET | `/v1/models` | List registered logical ids as OpenAI model objects. | | POST | `/v1/chat/completions` | Chat (and vision — `messages` content may be text + `image_url` parts). Streaming and non-streaming. | | POST | `/v1/embeddings` | Embeddings. `input` is a string or list of strings. | | POST | `/v1/audio/transcriptions` | Speech-to-text. Multipart form with `model` + `file`. | | POST | `/v1/audio/speech` | Text-to-speech. `{model, input, voice?}`; returns raw audio bytes. | | POST | `/v1/images/generations` | Text-to-image. `{model, prompt, size?, n?, steps?}`; returns `{data:[{b64_json}]}`. | ```bash curl http://localhost:8081/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model": "triage-fast", "messages": [{"role":"user","content":"Hello"}]}' ``` ### Model management (`/admin/*`) | Method | Path | Purpose | |---|---|---| | PUT | `/admin/models/{id}` | Register/replace a model under a logical id. | | GET | `/admin/models` | List registered models with live state. | | GET / DELETE | `/admin/models/{id}` | Read one / unregister (evicts then deletes). | | GET | `/admin/models/{id}/capabilities` | Probe tool-calling, vision, reasoning, max context, modalities. | | POST | `/admin/models/{id}:warm` / `:evict` | Pre-load / free the engine. | | GET | `/admin/models/{id}/logs?lines=200` | The tail of a managed model's engine log: `{logicalId, path, lines}`. `404 engine_log_not_found` for a reachable model or one whose engine hasn't run. | | GET | `/admin/engines` | Currently resident engines and `maxResident`. | | GET | `/admin/credentials` | List local credential names with `hasValue` (values are never read back). | | PUT / DELETE | `/admin/credentials/{name}` | Set / remove a local credential (values are write-only). | | GET | `/admin/catalog/models` | Browse installable models (`search`, `sort`, `limit`, `engines`, `size`, `modality`). | | GET | `/admin/catalog/models/{repo}` | Model detail with per-quant variants and RAM fit badges. | | POST | `/admin/catalog/install` | Start a download+install job. Body `{repo, engine, variantId, logicalId}`. | | GET | `/admin/catalog/downloads` / `/{job_id}` | Track download progress. | | POST | `/admin/catalog/downloads/{job_id}:cancel` | Cancel an in-flight install. | | GET | `/admin/export` | The registry as a bundle: every registration verbatim (runtime state stripped), catalog-install provenance, and credential **names** only — never values or weights. See [Import & export](../import-export/index.md). | | POST | `/admin/import` | Apply a registry bundle: already-registered ids are skipped; catalog-installed models re-download their weights in-plane (returned in `downloads`); a local-path model without install provenance registers with a `weights_unavailable` warning. Malformed body → `400 invalid_bundle`. | **Register a hosted, OpenAI-compatible model:** ```bash curl -X PUT http://localhost:8081/admin/models/claude-fast \ -H 'Content-Type: application/json' \ -d '{"binding":"openai-compatible","baseUrl":"https://api.example.com/v1","model":"provider-model-name","credentialRef":"secret://MY_API_KEY"}' ``` See [Remote models](../models/remote.md) for the reachable-binding fields. ### Health | Method | Path | Purpose | |---|---|---| | GET | `/healthz` | Liveness. | | GET | `/readyz` | Per-`(engine, modality)` readiness — each slot with a `reason` and install hint when not ready. Stays 200 while any engine is available. | ```bash curl http://localhost:8081/readyz ``` ## AI gateway Every path except the gateway's own is forwarded to the Anthropic API exactly as sent — `POST /v1/messages`, `count_tokens`, `GET /v1/models` and whatever Claude Code adds — and answered exactly as Anthropic answers. In team mode each request needs a gateway key (`X-TheYgent-Gateway-Key: tyg_…`, or the key as `Authorization: Bearer` for developers billed to the organization). The gateway's own refusals carry an `x-theygent-gateway-error` header; see [errors the gateway returns](../ai-gateway/running.md#errors-the-gateway-returns). Its own surface is `camelCase`: | Method | Path | Purpose | |---|---|---| | GET | `/healthz` | Liveness. | | GET | `/readyz` | `200` when ready; `503` while starting or until developers' keys are loaded (team mode). Names the gateway id, mode and configuration version. | | GET | `/admin/status` | Identity, mode, configuration and key-set versions, the request log (recorded, exported, waiting), upstream health. | | GET | `/admin/credentials` | Whether the organization's key and the service key are set, and where the org key came from — never a value. `readOnly` says why the page may not write it: `env`, `personal_mode` or `deployment_only`. | | PUT / DELETE | `/admin/credentials/org-api-key` | Set (`{"api_key": "…"}`) / remove the organization's key on this gateway. `409 org_key_pinned`, `org_key_not_applicable` or `org_key_deployment_only` when it cannot; `422 invalid_api_key`. | The `/admin/*` paths take your Ertir bearer (a session or a personal API key) and admit admins only — the gateway asks the control plane, and answers `503` if it cannot. A personal gateway with no control plane leaves them open, on loopback. ## Related pages - [Environment variables](environment.md) — the `THEYGENT_*` knobs these services read. - [Troubleshooting](troubleshooting.md) — what the error codes mean and how to fix them. - [Glossary](glossary.md) — terms used above (logical id, binding, run, trigger, artifact). --- Source: https://docs.ertir.dev/latest/reference/environment/ # Environment variables Every setting Ertir reads from the environment. You configure these in the `.env` file at the repository root, which `make up` and the services load automatically. Only `DATABASE_URL` is required — everything else has a working default for a local install. !!! tip "Where to set them" Copy `.env.example` to `.env` and edit it. The `Makefile` includes `.env` and exports every variable to the services it starts, so a value set there reaches every service it starts. After editing `.env`, run `make restart` to pick up the change. See [Installation](../getting-started/installation.md). ## Required | Variable | Default | Effect | |---|---|---| | `DATABASE_URL` | *(required)* | The Postgres connection string for the control plane and worker. Must be an async DSN: `postgresql+asyncpg://user:pass@host:5432/dbname`. The example value is `postgresql+asyncpg://theygent:theygent@localhost:5432/theygent`. Missing or unreachable → the control plane fails readiness and migrations refuse to run. | ## Connecting the planes | Variable | Default | Effect | |---|---|---| | `THEYGENT_INFERENCE_PLANE_URL` | `http://127.0.0.1:8081/v1` | Where the control plane reaches the inference data plane. **Must include the `/v1` suffix.** Point this at a remote inference host to run models on another machine. | | `THEYGENT_CONTROL_PLANE_HOST` | `127.0.0.1` | Address the control plane listens on. | | `THEYGENT_CONTROL_PLANE_PORT` | `8080` | Port the control plane listens on. | | `THEYGENT_INFERENCE_PLANE_HOST` | `127.0.0.1` | Address the inference plane listens on. | | `THEYGENT_INFERENCE_PLANE_PORT` | `8081` | Port the inference plane listens on. | | `THEYGENT_CORS_ORIGINS` | `localhost` / `127.0.0.1` on `:5173` and `:5174` when unset | Comma-separated list of browser origins allowed to call both planes. The shipped `.env.example` sets it to `http://localhost:5174`. An empty string disables CORS. Add your interface origin here if you serve it from a non-default port. | ## Control-plane behavior | Variable | Default | Effect | |---|---|---| | `THEYGENT_DURABLE` | off | Set to `1` (also `true`/`yes`/`on`) to run the durable runtime in-process, enabling crash-resumable runs and the `human`/`subgraph`/`loop`/`map` node types. Interactive runs are unchanged either way. See [Durable runs](../running/durable.md). | | `THEYGENT_INVOKE_TOKEN` | unset | A shared per-deploy bearer token for `POST /agents/{id}/invoke`. Personal [API keys](../running/users.md#personal-api-keys) (`tyk_…`) also open that endpoint, so this variable is optional; **with it unset and no API key presented, the endpoint is closed and every call returns 401.** | | `THEYGENT_OIDC_REDIRECT_URL` | `http://localhost:8080/auth/oidc/callback` | Pins the `auth.oidc_redirect_url` platform setting — the redirect URL [sign-in providers](../running/users.md#single-sign-on) send the browser back to, and the exact URL you register with each provider. Change it when the control plane sits behind a reverse proxy or a non-default bind. | | `THEYGENT_SECRET_KEY` | unset → ephemeral dev key | Fernet key(s) encrypting the connection secret store. **Unset means an in-memory key: stored secrets do not survive a restart** (you'll see a loud warning). Generate a real key and set it to persist secrets; supply a comma-separated list to rotate (the first encrypts, all decrypt). | | `THEYGENT_ARTIFACT_DIR` | `/theygent-artifacts` | Where generated audio/image artifacts are stored on local disk. | | `DBOS_SYSTEM_DATABASE_URL` | your `DATABASE_URL` | Optional override to put durable-runtime checkpoints on a different Postgres. State is confined to a separate `dbos` schema. | ## Observability See [Observability](../running/observability.md) for how these interact. | Variable | Default | Effect | |---|---|---| | `THEYGENT_IO_CAPTURE` | `full` | The ceiling for per-node input/output capture: `off`, `metadata` (byte sizes only), or `full` (payloads + sizes). This is a hard cap — no agent policy can capture above it. | | `THEYGENT_TOPOLOGY` | `local` | `local` defaults capture to `full`; `hosted` defaults it to `metadata` (raw payloads never land by default in a hosted database). An agent can still opt back up to the ceiling. | | `THEYGENT_IO_CAPTURE_MAX_BYTES` | `262144` (256 KiB) | Per-payload size cap. Larger values are stored truncated with the true byte count preserved, never silently dropped. | | `OTEL_EXPORTER_OTLP_ENDPOINT` | unset → export off | When set, span *scalars* (never payloads) are exported to this OTLP/HTTP endpoint in addition to the local waterfall. | | `THEYGENT_OTEL_REDACT_ATTRS` | unset | Comma-separated span-attribute names whose values are replaced with `[redacted]` on the exported copy (the local row keeps them). | ## Inference plane | Variable | Default | Effect | |---|---|---| | `THEYGENT_MAX_RESIDENT` | `2` | How many model engines may be loaded in memory at once. When full, the least-important, least-recently-used engine is evicted to make room. Raise it if you have RAM to spare and switch models often. | | `THEYGENT_INFERENCE_PLANE_STATE_DIR` | `~/.theygent/inference` | Where the local model registry (`registry.json`) and credentials (`credentials.json`) live. | | `THEYGENT_INFERENCE_PLANE_MODEL_DIR` | `/models` | Where catalog-installed model weights download. Point it at an external volume to keep large weights off your system disk. | ### Engine binary overrides Each engine binary is resolved from `PATH` by default; set the matching variable to point at a specific build. See [Engines](../models/engines.md). | Variable | Engine binary | |---|---| | `THEYGENT_LLAMACPP_BIN` | `llama-server` (chat, embeddings, vision) | | `THEYGENT_WHISPERCPP_BIN` | `whisper-server` (speech-to-text) | | `THEYGENT_MLX_BIN` | `mlx_lm.server` (chat, Apple Silicon) | | `THEYGENT_MLX_PYTHON` | The Python that mlx-lm is installed in, for [expert streaming](../models/engines.md#moe-models-bigger-than-memory-expert-streaming). Found from `mlx_lm.server` when unset; set it if that fails. | | `THEYGENT_MLX_VLM_BIN` | `mlx_vlm.server` (vision, Apple Silicon) | | `THEYGENT_MLX_AUDIO_BIN` | `mlx_audio.server` (text-to-speech, Apple Silicon) | | `THEYGENT_SDCPP_BIN` | `sd-cli` (image generation) | | `THEYGENT_MFLUX_BIN` | `mflux-generate` (image generation, Apple Silicon) | | `THEYGENT_VLLM_BIN` | `vllm` (chat, CUDA hosts) | ## Worker (server topology only) The standalone [worker](../running/durable.md) reads `DATABASE_URL` (required) and `THEYGENT_INFERENCE_PLANE_URL`. For the inference URL it also accepts a legacy fallback name: | Variable | Default | Effect | |---|---|---| | `THEYGENT_INFERENCE_PLANE_BASE_URL` | falls back to `THEYGENT_INFERENCE_PLANE_URL`, then `http://127.0.0.1:8081/v1` | Legacy alias the worker checks for the inference URL. Prefer `THEYGENT_INFERENCE_PLANE_URL`. | ## AI gateway The [AI Gateway](../ai-gateway/index.md) reads these at startup. A value that does not parse stops it with a message naming the variable. It never reads `ANTHROPIC_BASE_URL` — that variable points Claude Code *at* the gateway. | Variable | Default | Effect | |---|---|---| | `THEYGENT_AI_GATEWAY_AUTH_MODE` | `personal` | `personal`: loopback only, no keys, each caller's own credential. `team`: every request needs a developer's gateway key, synced from the control plane (needs the next two variables). | | `THEYGENT_CONTROL_PLANE_URL` | unset | The control plane the gateway reports to and takes its configuration and developers' keys from, e.g. `http://127.0.0.1:8080`. Required in team mode; with a service key it attaches a personal gateway too. | | `THEYGENT_AI_GATEWAY_SERVICE_KEY` | unset | The gateway's credential towards the control plane (`tygs_…`), minted by an admin under **AI Gateway → Settings → Service keys**. Required in team mode. | | `THEYGENT_AI_GATEWAY_ORG_API_KEY` | unset | The organization's Anthropic API key, for developers billed to it (team mode only). A key set here cannot be replaced from the Settings tab. | | `THEYGENT_AI_GATEWAY_STORE_ORG_KEY` | `1` | Whether an admin may set the organization's key from the Settings tab, kept by this gateway (team mode only). Set `0` for a gateway that runs as several replicas: each would keep its own copy, so the key must come from `THEYGENT_AI_GATEWAY_ORG_API_KEY`. Accepts `1`/`0`, `true`/`false`, `yes`/`no`, `on`/`off`. | | `THEYGENT_AI_GATEWAY_HOST` | `127.0.0.1` | Listen address. Personal mode refuses anything but loopback. | | `THEYGENT_AI_GATEWAY_PORT` | `8082` | Listen port. | | `THEYGENT_AI_GATEWAY_UPSTREAM_URL` | `https://api.anthropic.com` | Where requests are forwarded. It must not be the gateway's own address. | | `THEYGENT_AI_GATEWAY_STATE_DIR` | `~/.theygent/ai-gateway` | The gateway's state: records not yet delivered, the last known configuration, developers' key digests, a stored organization key. | | `THEYGENT_AI_GATEWAY_ID` | `gw-` | The name this gateway appears under on the Overview. | | `THEYGENT_AI_GATEWAY_KEY_SYNC_SECONDS` | `30` | How often team mode re-syncs developers' keys (1–3600); a revoked key is refused within one interval. | | `THEYGENT_AI_GATEWAY_HEARTBEAT_SECONDS` | `30` | How often the gateway reports itself and checks for a new configuration (1–3600). | | `THEYGENT_INFERENCE_PLANE_URL` | `http://127.0.0.1:8081/v1` | Where the gateway reaches [local models](../ai-gateway/local-models.md): the requests a local dial sends there, trials and local shadow runs — the same variable and default as the control plane's. **Must include the `/v1` suffix.** When nothing answers there, those requests go to the Anthropic API. | | `THEYGENT_AI_GATEWAY_ADMIN_ORIGINS` | `http://localhost:5174,http://127.0.0.1:5174` | Comma-separated browser origins allowed to call the gateway's admin surface — the interface's Settings tab. Add the interface's origin if you serve it elsewhere. | ## MCP server (coding agents) The [MCP server](../coding-agents/index.md) that coding agents start reads these from the environment the coding tool launches it with. | Variable | Default | Effect | |---|---|---| | `THEYGENT_API_KEY` | unset | A personal API key (`tyk_…`, from the profile menu → **API keys**). Required by the agent, run, sample and retrieval tools; the model tools work without it. | | `THEYGENT_CONTROL_PLANE_URL` | `http://127.0.0.1:8080` | The control plane the agent, run and retrieval tools call. | | `THEYGENT_INFERENCE_PLANE_URL` | `http://127.0.0.1:8081/v1` | The inference plane the model tools call directly. **Must include the `/v1` suffix.** | | `THEYGENT_MCP_DEFAULT_MODEL` | the first chat model | The logical model id `ask_model` uses when the coding agent names none. | | `THEYGENT_MCP_RUN_TIMEOUT_S` | `600` | Seconds an agent run or a model call may take before the tool reports a timeout. | ## Interface (build-time) The web app reads these at build time (Vite `VITE_*` variables). Change them only if you moved a service off its default address. | Variable | Default | Effect | |---|---|---| | `VITE_CONTROL_PLANE_URL` | `http://localhost:8080` | The control-plane base URL the browser calls. | | `VITE_INFERENCE_URL` | `http://localhost:8081` | The inference-plane base URL the browser calls. | | `VITE_AI_GATEWAY_URL` | `http://localhost:8082` | The AI gateway's address: what the Setup & keys snippets name as `ANTHROPIC_BASE_URL`, and where the Settings tab reaches the gateway. Each browser can also override it on Setup & keys. | ## Advanced tuning | Variable | Default | Effect | |---|---|---| | `THEYGENT_MCP_CALL_TIMEOUT_S` | `120` | Timeout, in seconds, for a single MCP tool call before it errors. Raise it for slow tools. | | `THEYGENT_CRAWLER_MAX_CHARGE_USD` | *(unset)* | Caps the **Settings → Crawlers** spend ceiling: the most one crawler call may charge, in USD. The setting stays editable below this value. See [Crawler](../building/nodes/crawler.md#spend). | | `THEYGENT_MCP_REGISTRIES` | *(unset)* | Extra MCP hubs for the catalog browser: a JSON array of `{"id", "label", "url"}` objects. Any registry speaking the standard subregistry API works; a self-hosted one doubles as an allowlist for air-gapped setups. A malformed value is a loud error, never a silently dropped registry. See [MCP servers](../mcp/index.md). | | `THEYGENT_OAUTH_REDIRECT_URL` | `http://localhost:8080/mcp/oauth/callback` | Where an MCP OAuth sign-in redirects the browser back to. Change it only if the control plane is reached at a different address than the default. | !!! note A number of other `THEYGENT_*` names exist only to drive tests and integration harnesses — they are not end-user configuration and are omitted here. If a name is not on this page, treat it as internal. ## Related pages - [Installation](../getting-started/installation.md) — setting up `.env` for the first time. - [Running Ertir](../getting-started/running.md) — starting the services that read these. - [Troubleshooting](troubleshooting.md) — symptoms that trace back to a misconfigured variable. --- Source: https://docs.ertir.dev/latest/reference/settings/ # Platform settings Platform settings configure how this installation behaves: what the run waterfall captures, how long tool calls may take, how much of a tool's result a model is shown, how documents are chunked. An admin edits them on the **Settings** page, one tab per group, or over the API. Each tab saves only its own group. ## How a setting gets its value Each setting resolves in this order: 1. **An environment variable**, if the setting has one and it is set to a non-empty value. Some variables **pin** the setting: the environment wins and the field is locked in the UI. Others **cap** it: you can still edit the setting, but only up to the environment's value. 2. **The stored value**, if an admin saved one. 3. **The default** listed below. Changes apply without a restart, in every control-plane process and the worker, within about 15 seconds. Some apply at a later point, which the UI names next to the field: at the next run, connection, ingest, crawl or sign-in. A stored value that no longer passes validation (for example, after an upgrade narrowed its range) is ignored in favor of the default, and the field says so. ## Tools Applies to every tool an `llm` node calls: builtin, REST, MCP and retrieval, in both interactive and durable runs. | Setting | Default | Range | What it does | |---|---|---|---| | `tools.result_max_chars` | `16000` | 1,000–1,000,000 | The most characters of one tool result the model is shown. A longer result is cut, and a note at the end tells the model how many characters were left out. Applies from the next run. An `llm` node's own `maxToolResultChars` overrides it; see [Tools the model can call](../building/nodes/llm.md#tools-the-model-can-call). | The default is about 4,000 tokens: a default retrieval (five 450-token chunks) reaches the model whole, while a set of crawled pages is cut. The run inspector still shows every tool's whole result, and the tool's span in the waterfall records the result's full length and whether it was cut. ## Telemetry | Setting | Default | Environment | What it does | |---|---|---|---| | `telemetry.io_capture` | `full` | `THEYGENT_IO_CAPTURE` (cap) | The deployment-wide ceiling on node input/output capture: `off`, `metadata` or `full`. Each agent's own policy applies under it. | | `telemetry.io_capture_max_bytes` | `262144` | `THEYGENT_IO_CAPTURE_MAX_BYTES` (cap) | The size cap for one captured payload. A larger payload is stored truncated, with its true size. | | `telemetry.io_retention_days` | `0` | — | Delete captured run inputs and outputs after this many days, for agents that set no retention of their own. `0` keeps them. | | `telemetry.otlp_enabled` | on only when an endpoint is set in the environment | — | Export spans to an OTLP collector. Storing `false` stops export even when the environment sets an endpoint. | | `telemetry.otlp_endpoint` | unset | `OTEL_EXPORTER_OTLP_ENDPOINT` (pin) | The OTLP/HTTP collector endpoint. | | `telemetry.otlp_headers` | unset | — | Headers sent with every export, such as collector auth. Write-only: reads return the header names, never the values. | | `telemetry.otlp_redact_attrs` | none | `THEYGENT_OTEL_REDACT_ATTRS` (pin) | Span attribute names replaced with `[redacted]` before export. The local span store keeps them. | See [Observability](../running/observability.md). ## RAG | Setting | Default | Range | What it does | |---|---|---|---| | `rag.chunk_max_tokens` | `450` | 50–4,000 | Token budget per chunk, at ingest. | | `rag.chunk_overlap_tokens` | `60` | 0–1,000 | Overlap between chunks, at ingest. | | `rag.crawl_desired_concurrency` | `2` | 1–16 | Concurrent page fetches a crawl aims for. | | `rag.crawl_max_concurrency` | `4` | 1–32 | The most concurrent page fetches during a crawl. | | `rag.max_upload_bytes` | 50 MiB | 1 MiB–1 GiB | The largest document you can upload. | | `rag.default_top_k` | `5` | 1–50 | How many passages a retrieval returns when the query or node sets no `topK`. | | `rag.default_embedding_model` | unset | — | The embedding model the new-source form suggests. Each source keeps the model it was created with. | See [RAG sources](../rag/index.md). ## MCP | Setting | Default | Environment | What it does | |---|---|---|---| | `mcp.call_timeout_s` | `120` | `THEYGENT_MCP_CALL_TIMEOUT_S` (pin) | Seconds one MCP tool call may take. Applies to new and reconnected server connections. | | `mcp.connect_timeout_s` | `30` | — | Seconds starting and connecting to one MCP server may take. | | `mcp.extra_registries` | none | `THEYGENT_MCP_REGISTRIES` (pin) | Self-hosted MCP registries to browse alongside the public ones. | | `mcp.oauth_redirect_url` | `http://localhost:8080/mcp/oauth/callback` | `THEYGENT_OAUTH_REDIRECT_URL` (pin) | Where an authorization server sends the browser back after consent. Change it behind a reverse proxy. | See [MCP servers](../mcp/index.md). ## Sign-in | Setting | Default | Environment | What it does | |---|---|---|---| | `auth.session_ttl_hours` | `168` | — | How long an interactive sign-in stays valid. Applies to sign-ins after the change. | | `auth.oidc_redirect_url` | `http://localhost:8080/auth/oidc/callback` | `THEYGENT_OIDC_REDIRECT_URL` (pin) | The redirect URI to register with each sign-in provider. | See [Users, roles & sign-in](../running/users.md). ## AI gateway | Setting | Default | What it does | |---|---|---| | `ai_gateway.request_retention_days` | `90` | Days each AI gateway request is kept in detail. Older days move to a compressed archive, and reports read them from daily totals. `0` keeps every request in detail. It is edited under **AI Gateway → Settings → Request records**. | ## Over the API Reading settings needs an admin, as does changing them. ```bash curl -s http://localhost:8080/settings -H "Authorization: Bearer $TOKEN" ``` ```bash curl -s -X PATCH http://localhost:8080/settings -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"tools.result_max_chars": 8000}' ``` A `PATCH` body maps keys to values; `null` resets a key to its default. The patch is all or nothing: if any key is unknown or out of range, the response is `422 invalid_setting` and nothing is stored. --- Source: https://docs.ertir.dev/latest/reference/glossary/ # Glossary The terms used across Ertir and this documentation, in alphabetical order. Where a concept has its own page, the definition links to it. ### Activity One of the three node [kinds](#kind). Activity nodes do non-deterministic work with side effects — calling a model (`llm`), running a tool (`tool`, `mcp_tool`), transcribing audio, generating an image. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### AI gateway The service between Claude Code and its models (port 8082). It records what each exchange cost and what a routing policy would do with it, and in [router mode](#router-mode) applies that policy: cheaper Claude models, or [local models](../ai-gateway/local-models.md) in the shares an admin sets. Runs in [personal or team mode](../ai-gateway/index.md#personal-or-team). See [AI Gateway](../ai-gateway/index.md). ### Agent A published, runnable AI workflow: a [graph](#graph) of [nodes](#node) plus its model and tool [bindings](#binding), published under an id. You build agents in the [editor](../building/editor.md) (where work-in-progress lives as a [draft](#draft)) and run them from the Bench, from chat, or behind a [trigger](#trigger). See [Agents & graphs](../concepts/agents-and-graphs.md). ### Artifact A blob of generated media — audio or an image — stored on local disk and passed between steps **by reference** (an `art_…` id) rather than inline. A voice agent's spoken reply and an image node's output are artifacts. Bytes stay in your trust domain and are never journaled into a run's history. See [Voice](../chat/voice.md). ### Binding How a logical model maps to something that can actually answer a request. The binding value is one of four: `mlx`, `vllm`, `llamacpp` (local engines Ertir manages), or `openai-compatible` (any reachable OpenAI-style server or hosted API). See [Models & engines](../concepts/models-and-engines.md). ### Boundary One of the three node [kinds](#kind): the entry, exit, and wait points of a graph — `input`, `output`, `human`, `subgraph`. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Capability A property of a model, probed from the engine: whether it supports tool-calling, structured output, vision, or reasoning, and its maximum context length. Capabilities drive which controls appear on the Bench and which badges show in the [Registries](../models/index.md) page. ### Channel What an [edge](#edge) carries between [ports](#port). `data` passes a value; `control` is pure sequencing (run-after, no value); `tool` is a capability wire connecting a tool node to an `llm` so the model may call it. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Connection A named, reusable credential for a tool or MCP server — kind `http_auth` or `mcp_server`. The non-secret config is stored plainly; the secret is encrypted, write-only, and never appears in the graph. Remote-server auth types: `bearer`, `api_key`, `basic`, `headers` (a whole header map), `oauth2_client_credentials`, or `oauth` (interactive sign-in, tokens stored encrypted behind the same secret reference). Everything the MCP page creates — hub installs, remote, OpenAPI/GraphQL, stdio-with-secrets — is a connection. See [Tools](../building/nodes/tools.md) and [MCP servers](../mcp/index.md). ### Content hash The `sha256:` fingerprint that gives an agent version its identity, computed over the graph with layout stripped and defaults filled. Rearranging the canvas, zooming, or changing a node's icon never changes it; changing a message, config value, or edge does. See [Agent versioning](../concepts/versioning.md). ### Control plane The hosted orchestration service (default `http://localhost:8080`) that owns agents, runs, sessions, triggers, connections, artifacts, and observability, backed by your Postgres. It reaches the [inference plane](#inference-plane) over HTTP only and never runs models itself. See [Architecture](../concepts/index.md). ### Draft The editor's autosaved work-in-progress: a mutable snapshot of the graph you are building, saved server-side as you edit. A draft may be structurally invalid, has no [content hash](#content-hash), and can't be run by reference or pinned — publishing graduates its content into an immutable [version](#version) and removes the draft. Drafts live in the **Drafts** strip on the Agents page. See [Drafts & publishing](../building/saving-agents.md). ### Durable run A run executed on the durable runtime, which checkpoints each completed step so the run survives a crash and resumes where it left off. Durable mode also unlocks the `human`, `subgraph`, `loop`, and `map` node types. See [Durable runs](../running/durable.md). ### Edge A wire connecting one node's out-[port](#port) to another's in-port, carrying a [channel](#channel). At most one `data` edge may feed a given in-port. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Embedding model A model that turns text into a vector so that similar meanings land near each other — the model behind retrieval. Each [RAG source](#rag-source) pins one embedding model (a [logical id](#logical-model-id)) at creation, because vectors from different models are not comparable. See [RAG sources](../rag/index.md). ### Engine The underlying server program that runs a model — `llama.cpp`, an MLX server, `whisper.cpp`, vLLM, or an image-generation CLI wrapper. Ertir lazily spawns and supervises managed engines; you never name an engine when calling a model. See [Engines](../models/engines.md). ### Expert streaming Running a [mixture-of-experts](#mixture-of-experts-moe) model on MLX with its experts left on disk: each layer reads only the experts its router picks, and recently used ones stay in a memory cache. It lets a model bigger than memory run, with exactly the answers the fully loaded model gives, at lower speed. Off by default; turned on per model. See [Engines](../models/engines.md#moe-models-bigger-than-memory-expert-streaming). ### Gateway key A developer's key for a team-mode [AI gateway](#ai-gateway) (`tyg_…`), created on **AI Gateway → Setup & keys** and shown once. It identifies the developer to the gateway and opens nothing else in Ertir. See [Set up Claude Code](../ai-gateway/setup.md). ### Generated server An MCP server Ertir mints in-process from an API you already have: an OpenAPI spec (every operation becomes a callable tool) or a GraphQL endpoint (two tools — schema introspection and a validated query runner). No subprocess; only the upstream API calls leave the process, carrying auth resolved server-side. See [MCP servers](../mcp/index.md). ### Graph The document that defines an [agent](#agent): a `camelCase` JSON envelope holding the model and tool [bindings](#binding), a list of typed [nodes](#node), and the [edges](#edge) wiring them, plus a never-hashed `view` block for canvas layout. Also called the IR (intermediate representation). See [Agents & graphs](../concepts/agents-and-graphs.md). ### Guardrail A node that checks its input against a rule (regex, length, allow/deny list, JSON-shape, PII) or an LLM judge, then routes to a `pass` or `block` port. Wire `block` to an output to refuse a request before doing expensive work. See [Guardrail](../building/nodes/guardrail.md). ### Hub (MCP registry) A catalog of published MCP servers you can browse and install from: the Official MCP Registry, the GitHub MCP Registry, or a self-hosted registry added via `THEYGENT_MCP_REGISTRIES` (which doubles as an allowlist for air-gapped setups). Installing an entry creates an `mcp_server` [connection](#connection) stamped with its origin (registry + version). See [MCP servers](../mcp/index.md). ### Hybrid search How a [RAG source](#rag-source) is searched: semantic (vector) similarity fused with keyword full-text ranking, so paraphrases and exact identifiers both surface. Runs entirely inside your Postgres. See [RAG sources](../rag/index.md). ### Inference plane The user-controlled service (default `http://localhost:8081`) that registers models under logical ids, spawns engines, and answers OpenAI-compatible requests. It runs wherever you point it and keeps its registry and weights on your machine. See [Architecture](../concepts/index.md). ### I/O capture The policy governing whether a run's per-node input/output payloads are recorded for the observability waterfall: `off`, `metadata` (byte sizes only), or `full`. Capped by an environment ceiling. See [Observability](../running/observability.md). ### Kind The category of a [node](#node): `activity`, `orchestration`, or `boundary`. Every node type has exactly one correct kind (the `guardrail` is the one exception, whose kind follows its check type). See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Local trial A slice of conversations, just above a local dial's share, that the [AI gateway](#ai-gateway) still sends to Anthropic and then tries on the local model, recording whether it picked the same next action. It measures a model before it serves. See [Local models](../ai-gateway/local-models.md). ### Logical model id The name you give a model when you register it (for example `triage-fast`) and the only thing you ever put in a `model` field. It hides the engine and weights behind it, so swapping a local model for a hosted one is one registration change with no graph edits. See [Models & engines](../concepts/models-and-engines.md). ### MCP (Model Context Protocol) An open protocol for exposing external tools to models. Ertir connects to MCP servers — spawned locally over `stdio`, reached remotely over HTTP or SSE, or [generated in-process](#generated-server) from an OpenAPI spec or GraphQL endpoint — and lets your agents call the tools they expose through the `mcp_tool` node. Servers are added by hand or installed from a [hub](#hub-mcp-registry). See [MCP servers](../mcp/index.md) and [MCP tools](../building/nodes/mcp.md). ### Mixture of experts (MoE) A model design that splits most of a model's weights into many small "experts" and uses only a few of them for each token, chosen by a small router in every layer — Qwen3.6-35B-A3B has 35 billion parameters but uses about 3 billion per token (the "A3B"). It computes like a small model but needs the memory of a large one, unless its experts are [streamed](#expert-streaming). The catalog marks these models **MoE**. ### Modality What kind of input/output a model handles: `chat`, `vision`, `embeddings`, `audio.transcription`, `audio.speech`, or `images.generation`. A model is registered for one modality; it is a third axis orthogonal to [binding](#binding) and [source](#source). See [Models & engines](../concepts/models-and-engines.md). ### Node One step in a [graph](#graph): a typed box with a config, in-[ports](#port), and out-ports. Seventeen types are executable, across the three [kinds](#kind). See the [node reference](../building/nodes/index.md). ### Observer mode The [AI gateway](#ai-gateway) mode that records every exchange — its cost and what routing would have done — without changing it. Its counterpart, *passthrough*, forwards without recording. See [AI Gateway](../ai-gateway/index.md#observer-router-and-passthrough). ### Router mode The [AI gateway](#ai-gateway) mode that applies the routing decisions observer mode only records: a routed request runs on the target model — a cheaper Anthropic model, with nothing else changed, or a [local model](../ai-gateway/local-models.md). See [AI Gateway](../ai-gateway/index.md#observer-router-and-passthrough). ### Orchestration One of the three node [kinds](#kind): deterministic control flow — `router`, `transform`, `loop`, `map`. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Port A named connection point on a [node](#node), on the in or out side, carrying a [channel](#channel) (`data`, `control`, or `tool`). A required in-port must be fed by exactly one data edge for the graph to validate. See [Nodes, ports & edges](../concepts/nodes-ports-edges.md). ### Publish The deliberate act that turns the graph on the canvas into an immutable, content-addressed [version](#version) in the registry — visible to everyone who can reach the control plane, runnable by reference, and pinnable by triggers and composing agents. Contrast with the automatic [draft](#draft) tier. See [Drafts & publishing](../building/saving-agents.md). ### RAG source A named document collection agents retrieve from — filled by uploading files or crawling a site, chunked and embedded against its pinned [embedding model](#embedding-model), stored in your Postgres, and searched by the [rag node](../building/nodes/rag.md). Referenced by a stable id, so re-ingesting never changes an agent's version. See [RAG sources](../rag/index.md). ### Run One execution of an agent or graph. It moves through statuses `created → streaming → waiting → completed | failed`, persists in Postgres, and survives restarts. Its output is the value that reached the `output` node. See [Runs & sessions](../concepts/runs-and-sessions.md). ### Service key An [AI gateway](#ai-gateway)'s credential towards the control plane (`tygs_…`), minted by an admin under **AI Gateway → Settings → Service keys**. It lets the gateway fetch its configuration and developers' keys and deliver its records. See [Running the gateway](../ai-gateway/running.md#attach-it-to-the-control-plane). ### Shadow run A copy of a request the [AI gateway](#ai-gateway) sends, after the real exchange, to a model that did not answer it — the target routing would have used, or a local model on [trial](#local-trial) — to measure it. Claude Code never sees it. See [Reports & policy](../ai-gateway/reports.md#the-report-tab). ### Side request A small request Claude Code sends besides the conversation itself — a session title, a suggestion, a check. The [AI gateway](#ai-gateway)'s Catalog lists each kind it has seen and what routing may do with it. See [Reports & policy](../ai-gateway/reports.md#catalog-admins). ### Session Opt-in conversational memory: a stored thread of turns that a run can read back and append to, so a model remembers earlier messages. Passing a `session_id` on a run turns it on; without one, the run is a one-shot. See [Runs & sessions](../concepts/runs-and-sessions.md). ### Source Where a managed model's weights come from: `hf` (a Hugging Face repo), `local-path` (a file or directory on disk), or `url`. Separate from the [binding](#binding) — Hugging Face is a source of weights, not an engine. See [Models & engines](../concepts/models-and-engines.md). ### Span One node in a run's trace: a timed bar in the [observability](../running/observability.md) waterfall, with a status (`ok`, `err`, `skipped`, `running`) and, on model-generation phases, token counts. Spans carry timing and metadata; payloads are stored separately and gated by the [I/O capture](#io-capture) policy. ### Trigger A deployed entry point that fires a published, pinned agent unattended: a `schedule` (cron) or a `webhook` (an HTTP POST with an HMAC signature). A published agent can also be reached through the always-on token-invoke endpoint, which is gated by `THEYGENT_INVOKE_TOKEN` rather than created as a trigger. See [Triggers & webhooks](../running/triggers.md). ### Version An immutable, content-addressed snapshot of an agent, identified by its `version` string and [content hash](#content-hash). Publishing new content mints a new version; the old ones remain runnable. See [Agent versioning](../concepts/versioning.md). ### Webhook A [trigger](#trigger) kind that runs an agent when an external system POSTs to `/hooks/{id}`. The request body becomes the run input; a per-webhook secret authenticates the call via an HMAC signature. See [Triggers & webhooks](../running/triggers.md). ### Worker The standalone process (`theygent-worker`) that runs the durable queue in a server or air-gapped deployment. On your own machine you don't need it — setting `THEYGENT_DURABLE=1` runs the same runtime inside the control plane. See [Durable runs](../running/durable.md). ## Related pages - [Architecture](../concepts/index.md) — how the two planes fit together. - [API reference](api.md) — the endpoints these terms appear in. - [Troubleshooting](troubleshooting.md) — when one of these goes wrong. --- Source: https://docs.ertir.dev/latest/reference/troubleshooting/ # Troubleshooting Common problems, what causes them, and how to fix them. Most issues fall into one of a few buckets: a service didn't start, a model won't load, or a run surfaced an error code. When you hit an error code you don't recognize, the [API reference](api.md) lists where each one comes from. !!! tip "First stop: the health checks" Before digging in, check both planes are up and ready: ```bash curl http://localhost:8081/readyz # inference plane curl http://localhost:8080/readyz # control plane ``` The inference plane's `/readyz` tells you exactly which `(engine, modality)` slots are missing and why; the control plane's tells you if Postgres or the inference plane is unreachable. See [Running Ertir](../getting-started/running.md). ## Services and startup ### A service didn't start — "already on :PORT — skipping" **Cause.** Something is already listening on that port (a stale instance, or another app on `:8080`, `:8081`, or `:5174`). `make up` refuses to clobber it and skips that service loudly. **Fix.** Stop the other process, then `make restart`. To stop Ertir's own services, `make down`. Confirm what's up with `make status`, and read the per-service logs under `.run/` (`.run/control-plane.log`, `.run/inference-plane.log`, `.run/interface.log`) with `make logs`. ### The control plane won't come up / `/readyz` says the database is down **Cause.** `DATABASE_URL` is unset, wrong, or points at a Postgres that isn't running. The control plane requires Postgres and fails readiness without it. **Fix.** Set `DATABASE_URL` in `.env` to a reachable async DSN — `postgresql+asyncpg://user:pass@host:5432/dbname` — and make sure the database and user exist. You supply your own Postgres; Ertir does not provision one. See [Installation](../getting-started/installation.md) and [`DATABASE_URL`](environment.md#required). ## Database and migrations ### "DATABASE_URL is not set (copy .env.example to .env)" **Cause.** `make migrate` (or `make up`) ran without `DATABASE_URL` in the environment. **Fix.** `cp .env.example .env`, set `DATABASE_URL`, and re-run. Migrations use the same variable — the migration tool raises `DATABASE_URL must be set for migrations` when it's missing. ### Migrations fail or the schema looks out of date **Cause.** The control-plane schema hasn't been applied to your database, or a pull added new migrations. **Fix.** Apply them: ```bash make migrate # or, directly: uv run --package theygent-control-plane alembic upgrade head ``` Durable-runtime tables live in a separate `dbos` schema on the same Postgres and are migrated automatically when durable mode starts — there is no separate step for them. ## Models and engines ### `model_not_found` (404) **Cause.** Either the `model` field is a [logical id](glossary.md#logical-model-id) you never registered, or you put an **engine name** (like `llamacpp` or `mlx`) where a logical id belongs. The `model` field is always a logical id, never an engine name. **Fix.** Register the model on the [Registries](../models/index.md) page (or `PUT /admin/models/{id}`), and use that logical id. Check what's registered with `GET /admin/models` or `GET /v1/models`. ### `engine_unavailable` (503) **Cause.** The model is registered, but the engine binary it needs isn't installed on this machine (for example an `mlx` model on a host without the MLX servers). **Fix.** Install the engine. On macOS, `make engines` installs the local engine servers; otherwise install `llama-server`, `whisper-server`, and `ffmpeg` from your package manager. `GET /readyz` on the inference plane names the missing slot with an install hint. See [Engines](../models/engines.md). ### `/readyz` shows a slot as not-ready **Cause.** That `(engine, modality)` combination has no runnable binary — for example `llamacpp:audio.transcription` needs `whisper-server`, and image generation needs `sd-cli` or `mflux-generate`, which `make engines` does **not** install. **Fix.** Install the named binary, or point the matching [binary-override variable](environment.md#engine-binary-overrides) (e.g. `THEYGENT_WHISPERCPP_BIN`, `THEYGENT_SDCPP_BIN`) at your build, then restart the inference plane. The service stays generally "ready" as long as any one engine works, so a single missing slot doesn't take everything down. ### `incomplete_weights` (422 on install or registration, 503 on use) **Cause.** The weights are only part of a model. Text-to-image repositories often publish the **denoiser** by itself, without the VAE and text encoder it needs to render — the file is the right size and the right format, but nothing can generate an image from it alone. The same code also catches a language model registered under a `images.generation` modality. **Fix.** Install a complete text-to-image checkpoint: one file that bundles the denoiser, the VAE, and the text encoder. The message names which parts are absent. See [Incomplete image models](../models/index.md#incomplete-image-models). ### `no_capacity` (503) **Cause.** Every resident engine slot is full and busy — the inference plane keeps at most `THEYGENT_MAX_RESIDENT` engines loaded (default 2), and none could be evicted because they're all handling requests. **Fix.** Wait for an in-flight request to finish, evict a model you're not using (the **Evict** action on Registries, or `POST /admin/models/{id}:evict`), or raise `THEYGENT_MAX_RESIDENT` if you have the RAM. ### `engine_failed` (503) **Cause.** The model's engine stopped being able to compute — llama.cpp answers `Compute error` once its GPU backend has failed, most often because the model ran out of GPU memory — or the engine process died. Ertir has shut that engine down; the next request starts a new one. **Fix.** Retry. If it fails again, the model doesn't fit with its current settings: lower its context with the `ctxSize` [launch setting](../models/engines.md#llamacpp-launch-settings), pick a smaller quantization, or close other GPU-heavy apps. The message names the engine log, which has the engine's own error (also under the model's **Logs** button). ### `engine_dropped_request` (502, or an error at the end of a stream) **Cause.** The engine closed the connection before it finished answering, so that request was lost. The engine itself keeps running and serves the next request. On MLX, a known cause is mlx-lm's tool-call parser: when a model fills a **list or object parameter** of a tool with plain text instead of JSON (for example `ramen restaurant` where the tool expects `["ramen restaurant"]`), mlx-lm 0.31.3 fails to parse the call and drops the request. A streamed reply that stops this way ends with this error, not with a finished but empty answer. For a [remote model](../models/remote.md), the server behind it closed the connection, and the message says so. A remote model's streamed reply is checked for this only when its registration has [**Report a reply cut off mid-stream**](../models/remote.md#a-reply-cut-off-mid-stream-can-read-as-finished) (`requireFinishReason`) turned on. If that option is on and *every* streamed reply fails this way right after the full answer, the server doesn't send a finish reason, so turn it off. **Fix.** Open the model's **Logs**: the engine's traceback names the cause. For the MLX tool-call case, give the tool a single `string` parameter in place of the list, or describe the expected JSON in the parameter's description. A model writes a single value reliably. A retry can succeed if the model writes the argument differently, but at temperature 0 it usually repeats the same call. The inference plane doesn't retry a request to a local engine, because one that failed this way usually fails the same way again. For a remote model, the server's own log, or a hosted provider's status page, has the reason. ### `modality_not_supported` (404) **Cause.** The engine answered the endpoint with a 404 — usually you called a modality the model can't do (embeddings against a chat-only model), or, for a remote `openai-compatible` model, the `baseUrl` is missing its `/v1` suffix. **Fix.** Use a model registered for the right modality, or fix the base URL. Hosted APIs usually need a trailing `/v1`. See [Remote models](../models/remote.md). ### A model download shows a "too large" fit badge, or loads slowly / fails **Cause.** The variant's estimated memory need exceeds a safe fraction of your RAM. The fit badge warns before you install. **Fix.** Pick a smaller quantization variant (a lower-precision quant), or a smaller model. The catalog marks one variant `recommended` — the largest that comfortably fits. If the card is marked **MoE** and you're on an Apple Silicon Mac, the MLX variant can run anyway with **Stream experts from disk** (offered in the install dialog). See [Installing local models](../models/installing.md). ### A model with expert streaming won't start, or is slow **Cause.** Expert streaming works only for mixture-of-experts chat models in MLX format, and is refused when its engine starts for anything else — a model without experts, one whose experts are a kind it doesn't read, or a checkpoint that wasn't converted for MLX. The reason is in the model's engine log. When it does run, its speed depends on how many of the experts a reply needs are already in its cache, then on your SSD. **Fix.** Open the model's **Logs** to read the refusal; for a model that can't stream, untick **Stream experts from disk** in its settings. If a streamed model is slow, give it a bigger **Expert cache (GB)**, and expect the first reply to a long prompt to be slow from a cold disk. If the model fits in memory, it's faster with streaming off. See [expert streaming](../models/engines.md#moe-models-bigger-than-memory-expert-streaming). ### `logical_id_exists` (409) when installing from the catalog **Cause.** You chose a logical id that's already registered. **Fix.** Pick a different logical id in the install dialog, or delete the existing registration first. ### Text-to-speech returns a 502 "try rephrasing" **Cause.** The speech engine aborted mid-synthesis or produced empty audio — some input texts trip the engine's text processing. **Fix.** Rephrase or shorten the input text and try again. This is a clean, non-fatal error, not a crash. ### Speech-to-text won't load the weights **Cause.** The transcription engine expects `whisper.cpp`'s native `ggml-*.bin` weights; a chat-style GGUF conversion won't work. **Fix.** Point the model at the exact native weights file. See [Engines](../models/engines.md). ## Runs ### A run is stuck at status `waiting` **Cause.** The run reached a `human` node and is paused for input. This is expected — waiting runs are excluded from the crash sweep and can wait indefinitely. **Fix.** Deliver the input: on the run detail page use the **Resume** panel, or `POST /runs/{id}/resume` with `{input: …}`. See [Human-in-the-loop](../running/human-in-the-loop.md). ### `durable_required` (400) **Cause.** You tried to run an agent that contains a `human`, `subgraph`, `loop`, or `map` node — or you asked for a durable run — but the control plane isn't in durable mode. **Fix.** Set `THEYGENT_DURABLE=1` in `.env` and `make restart`, then run it with **Run durably** (or `POST /agents/{id}/durable-runs`). See [Durable runs](../running/durable.md). ### A run failed with "interrupted: control-plane restarted while run was in-flight" **Cause.** The control plane restarted while a non-durable run was still executing. On startup, any run left `created` or `streaming` is swept to `failed` because its progress is unknown. **Fix.** Re-run it. To make unattended runs survive restarts and resume from the last completed step, run them in [durable mode](../running/durable.md) — durable and waiting runs are excluded from the sweep. ### `template_error`, `router_error`, or `transform_error` (422) **Cause.** An input-reference problem. `template_error` = a `$in` token that didn't resolve (an unknown port or a missing field). `router_error` = a `router`'s `select` resolved to no handle or an undeclared one — there is no fallback guessing. `transform_error` = a `transform` node's expression is invalid or references something absent. **Fix.** Check the token against the node's actual ports. Remember the first segment after `$in.` is always a **port name** (`$in.in.field` drills into the default `in` port). See [Referencing inputs](../building/input-references.md). ### The run completed but the output is empty (with an amber note) **Cause.** The graph reached no `output` node on the taken branch, an upstream node errored, or a reasoning model spent its whole token budget thinking (`finish_reason=length`). The run stays `completed` and the note explains which. **Fix.** If it's the token budget, raise `maxTokens` on the model binding (or Max tokens in chat). Otherwise check that your branches route to an output node. See [Runs & sessions](../concepts/runs-and-sessions.md). ### "a second output node executed — the run output would be ambiguous" **Cause.** Two `output` nodes were live in the same run. Ertir refuses to guess which one is the answer. **Fix.** Route exclusive branches (with a `router` or `guardrail`) so at most one output node is reachable per run. Having multiple output nodes is fine as long as only one executes. ### A tool node shows red in the waterfall but the run is green **This is normal, not a bug.** A tool or MCP error is delivered as structured output on the node's `err` port and the run continues (`completed`). The node's bar shows red while the run stays green. **Fix (if you want it to stop the run).** Wire the `err` port somewhere that handles the failure, rather than leaving it dead. See [Tools](../building/nodes/tools.md). ## Agents and publishing ### `version_conflict` (409) when publishing **Cause.** You tried to publish different content under a version string that already exists. Published versions are immutable. **Fix.** Bump the `version` field in the editor toolbar and publish again. (Re-publishing *identical* content under the same version is a harmless no-op.) ### `agent_exists` (409) **Cause.** You tried to create a brand-new agent with an id that's already taken. **Fix.** Publish it as a new *version* of that agent instead (`POST /agents/{id}/versions`). The editor does this automatically when you publish a graph whose id already exists. See [Drafts & publishing](../building/saving-agents.md). ## Triggers, webhooks, and invocation ### Webhook returns `invalid_signature` (401) **Cause.** The `X-TheYgent-Signature` header didn't match. The signature must be the HMAC-SHA256 of the **exact raw request body**, keyed by the trigger's `config.secret`. **Fix.** Sign the raw bytes you send (any reformatting of the JSON changes the signature). Both `sha256=` and a bare hex digest are accepted. Confirm you're using the right secret. See [Triggers & webhooks](../running/triggers.md). ### `invoke` returns `unauthorized` (401) **Cause.** `POST /agents/{id}/invoke` requires the bearer token `THEYGENT_INVOKE_TOKEN`, and it's either unset (the endpoint is closed by default) or your `Authorization` header doesn't match. **Fix.** Set `THEYGENT_INVOKE_TOKEN` in `.env`, restart, and send `Authorization: Bearer `. ### `trigger_disabled` (409) or `trigger_not_found` (404) on a webhook **Cause.** The trigger is disabled, or the id isn't a webhook trigger. **Fix.** Enable it (`PATCH /triggers/{id}` with `{enabled: true}`) or check the id. A schedule that stops firing after downtime is expected — schedules don't backfill missed windows. ## Secrets and credentials ### Connection secrets disappear after a restart **Cause.** `THEYGENT_SECRET_KEY` is unset, so the secret store uses an ephemeral in-memory key (you'll have seen a loud warning at startup). Encrypted secrets don't survive a restart with an ephemeral key. **Fix.** Generate a real Fernet key and set `THEYGENT_SECRET_KEY` in `.env`. To rotate later, supply a comma-separated list — the first key encrypts, all decrypt. ### `credential_error` (502) from a remote model **Cause.** A model's `credentialRef` (`secret://NAME`) couldn't be resolved — the named credential isn't in the local store and isn't a process environment variable. **Fix.** Add the credential under **Settings → Local credentials** (or `PUT /admin/credentials/{name}`). Credential values are write-only and resolve on your machine only. See [Remote models](../models/remote.md). ### A tool with a connection binds an `err` at run time **Cause.** The connection was deleted, disabled, or its secret can't be decrypted (for example after rotating away the key that encrypted it). Connections resolve at step time, not at save time, so this surfaces as a clean `err` — never a crash. **Fix.** Re-enable or re-create the connection, or set the correct `THEYGENT_SECRET_KEY`. ## Import and export ### `invalid_include` (400) on export **Cause.** `POST /export` was called with an empty `include` list or a value it doesn't know. The valid sections are `agents`, `runs`, `traces`, `sessions`, `mcp`, and `rag`. **Fix.** Send a non-empty subset of those values. Note `traces` implies `runs` — the server adds them. See [Import & export](../import-export/index.md). ### `invalid_bundle` (400) on import **Cause.** The body isn't a bundle that plane understands. On the control plane (`POST /import`) it means the body isn't a JSON object carrying `format_version`; on the inference plane (`POST /admin/import`) it means the body is malformed or carries fields the plane doesn't recognize — unknown keys are rejected loudly rather than silently dropped. **Fix.** Import the unmodified file an export produced — `control-plane.json` to `:8080/import`, `inference/models.json` to `:8081/admin/import` — or let the Settings **Import / Export** tab unpack the zip and route the halves for you. ### `unsupported_bundle_version` (400) **Cause.** The bundle's `format_version` is newer than this install understands — it was exported by a newer Ertir. **Fix.** Upgrade the importing install, then re-run the import. ### The import report lists `version_conflict` warnings **Cause.** The target already has that agent version with **different** content. Published versions are immutable, so the import refuses to overwrite it — that version is skipped while the rest of the bundle lands. **Fix.** Usually nothing — the target's version wins by design. If you want the bundled graph too, publish it under a new version on the target. ### An imported model warns `weights_unavailable` **Cause.** The registration points at a file path from the *exporting* machine and carries no catalog-install provenance, so there is nothing to re-download. It is registered but will fail at first use. **Fix.** Put the weights at that path on this machine, or delete the registration and [reinstall the model](../models/installing.md) under the same logical id. Catalog-installed models don't hit this — they re-download automatically. ### The import report lists `download_in_progress` or `params_path_dropped` **Cause.** `download_in_progress`: the model's weights are already downloading (usually from a previous run of the same import), so this run skipped it rather than start a duplicate download. `params_path_dropped`: the exported registration carried a machine-absolute auxiliary path (such as a vision projector file) from the old machine; the import dropped it so the model can start here — the engine finds the file next to the re-downloaded weights instead. **Fix.** Nothing for either — wait for the in-flight download to finish (notification center), and the dropped path re-resolves automatically. If a vision model can't find its projector after import, re-set it in the registration's parameters. ### The import report lists `invalid_section` **Cause.** A top-level section of the bundle (`runs`, `sessions`, …) isn't the JSON list the format defines — the file was hand-edited or truncated. The section was skipped; everything else imported. **Fix.** Re-export and import the unmodified file. ### An imported webhook trigger doesn't fire **Cause.** Webhook signing secrets never travel in a bundle, so webhook triggers import **disabled** and secretless — an armed webhook with no secret would reject every caller. **Fix.** Set a fresh secret with `PATCH /triggers/{id}` (body `{"config": {"secret": "..."}}`), then enable the trigger. The same rule explains imported connections and MCP servers failing with auth errors: re-enter their secrets and env values — only key *names* were exported. ## AI gateway ### Claude Code shows an error from the Ertir AI gateway The gateway names the reason in its message (and in the `x-theygent-gateway-error` header): a missing or revoked gateway key, no Anthropic credential where you are billed to your own, no organization key on the gateway, or Anthropic unreachable. Each code and its fix: [errors the gateway returns](../ai-gateway/running.md#errors-the-gateway-returns). ### The AI Gateway page shows no gateway, or no traffic A gateway appears on the **Overview** only when it is attached to the control plane — `THEYGENT_CONTROL_PLANE_URL` and a service key ([Running the gateway](../ai-gateway/running.md#attach-it-to-the-control-plane)). An attached gateway records nothing until an admin saves a configuration in **Observer** mode on the Settings tab; and Claude Code reaches it only with `ANTHROPIC_BASE_URL` pointing at it ([Set up Claude Code](../ai-gateway/setup.md)). ### The Settings tab says it can't reach the gateway The organization's key is read and written on the gateway itself, from your browser. Set the gateway's address on **Setup & keys**, and if the interface is served from another origin than `http://localhost:5174`, add that origin to `THEYGENT_AI_GATEWAY_ADMIN_ORIGINS`. ### Requests I expected on a local model went to Anthropic Open the request on the **Sessions** tab: its reasons say why. *No local model fits*: no listed model takes tool calls, images or a prompt that size — list one that does, with its real context. *Outgrew local model*: the conversation grew past every listed model that could take it — list a larger one, and make sure developers use the current setup from **Setup & keys**, which tells Claude Code to compact in time. *Target refused*: the local model could not take it — an error, no answer in time, an unusable answer, a document or server tool it can't be given, or no inference plane at `THEYGENT_INFERENCE_PLANE_URL`; the conversation then stays with Anthropic until its next cold point. *Would route →* a local model: the gateway is in observer mode, which only records. No local reason at all: the conversation is outside the share, or it started before the share was raised and moves at its next cold point. Some requests never go local, such as Claude Code's quota check, and compaction goes local only with its conversation. See [Local models](../ai-gateway/local-models.md). ### Claude Code shows a higher cost than the Report In router mode Claude Code prices every token at the model it asked for, even when a cheaper model or a local one answered. The Report prices what actually ran, and shows Claude Code's figure as *without routing*. See [Claude Code's own cost display](../ai-gateway/setup.md#claude-codes-own-cost-display). ### The Report warns that tool search is off Claude Code turns MCP tool search off for any custom base URL and then sends every tool's definition with every request. Set `ENABLE_TOOL_SEARCH=true` (the desktop app: tool search on). See [why each setting](../ai-gateway/setup.md#why-each-setting). ## Browser and interface ### The interface can't reach a plane / CORS errors in the console **Cause.** Your browser origin isn't in the allowed CORS list, or a plane is on a non-default port the interface wasn't built to call. **Fix.** Add your origin to `THEYGENT_CORS_ORIGINS` (comma-separated), and if you moved a plane's port, rebuild the interface with the matching `VITE_CONTROL_PLANE_URL` / `VITE_INFERENCE_URL`. See [Environment variables](environment.md). ### The model catalog is empty — "No local engine is ready" **Cause.** The catalog only lists models you can actually run, and no local engine is installed yet. **Fix.** Install at least one engine (`make engines` on macOS, or `llama.cpp` on any platform), then refresh. See [Installing local models](../models/installing.md). ## Related pages - [API reference](api.md) — where each error code originates. - [Environment variables](environment.md) — the settings behind most configuration issues. - [Running Ertir](../getting-started/running.md) — starting, stopping, and checking the services.