Inference

The inference component calls a language model. It renders a prompt, sends it to a provider, and returns the model's text along with any structured tool output, priced per call. It is a component like any other, so its result flows to later steps the same way an HTTP response does.

That last point is the whole idea. Because a model call is just a step, you chain model calls the same way you chain anything else. One call extracts the fields you need, the next reasons over them, a third writes the final answer, and every one of them can hand structured data to the step after it. Most of the interesting work here is a short chain of model calls with ordinary steps (an HTTP fetch, a file write, a query) threaded between them.

A basic call

- name: summary
  component: inference
  vars:
    provider: anthropic
    model: claude-opus-4-20250514
    prompt: "Summarise in three bullet points: {{ page.content }}"
    max_tokens: 1024

The result carries content (the model's text), provider, model, prompt (the fully rendered prompt), and tools (structured output, covered below). Read the text downstream as {{ summary.content }}.

Providers and parameters

provider and model are required. The provider registry covers OpenAI, Anthropic, Google, OpenRouter, and any OpenAI-compatible endpoint.

Prompts as files

A prompt can be a string, or a file: path to a prompt template. A prompt file separates the tool definition, the system prompt, and the user prompt with section markers, and declares its own default variables in frontmatter. Values from prompt_vars fill the template.

- name: translated
  component: inference
  cache:
    for: 24h
  retry:
    count: 3
    delay: 10
  vars:
    provider: openai
    model: gpt-5
    prompt: file:./generate_rows.md
    prompt_vars:
      language_from: "{{ var.language_from }}"
      language_to: "{{ var.language_to }}"
      rows: "{{ var.rows | default:[] }}"
    tools:
      - translate_rows
    tool_choice: specific

The prompt file it points at:

---
language_from: 'en'
language_to: 'lt'
rows: []
---

---tool

name: translate_rows
description: |
  Return the translated rows. Each output row MUST preserve the original key exactly.
parameters:
  type: object
  required: [rows]
  properties:
    rows:
      type: array
      items:
        type: object
        required: [key, value]
        properties:
          key: { type: string }
          value: { type: string }

---system

You are a professional localisation translator.

---user

Translate the following rows from `{{ language_from }}` into `{{ language_to }}`:

```json
{{ rows | json:'pretty' }}
```

The frontmatter block holds default values. The ---tool section is a tool definition (a name, a description, and a JSON-schema parameters block). The ---system and ---user sections are the two halves of the conversation, and they support the same {{ }} expressions and {{ if }} blocks as everywhere else.

Structured output through tools

There is no separate JSON-mode or schema field. Structured output is a forced tool call. You define a tool whose parameters are the JSON Schema of the shape you want, then set tool_choice: specific so the model must call it. The model fills the tool's arguments, and those arguments are the structured result.

The result's tools field is a map from tool name to the raw JSON the model produced. Read a field out of it with the modifier chain below.

- name: output
  component: var
  vars:
    var: rows
    value: "{{ translated.tools | key:'translate_rows' | from:'json' | key:'rows' | default:[] }}"

The chain reads the translate_rows entry, parses it from JSON, then pulls the rows field. The default:[] guards against a call that returned nothing. Free-form answers still come back as content. Tool output and text are not mutually exclusive.

Tools point in both directions. A workflow uses tools to get structured data out of a model, and a workflow that declares its inputs and an output_schema is itself a tool an external model or agent can call over MCP. The same typed contract that shapes a model's answer inside a run is what lets an agent outside the run invoke the workflow and read its result. Building a workflow is building something a model can use.

Pricing every call

Every inference is priced by the cost accountant in two currencies at once. The provider cost is computed in USD, and the client cost is converted into the call's currency. Both are tracked separately across the estimate, the input tokens, the output tokens, and any reasoning tokens, then attached to the step's metrics. When a model's pricing or tokenizer is unknown the cost degrades to zero rather than failing the run. See lifecycle for how those numbers are aggregated per run.

Estimating before you spend

Two companion components help you plan a call without paying for it. inference:estimate renders the prompt, counts the tokens, and returns the projected input cost without calling the model, so an if can abort an expensive run. inference:models lists the models a provider exposes at runtime.

- name: estimate
  component: inference:estimate
  vars:
    provider: openai
    model: gpt-5
    prompt: "{{ var.prompt }}"
    currency: EUR

The estimate returns tokens_input, cost_input_usd, and cost_input_converted, which pairs naturally with the budget-guard error step from control flow.

Once a workflow is doing real work, composition shows how to build a larger one out of it.