DocsWorkflowsComponentsLLMllm:generate

llm:generate

Makes LLM inference requests using the configured provider and its model. Supports tools, LLM parameter settings and cost calculation with user currency conversion. Pass a template path to the `prompt` field, or a string prompt to render dynamic prompts with variables.

Options

Name	Type	Description
`prompt`	`string`	The prompt to be used for the inference it should be a string or a file path
`prompt_vars`	`map[string]any`	A map of variables to be used in the prompt template
`provider`	`string`	The inference provider (i.e. "ollama", "openai", etc.)
`connection`	`string`	Names the approved connector carrying this provider's API key. It is optional when exactly one approved inference connector is available.
`model`	`string`	The model to be used for the inference (i.e. "mistral:latest", "gpt-4", etc.)
`temperature`	`float32`	The model temperature setting
`top_p`	`float32`	The model's top_p setting
`max_tokens`	`int`	The maximum number of tokens to be generated by the model Providers like Anthropic require this to be set
`max_thinking_tokens`	`int`	The maximum number of tokens to be used for thinking
`currency`	`string`	The currency to be used for the cost calculation
`think`	`bool`	Whether to enable model thinking before answering
`think_level`	`int`	The level of thinking to be used by the model: `0` - no thinking `1` - basic thinking `2` - advanced thinking
`tools`	`[]string`	Tool names to their configurations
`tool_choice`	`ToolChoice`	The tool choice strategy to be used by the model, which is one of the four the inference package accepts. `none` - the model will not use tools `auto` - the model will choose tools by itself based on the prompt `required` - the model will call some tool, and picks which `named` - the model will call the tools ToolNames lists, or the only tool offered when it lists none A value outside those four fails the request rather than falling back to a strategy the workflow did not ask for.

Outputs

Name	Type	Description
`provider`	`string`	The inference provider
`model`	`string`	The model used for the inference (i.e. "mistral:latest", "gpt-4", etc.)
`prompt`	`string`	The full prompt rendered
`tools`	`map[string]string`	The tools generated by the model in `{tool_name: raw json}` format
`content`	`string`	The text response from the model

Examples

name: Summarize a page
actions:
  - name: page
    component: http:request
    vars:
      url: "{{ var.url }}"

  - name: summary
    component: llm:generate
    vars:
      provider: anthropic
      model: claude-opus-4-20250514
      prompt: "Summarize the following in three bullet points:\n\n{{ page.body }}"
      max_tokens: 1024

outputs:
  summary: "{{ summary.content }}"

Structured output

Gets JSON back instead of free text. There is no separate JSON mode. The prompt file defines a tool whose parameters are the JSON Schema of the wanted shape, tool_choice: specific forces the model to call it, and the arguments the model fills in come back in the result's tools map as raw JSON, parsed out with the standard key/from:'json' chain. The inference guide covers the prompt file's ---tool section.

name: Extract invoice fields
actions:
  - name: extracted
    component: llm:generate
    cache:
      for: 24h
    retry:
      count: 3
      delay: 10
    vars:
      provider: openai
      model: gpt-5
      prompt: file:./extract_invoice.md
      prompt_vars:
        text: "{{ var.invoice_text }}"
      tools:
        - extract_invoice
      tool_choice: specific

  - name: invoice
    component: var
    vars:
      var: fields
      value: "{{ extracted.tools | key:'extract_invoice' | from:'json' | default:[] }}"

outputs:
  invoice: "{{ invoice.fields }}"

Chained models

Chains two calls. The first extracts the facts, the second writes from what the first returned, and the only wiring between them is one template reference.

name: Facts, then a post
actions:
  - name: facts
    component: llm:generate
    vars:
      provider: openai
      model: gpt-5
      prompt: "List three surprising facts about {{ var.topic }}."

  - name: post
    component: llm:generate
    vars:
      provider: anthropic
      model: claude-opus-4-20250514
      prompt: "Turn these facts into one short, upbeat post.\n\n{{ facts.content }}"
      max_tokens: 1024

outputs:
  post: "{{ post.content }}"

Thinking

Enables extended reasoning. think turns it on, think_level picks the depth, and max_thinking_tokens caps the reasoning budget. Reasoning tokens are priced and tracked like input and output tokens.

name: Reason through a decision
actions:
  - name: answer
    component: llm:generate
    vars:
      provider: anthropic
      model: claude-opus-4-20250514
      prompt: "Given the brief below, which supplier quote is the best fit, and why?\n\n{{ var.brief }}"
      think: true
      think_level: 2
      max_thinking_tokens: 2048
      max_tokens: 1024

outputs:
  answer: "{{ answer.content }}"

Edit or read the source on GitHub