LLM step

What it does#

An LLM step sends your instructions to an AI model and keeps its answer. If it's the last step in the flow, that answer is the reply the customer gets. Otherwise later steps can use it as {{StepName}}.

Use one LLM step for a simple assistant, or several for a team: one classifies the message, another writes the answer, a third checks it.

Set up an LLM step#

1. 1

   Open the step

   Double-click the step on the canvas. New steps are LLM steps; if it shows another type, pick LLM in the type menu in the top right.

2. 2

   Name it

   Click the name at the top and type a short name such as Answer or Classify. Use English letters, numbers and _ only. Later steps use this name.

3. 3

   Choose a model

   Under Model, open the picker and choose one. Start with the one marked Default.

4. 4

   Write the System prompt

   Under Instructions, describe the role, tone and rules. Type @ to insert a variable.

5. 5

   Write the Prompt

   This is the message the model answers. New steps start with {{question}}, the customer's latest message.

6. 6

   Test it

   Close the panel and click Preview, or right-click the step and choose Run once. See Run, preview and publish.

Changes in the panel apply to the flow right away; they're saved the next time you click Preview or Publish (inside a chatbot they save on their own).

Choose a model#

The picker shows a Recommended group first, then every model grouped by provider. Each model has tags and details:

• Default: the model new steps get.
• Recommended, Fast or Powerful: a quick guide to cost and quality.
• Capabilities such as Tool use, Image input, Structured output and Answers only from your sources.
• The context size and price per 1M tokens. On accounts that use AI credits, you also see about how many credits per 100 replies the model uses.

A good rule: a Fast model for short jobs (classify, extract, yes or no), and a Recommended or Powerful model for the reply your customer reads.

Some entries need one more choice:

• Open models on Together AI: pick the exact model under Choose your model.
• Azure OpenAI deployment: runs on your own Azure deployment. Type its name in Azure deployment name.

If a model is retired by its provider, the step shows a notice and a Switch to … button. Click it so the step keeps working the way you tested it.

Set the model from a variable#

Turn on Set the model from a variable to choose the model while the flow runs, for example from a field an earlier step returned. Write a model id, or type @ to insert a variable.

{{Router.model}}

The value must be a model id Officely knows, such as gpt-4o-mini or claude-haiku-5-5. If it isn't, the step stops and the error lists the ids you can use.

Write the instructions#

The System prompt sets the rules. The Prompt is what the model answers. Both accept variables:

You are Dana, the friendly support assistant of Bloom Flowers.
Answer in the customer's language, in 2 to 4 short sentences.
If the order needs a refund, say a teammate will follow up.

Use this information to answer:
{{Knowledge.context}}

Conversation so far:
{{chat_history}}

Question: {{question}}

Turn on Answer only from the sources when the step answers from your knowledge. It adds a tested instruction: answer only from the information given, say "I don't know" otherwise, and never invent prices, hours or policies.

A few models (tagged Answers only from your sources) read only the system prompt. The step tells you when you pick one: put {{Knowledge.context}} in the System prompt and the question in the Prompt.

Answer with a schema#

Turn on Answer with a schema under Output schema to get named fields instead of free text. It's the easiest way to feed a Filter step or a Python step. The section appears only for models that support structured output.

1. 1

   Turn it on

   Switch on Answer with a schema. The panel shows how later steps use the fields, such as {{Classify.field}}.

2. 2

   Add fields

   Pick a preset under Presets (Lead, Classification, Sentiment, Yes or no), click Add field, or Paste a JSON example and click Create fields.

3. 3

   Describe each field

   Give each field a Field name (English letters, numbers and _), a Description that tells the AI what to put there, and a Type.

4. 4

   Check the shape

   Open the Preview tab to see the Answer shape and copy variables with Copy variable or Copy for Python.

{
  "intent": "order_status",
  "urgent": false,
  "order_id": "3234379"
}

Pasting this creates three fields: intent (Text), urgent (True/False) and order_id (Text). Change intent to One of… with the values order_status, refund and other so the answer is always one you can filter on. Lists and nested objects are skipped.

Each field is Required by default: the AI must always fill it. Mark a field Optional if the customer may not have given it, like an email address. A missing optional field is empty for later steps.

Common mistakes#

• "Missing placeholders" error. A prompt uses a variable that doesn't exist, often after renaming a step. Type @ and pick the variable again.
• The step ignores your knowledge. Check the prompt really contains {{Knowledge.context}} and that the Knowledge step runs before this one.
• Can't find Answer with a schema. The selected model doesn't support structured output. Pick a model with the Structured output capability.
• The model from a variable fails. The variable held a label like "GPT-4o mini" instead of an id like gpt-4o-mini.
• Answers vary too much for a Filter. Use One of… fields instead of free text, and describe each allowed value.