Skip to main content

Generate Chat Completion

Generates conversational AI responses using a local Ollama model with support for message history and context. This node is designed for building chatbots, interactive assistants, and multi-turn conversations.

tip

For single-prompt text generation without conversation history, use Generate Completion instead for better performance.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.
info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs​

  • Client ID - The Client ID from the Connect node. Optional if Host URL is provided.
  • Model - The name of the Ollama model to use (e.g., llama3, mistral, gemma).
  • Messages - An array of message objects representing the conversation history. Each message must have:
    • role - Either "system", "user", or "assistant"
    • content - The text content of the message

Output​

  • Response - The generated chat response from the AI model.

Options​

  • Options - A JSON object containing model parameters to control generation behavior:
    • temperature (0.0-2.0) - Controls randomness. Lower = more focused, Higher = more creative. Default: 0.8
    • top_p (0.0-1.0) - Nucleus sampling for diversity control. Default: 0.9
    • top_k (integer) - Limits vocabulary to top K tokens
    • num_predict (integer) - Maximum tokens to generate. Default: 128, -1 for unlimited
    • repeat_penalty (0.0-2.0) - Penalty for repeating tokens. Default: 1.1
    • seed (integer) - Random seed for reproducible outputs
    • num_ctx (integer) - Context window size in tokens. Default: 2048
  • Host URL - Ollama server URL (optional). Use instead of Client ID for direct connection. Example: http://localhost:11434

How It Works​

The Generate Chat Completion node:

  1. Connects to the Ollama server (via Client ID or Host URL)
  2. Processes the conversation history from the Messages array
  3. Sends the complete context to the model
  4. Receives the AI-generated response
  5. Returns the response text

Message Format​

Messages must be structured as an array of objects:

[
  {
    "role": "system",
    "content": "You are a helpful customer service assistant."
  },
  {
    "role": "user",
    "content": "What is your return policy?"
  },
  {
    "role": "assistant",
    "content": "Our return policy allows returns within 30 days."
  },
  {
    "role": "user",
    "content": "What about exchanges?"
  }
]

Role Types​

  • system - Sets the AI's behavior, personality, and context (optional but recommended)
  • user - Messages from the human user
  • assistant - Previous responses from the AI model

Usage Examples​

Example 1: Simple Customer Service Bot​

Inputs:
- Model: "llama3"
- Messages: [
    {
      "role": "system",
      "content": "You are a friendly customer support agent."
    },
    {
      "role": "user",
      "content": "I need help with my order #12345"
    }
  ]

Output:
- Response: "I'd be happy to help you with order #12345. Could you please tell me what issue you're experiencing with your order?"

Example 2: Multi-Turn Conversation​

Inputs:
- Model: "mistral"
- Messages: [
    {
      "role": "system",
      "content": "You are a technical support specialist."
    },
    {
      "role": "user",
      "content": "My computer won't start"
    },
    {
      "role": "assistant",
      "content": "Let's troubleshoot this. Are you seeing any lights or hearing any sounds when you press the power button?"
    },
    {
      "role": "user",
      "content": "No lights or sounds at all"
    }
  ]

Output:
- Response: "This suggests a power supply issue. Please check if the power cable is firmly connected to both the computer and the wall outlet. Also, try a different power outlet to rule out outlet problems."

Example 3: Data Analysis Assistant​

Inputs:
- Model: "llama3"
- Messages: [
    {
      "role": "system",
      "content": "You are a data analyst. Provide insights in bullet points."
    },
    {
      "role": "user",
      "content": "Analyze this sales data: Q1: $50k, Q2: $75k, Q3: $60k, Q4: $90k"
    }
  ]

Output:
- Response: "
  • Revenue shows strong growth trend (+80% from Q1 to Q4)
  • Q2 peak likely due to seasonal demand
  • Q3 dip suggests market correction or competitive pressure
  • Q4 surge indicates successful year-end strategy
  • Overall annual performance: positive trajectory
"

Example 4: Code Review Assistant​

Inputs:
- Model: "codellama"
- Messages: [
    {
      "role": "system",
      "content": "You are an expert code reviewer. Provide constructive feedback."
    },
    {
      "role": "user",
      "content": "Review this code: function add(a,b){return a+b}"
    }
  ]
- Options: {
    "temperature": 0.3
  }

Output:
- Response: "The function works but could be improved:
  1. Add type annotations for parameters
  2. Add input validation
  3. Consider edge cases (null, undefined)
  4. Add JSDoc comments
  Suggested: function add(a: number, b: number): number { ... }"

Example 5: Building Conversation History in Loop​

// Initialize conversation
let conversationHistory = [
  {
    role: "system",
    content: "You are a helpful assistant for RPA automation."
  }
];

// First user message
conversationHistory.push({
  role: "user",
  content: "How do I read Excel files?"
});

// Get AI response (use Generate Chat Completion node)
// Response: "You can use the Excel package in Robomotion..."

// Add assistant's response to history
conversationHistory.push({
  role: "assistant",
  content: response
});

// Continue conversation
conversationHistory.push({
  role: "user",
  content: "Can you show me an example?"
});

// Generate next response with full context

Requirements​

  • Ollama service must be running
  • The specified model must be pulled locally (use Pull Model)
  • Valid Client ID from Connect node OR Host URL provided
  • Messages array must contain at least one message

Common Use Cases​

  • Customer service chatbots
  • Interactive technical support systems
  • Conversational data analysis
  • Educational tutoring bots
  • Code review and assistance
  • Multi-step troubleshooting workflows
  • Interactive form filling assistance
  • Personalized recommendation systems

Tips​

Effective System Prompts​

Good system prompts set clear expectations:

{
  "role": "system",
  "content": "You are a professional email writer. Write concise, formal emails. Always include a greeting and signature. Keep responses under 200 words."
}

Managing Conversation Length​

Long conversations can slow down responses and consume memory:

  • Limit message history to last 10-20 messages
  • Summarize older messages into a single context message
  • Remove unnecessary intermediate messages
  • Use higher num_ctx for longer conversations

Maintaining Context Across Nodes​

Store conversation history in a message variable:

  1. Initialize as empty array
  2. Add user message before each chat completion
  3. Add AI response after each chat completion
  4. Pass updated history to next chat completion node

Temperature Guidelines for Chat​

  • 0.1-0.3 - Consistent, factual responses (support, documentation)
  • 0.5-0.7 - Balanced responses (general chatbot)
  • 0.8-1.0 - Engaging, varied responses (creative assistant)

Error Handling​

Common errors you might encounter:

  • "Model name cannot be empty" - Provide a valid model name
  • "Either Host URL or Client ID must be provided" - Provide one connection method
  • Invalid messages format - Ensure messages is an array of objects with role and content
  • Empty messages array - Provide at least one message
  • Model not found - Pull the model using Pull Model node

Best Practices​

  1. Always include a system message - It guides the AI's behavior
  2. Keep message history manageable - Trim old messages to improve performance
  3. Use consistent role patterns - Alternate between user and assistant
  4. Store conversation state - Save message history in variables or database
  5. Handle long responses - Use num_predict to control output length
  6. Test with different models - Some models excel at conversation over others

Advanced Example: Stateful Chatbot​

// In your flow, maintain state
if (!msg.conversationHistory) {
  msg.conversationHistory = [
    {
      role: "system",
      content: "You are a helpful RPA automation expert."
    }
  ];
}

// Add new user message
msg.conversationHistory.push({
  role: "user",
  content: msg.userInput
});

// Generate Chat Completion node receives: msg.conversationHistory
// Outputs: response

// After getting response, add it to history
msg.conversationHistory.push({
  role: "assistant",
  content: msg.response
});

// Trim history if too long (keep last 20 messages + system)
if (msg.conversationHistory.length > 21) {
  msg.conversationHistory = [
    msg.conversationHistory[0], // Keep system message
    ...msg.conversationHistory.slice(-20) // Keep last 20
  ];
}