Skip to main content

Generate Chat

Generates chat responses using Google Vertex AI's chat models.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.
info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs​

  • Connection Id - The unique identifier of the connection to Vertex AI, typically obtained from the Connect node.
  • Author of The Message - The author of the message (e.g., "user"). Default is "user".
  • Content of The Message - The content of the message to send to the chat model (e.g., "How many planets are there in the solar system").

Options​

  • Context - Additional context for the conversation (optional).
  • Examples Input - Example input for the model (must be paired with Examples Output).
  • Examples Output - Example output for the model (must be paired with Examples Input).
  • Temperature - Controls randomness in the response (0.0-1.0). Lower values make responses more deterministic.
  • Max Output Tokens - Maximum number of tokens in the response (1-2048).
  • TopK - Limits the number of highest probability vocabulary tokens considered at each step.
  • TokP - Nucleus sampling probability threshold (0.0-1.0).
  • Stop Sequence - Sequences where the API will stop generating further tokens.
  • Candidate Count - Number of response candidates to generate. Default is 1.
  • Model - The chat model to use. Options include:
    • Custom Model
    • Chat Bison@001
    • Chat Bison
    • Chat Bison 32k
    • Default is Chat Bison@001.
  • Custom Model - Custom model name if "Custom Model" is selected.
  • Locations - Google Cloud region for the service. Default is "us-central1".
  • Publishers - Publisher of the model. Default is "google".

Output​

  • Response - The generated chat response from the Vertex AI model as an object.

How It Works​

The Generate Chat node sends a message to a Vertex AI chat model and returns the model's response. When executed, the node:

  1. Validates the connection ID and retrieves the authentication token
  2. Validates all required inputs (message author and content)
  3. Collects all optional parameters and configurations
  4. Constructs a request with the message and parameters
  5. Sends the request to the Vertex AI chat model endpoint
  6. Processes the response and returns it

Requirements​

  • A valid connection to Vertex AI established with the Connect node
  • Valid Google Cloud credentials with appropriate permissions
  • A properly configured Vertex AI chat model

Error Handling​

The node will return specific errors in the following cases:

  • Empty or invalid Connection ID
  • Empty message author or content
  • Invalid parameter values (e.g., temperature outside 0.0-1.0 range)
  • Missing required parameters
  • Invalid model selection
  • Network connectivity issues
  • Vertex AI service errors
  • Authentication failures

Usage Notes​

  • The Connection ID must be obtained from a successful Connect node execution
  • The Author of The Message is typically "user" for user messages
  • The Content of The Message should be a clear, well-formulated question or instruction
  • Temperature controls creativity: lower values (0.2) for factual responses, higher values (0.8) for creative responses
  • Max Output Tokens limits response length
  • Context can be used to provide background information for the conversation
  • Examples Input/Output pairs can help guide the model's behavior
  • Stop sequences can control where the model stops generating text
  • Multiple candidates can be generated by increasing Candidate Count
  • Different models offer different capabilities and token limits
  • The Locations parameter should match where your Vertex AI resources are deployed