Skip to main content

Generate Text

Generates high-quality text content using Google Vertex AI's PaLM 2 text generation models (Text Bison).

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.
info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs​

  • Connection Id - Vertex AI client session identifier from Connect node (optional if credentials provided directly).
  • Credentials - Google Cloud service account credentials (optional if using Connection ID).
  • Project Id - Google Cloud Project ID (required if using direct credentials).
  • Prompt - Text generation prompt describing what content to generate (required).

Options​

Generation Parameters​

  • Temperature - Controls randomness (0.0-1.0). Lower values are more deterministic. Higher values are more creative.
  • Max Output Tokens - Maximum tokens to generate. Default is 256.
    • text-bison@latest and text-bison-32k: 1-2048 tokens
    • text-bison@001: 1-1024 tokens
  • TopK - Top-K sampling parameter for token selection. Default is 40. Limits vocabulary at each step.
  • TopP - Top-P (nucleus) sampling parameter (0.0-1.0). Default is 0.95. Controls diversity of responses.
  • Stop Sequence - Array of sequences that stop generation when encountered. Optional.
  • Candidate Count - Number of response variations to generate. Default is 1.

Model Configuration​

  • Model - Vertex AI text generation model to use:
    • text-bison@001 - Stable version for production (default)
    • text-bison - Latest version with improvements
    • text-bison-32k - Extended context window (32k tokens)
    • Custom Model - Specify your own model name
  • Custom Model - Custom model name when "Custom Model" is selected.

Endpoint Configuration​

  • Locations - Google Cloud region for the Vertex AI endpoint. Default is "us-central1".
  • Publishers - Model publisher (typically "google"). Default is "google".

Output​

  • Response - Full API response object containing generated text.

Response structure:

{
  "predictions": [
    {
      "content": "Generated text content here...",
      "safetyAttributes": {
        "categories": [],
        "blocked": false,
        "scores": []
      },
      "citationMetadata": {
        "citations": []
      }
    }
  ]
}

How It Works​

The Generate Text node creates text content using Google's PaLM 2 models. When executed:

  1. Validates connection (either via Connection ID or direct credentials)
  2. Retrieves authentication token and project ID
  3. Validates that prompt is not empty
  4. Parses and validates generation parameters (temperature, tokens, etc.)
  5. Constructs request payload with prompt and parameters
  6. Sends POST request to Vertex AI predict endpoint
  7. Processes response and extracts generated text
  8. Returns complete response object with text and metadata

The Text Bison models are optimized for a wide range of text generation tasks including summarization, question answering, creative writing, and content generation.

Requirements​

  • Either:
    • Connection ID from Connect node, OR
    • Direct credentials + Project ID
  • Prompt (non-empty string)
  • Max Output Tokens (required, must be set)
  • Vertex AI API enabled in Google Cloud project
  • IAM permissions: aiplatform.endpoints.predict

Error Handling​

Common errors and solutions:

ErrorCauseSolution
ErrInvalidArgEmpty promptProvide a valid text generation prompt
ErrInvalidArgMax Output Tokens emptySet Max Output Tokens value
ErrInvalidArgConnection ID or credentials missingUse Connect node or provide credentials
ErrInvalidArgInvalid model selectionSelect a valid model or provide custom model name
ErrNotFoundConnection not foundVerify Connection ID from Connect node
ErrStatusAPI error (quota, safety)Check Google Cloud Console for API status
Parse errorInvalid parameter valueVerify temperature, topK, topP are valid numbers
Content blockedSafety filters triggeredAdjust prompt or review safety settings

Example Use Cases​

Product Description Generation​

Prompt: "Write a compelling product description for wireless noise-cancelling headphones with 30-hour battery life and premium sound quality. Include key features and benefits."
Temperature: 0.7
Max Output Tokens: 200
Candidate Count: 1

Email Template Creation​

Prompt: "Generate a professional follow-up email template for sales prospects who attended our product demo but haven't responded yet. Keep it friendly and concise."
Temperature: 0.5
Max Output Tokens: 300
TopP: 0.9

Content Summarization​

Prompt: "Summarize the following article in 3 bullet points: [article text here]"
Temperature: 0.2
Max Output Tokens: 150
Model: text-bison@001

Creative Writing​

Prompt: "Write a short story opening about a detective discovering an unusual clue at a crime scene. Make it suspenseful and engaging."
Temperature: 0.9
Max Output Tokens: 500
Candidate Count: 3
Use Case: Generate multiple variations and pick the best

FAQ Generation​

Prompt: "Generate 5 frequently asked questions and answers about cloud backup services for small businesses."
Temperature: 0.4
Max Output Tokens: 600
Stop Sequence: ["---"]

Data Extraction and Formatting​

Prompt: "Extract the invoice number, date, total amount, and vendor name from this text and format as JSON: [invoice text]"
Temperature: 0.0
Max Output Tokens: 100
Model: text-bison@001

Tips​

  • Temperature Settings:

    • 0.0-0.3: Factual, deterministic outputs (summaries, data extraction)
    • 0.4-0.7: Balanced creativity and consistency (emails, descriptions)
    • 0.8-1.0: Highly creative outputs (stories, brainstorming)

  • Model Selection:

    • text-bison@001: Stable, predictable for production
    • text-bison: Latest improvements, may change over time
    • text-bison-32k: For longer context (documents, conversations)

  • Token Management:

    • ~4 characters = 1 token
    • 100 tokens ≈ 75 words
    • Set Max Output Tokens based on expected response length

  • Prompt Engineering:

    • Be specific and clear in your prompts
    • Provide examples for better results
    • Use delimiters (""", ###) to separate sections
    • Specify format (bullet points, JSON, paragraph)

  • Quality Control:

    • Use Candidate Count > 1 for critical content
    • Lower temperature for consistency
    • Test prompts with different parameter combinations
    • Review safetyAttributes in response

Common Patterns​

Batch Text Generation​

Flow:
1. Connect (once)
2. Loop through items:
   - Set prompt from item data
   - Generate Text
   - Store response with item
3. Disconnect
Benefit: Efficient for multiple generation requests

Multi-Candidate Selection​

Configuration:
- Candidate Count: 3
- Temperature: 0.7
Process:
1. Generate 3 variations
2. Evaluate each candidate
3. Select best based on criteria
4. Use selected text

Controlled Output Length​

Settings:
- Max Output Tokens: 150
- Stop Sequence: ["\n\n", "---"]
Result: Generation stops at token limit or stop sequence

Parameter Guidelines​

Temperature vs TopP​

  • Temperature: Controls randomness globally
  • TopP: Limits cumulative probability of tokens
  • Recommendation: Adjust temperature first, then TopP if needed

TopK Settings​

  • Low (1-20): Very focused, repetitive
  • Medium (20-60): Balanced (40 is default)
  • High (60-100): More diverse vocabulary

Stop Sequences​

  • Use for structured outputs (JSON, lists)
  • Common: ["\n\n", "###", "---", "END"]
  • Can specify multiple sequences

Performance Optimization​

  • Connection Reuse: Use same connection for multiple requests
  • Batch Processing: Generate multiple items in sequence
  • Token Efficiency: Set appropriate Max Output Tokens
  • Regional Endpoints: Use closest region
  • Caching: Cache common generation patterns
  • Prompt Templates: Reuse well-tested prompts

Best Practices​

  • Start with lower temperature and increase if needed
  • Test prompts with small batches before production
  • Monitor token usage and costs in Google Cloud Console
  • Use text-bison@001 for stable production workloads
  • Implement error handling for content safety blocks
  • Log prompts and responses for quality improvement
  • Use specific, detailed prompts for better results
  • Consider using text-bison-32k for long-context tasks
  • Set reasonable Max Output Tokens to control costs
  • Review and validate generated content before use

Safety Attributes​

The response includes safety metadata:

  • categories: Content safety categories
  • blocked: Whether content was blocked
  • scores: Safety scores for each category

Monitor these attributes to ensure generated content meets your standards.