Skip to main content

Actions

Performs image-based automation actions such as clicking images and typing text based on image recognition.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.

Inputs​

This node does not have direct inputs. It works with regions defined in the workflow designer.

Options​

  • Confidence - Confidence level for image recognition (default: 0.95).

Outputs​

This node does not have direct outputs.

How It Works​

The Actions node performs image-based automation tasks. When executed, the node:

  1. Downloads and prepares the reference image if needed
  2. Takes a screenshot of the current screen
  3. Identifies target regions defined in the workflow
  4. For each target region:
    • If it's a ClickImage action, performs a click at the identified location
    • If it's a ClickType action, clicks at the location and types the specified text
  5. Uses image recognition to find anchor points and target coordinates
  6. Applies confidence thresholds to ensure accurate image matching

Requirements​

  • Valid reference image for image recognition
  • Confidence value between 0 and 1
  • Properly defined regions in the workflow designer

Error Handling​

The node will return specific errors in the following cases:

  • Empty image - "Empty Image"
  • No target selected - "No target selected"
  • Could not read image - "Could not read image"
  • Low confidence - "Max confidence is too low(). Could not find template in scene"
  • Invalid confidence value - "Confidence can not be empty", "Confidence is not valid", "Confidence must be between 0 and 1"
  • Empty text for typing actions - "Text can not be empty"

Usage Notes​

  • This node is designed for complex image automation workflows with multiple actions
  • It supports both ClickImage and ClickType actions within a single node
  • The confidence level affects the accuracy of image recognition
  • Higher confidence values require more precise image matching
  • Lower confidence values may result in false positives but can handle variations better
  • The node uses computer vision libraries (OpenCV) for image processing
  • Coordinates are calculated based on screen resolution and image dimensions
  • Actions are performed using pyautogui for mouse and keyboard automation