Skip to main content

Click & Type

Locates an image on the screen, clicks it, and types specified text into the clicked area.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.

Inputs​

  • Text - The text to type after clicking the image.

Options​

  • Confidence - Confidence level for image recognition (default: 0.95).
  • Clear Before Typing - Whether to clear existing content before typing (boolean).
  • Press Enter - Whether to press Enter after typing (boolean).

Outputs​

This node does not have outputs.

How It Works​

The Click & Type node finds a specified image on the screen, clicks it, and types text. When executed, the node:

  1. Downloads and prepares the reference image if needed
  2. Uses pyautogui to locate the image on screen with the specified confidence level
  3. Moves the mouse cursor to the image's position (adjusted by deltaX and deltaY if specified)
  4. Clicks the left mouse button at the image location
  5. Optionally clears existing content (if Clear Before Typing is enabled)
  6. Types the specified text
  7. Optionally presses Enter (if Press Enter is enabled)

Requirements​

  • Valid reference image for matching
  • Valid text to type
  • Confidence value between 0 and 1

Error Handling​

The node will return specific errors in the following cases:

  • Empty image - "Empty Image"
  • Could not read image - "Could not read image"
  • Empty text - "Text can not be empty"
  • Missing confidence - "Confidence can not be empty"
  • Invalid confidence format - "Confidence is not valid"
  • Confidence out of range - "Confidence must be between 0 and 1"
  • Image not found - "Could not find image. Decreasing confidence may help"

Usage Notes​

  • This node is useful for automating form filling or text input in UI elements
  • The confidence level affects the accuracy of image matching
  • Higher confidence values require more precise image matching
  • Lower confidence values may result in false positives but can handle variations better
  • The deltaX and deltaY properties can be used to adjust the click position relative to the found image
  • The Clear Before Typing option is useful for replacing existing content
  • The Press Enter option is useful for submitting forms after text input
  • Works best with high-contrast, static images that represent clickable input fields