Skip to main content

Wait Table

Waits until a new table matching the pattern is created in the Microsoft SQL Server database.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.
info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs​

  • Connection Id - The unique identifier of the database connection to use (optional if credentials provided directly).
  • Table Name - Table name pattern to monitor. Can include schema (e.g., dbo.TableName) and supports LIKE wildcards (%, _).

Options​

  • Poll Time (s) - Time interval in seconds between checks for new tables (default: 1.0 second).
  • Credentials - SQL Server database credentials (optional). Use this if you want to monitor without a Connect node.

Output​

  • None - The node completes when a new table matching the pattern is detected.

How It Works​

The Wait Table node monitors the database for new tables matching a specified pattern. When executed, the node:

  1. Retrieves the database connection (either from connection ID or creates new connection from credentials)
  2. Validates the table name pattern is provided
  3. Queries INFORMATION_SCHEMA.TABLES to count tables matching the pattern
  4. On first execution, records the initial count of matching tables
  5. Continuously polls the database at the specified interval
  6. When the count increases (new table created), the node completes
  7. Uses SQL LIKE pattern matching for flexible table name patterns

Requirements​

  • Either: A valid connection ID from Connect node OR valid database credentials
  • Valid table name pattern (can include wildcards)
  • Appropriate SELECT permission on INFORMATION_SCHEMA views
  • Connection must remain open during monitoring

Error Handling​

The node will return specific errors in the following cases:

  • ErrInvalidArg - Table name pattern is empty
  • ErrNotFound - Connection ID not found
  • ErrConnection - Cannot connect to SQL Server (when using credentials mode)
  • ErrRuntime - Query execution failed or connection closed during monitoring

Table Name Patterns​

Exact Table Name​

Table Name: MyNewTable

Waits for a table named exactly "MyNewTable" in the default schema.

With Schema​

Table Name: dbo.Orders

Waits for table "Orders" in schema "dbo".

With Wildcards​

Table Name: Temp%

Waits for any new table starting with "Temp" (e.g., TempData, TempResults).

Table Name: %_Archive

Waits for any new table ending with "_Archive".

Table Name: Report_____

Waits for any table matching pattern "Report" followed by exactly 5 characters (using _ wildcard).

Schema with Wildcards​

Table Name: staging.Import%

Waits for any new table in "staging" schema starting with "Import".

Usage Examples​

Wait for Specific Table​

Connect
  ↓
Wait Table
  - Table Name: dbo.ProcessedData
  - Poll Time: 2.0
  ↓
(triggered when ProcessedData table is created)
  ↓
Query (SELECT * FROM dbo.ProcessedData)
  ↓
Disconnect

Wait for Temp Table Pattern​

Wait Table
  - Table Name: #Temp%
  - Poll Time: 1.0
  ↓
(triggered when any temp table starting with #Temp is created)
  ↓
Process Data

Monitor Staging Schema​

Connect
  ↓
Wait Table
  - Table Name: staging.%
  - Poll Time: 5.0
  ↓
(triggered when any table is created in staging schema)
  ↓
Query (get new table name)
  ↓
Process New Table
  ↓
Disconnect

Common Use Cases​

  1. ETL Monitoring - Wait for staging tables to be created by ETL processes
  2. Data Pipeline Triggers - Trigger automation when upstream creates result tables
  3. Report Generation - Wait for report tables to be created before processing
  4. Multi-System Integration - Coordinate with external systems that create tables
  5. Batch Processing - Wait for batch result tables before continuing
  6. Data Warehouse Loading - Monitor for new fact or dimension tables
  7. Temporary Table Processing - Wait for stored procedures to create temp tables

Tips for Effective Use​

  • Specific Patterns - Use specific patterns to avoid false triggers
  • Poll Interval - Adjust poll time based on expected frequency (longer for infrequent events)
  • Schema Specification - Include schema to avoid ambiguity
  • Wildcard Usage - Use wildcards for flexible pattern matching
  • Connection Management - Consider using credentials mode for long-running monitors
  • Timeout Handling - Implement timeout logic if table creation might never happen
  • Combine with Other Waits - Chain with Wait Column or Wait Row for complete monitoring

Polling Behavior​

How Polling Works​

  1. Initial Check - Counts existing tables matching pattern
  2. Wait Period - Sleeps for Poll Time seconds
  3. Subsequent Check - Counts tables again
  4. Compare - If count increased, triggers; otherwise continues polling
  5. Repeat - Steps 2-4 repeat until new table detected

Poll Time Considerations​

Fast Polling (0.5 - 1 second):

  • Quick response to new tables
  • More database queries
  • Higher resource usage
  • Good for time-sensitive operations

Slow Polling (5 - 10 seconds):

  • Lower resource usage
  • Delayed response
  • Fewer database queries
  • Good for background monitoring

Recommended:

  • 1-2 seconds for active monitoring
  • 5-10 seconds for background monitoring
  • 30+ seconds for very infrequent events

INFORMATION_SCHEMA Query​

The node queries SQL Server's INFORMATION_SCHEMA:

Without Schema in Pattern​

SELECT COUNT(*)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE 'pattern'

With Schema in Pattern​

SELECT COUNT(*)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'schema_name'
  AND TABLE_NAME LIKE 'table_pattern'

Advanced Examples​

Wait for Daily Report Table​

Wait Table
  - Table Name: Report_{{year}}{{month}}{{day}}
  - Poll Time: 60.0
  ↓
(waits for today's report table, e.g., Report_20240115)
  ↓
Query (load report data)
  ↓
Generate Report
  ↓
Send Email

Multiple Table Monitoring​

// Monitor for multiple specific tables
const tablesToMonitor = [
  'dbo.SalesData',
  'dbo.InventoryData',
  'dbo.CustomerData'
];

// Run Wait Table in parallel for each
// Use parallel flows or loop with threading

Wait with Timeout​

Start Timer
  ↓
Parallel:
  Branch 1: Wait Table (table pattern)
  Branch 2: Wait (timeout duration, e.g., 1 hour)
  ↓
First to complete triggers
  ↓
If Wait Table: Process new table
If Timeout: Handle timeout scenario

Cascade Wait Pattern​

Wait Table (for table creation)
  ↓
Wait Column (for specific column to be added to the table)
  ↓
Wait Row (for first data row)
  ↓
Process Complete Data

Monitor Schema Changes​

Loop (continuous monitoring):
  Wait Table (staging.Import%)
    ↓
  Query (get table details from INFORMATION_SCHEMA)
    ↓
  JavaScript (identify which table was created)
    ↓
  Process New Import Table
    ↓
  Non Query (DROP TABLE after processing)
    ↓
  (loop continues)

Pattern Matching Examples​

Standard Wildcards​

PatternMatchesDoesn't Match
Sales%Sales, SalesData, Sales2024DailySales
%ReportDailyReport, ReportReports, Report2024
%Archive%DataArchive, Archive_2024Archived
Temp_____Temp_12345 (5 chars)Temp_123 (3 chars)
Log[ABC]%LogA, LogB123LogD

Schema Patterns​

dbo.%           - All tables in dbo schema
staging.%       - All tables in staging schema
%.CustomerData  - CustomerData in any schema

Detecting Which Table Was Created​

After Wait Table triggers, you may want to know which table was created:

-- Query to get most recently created table matching pattern
SELECT TOP 1
  TABLE_SCHEMA,
  TABLE_NAME,
  CREATE_DATE = (
    SELECT create_date
    FROM sys.tables t
    INNER JOIN sys.schemas s ON t.schema_id = s.schema_id
    WHERE s.name = TABLES.TABLE_SCHEMA
      AND t.name = TABLES.TABLE_NAME
  )
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '{{pattern}}'
ORDER BY CREATE_DATE DESC

Common Errors and Solutions​

Table Name Pattern Cannot Be Empty​

Error: ErrInvalidArg: Table name pattern cannot be empty

Solutions:

  • Provide a table name or pattern in the Table Name field
  • Ensure the variable containing the pattern has a value

Connection Closed During Monitoring​

Error: Connection closed or lost during monitoring

Solutions:

  • Use credentials mode to create a dedicated connection
  • Ensure connection timeout is sufficient for monitoring duration
  • Check network stability
  • Implement reconnection logic

False Triggers​

Problem: Node triggers for unintended tables

Solutions:

  • Make pattern more specific
  • Include schema name in pattern
  • Use exact table names instead of wildcards
  • Filter by naming conventions

Never Triggers​

Problem: Node waits indefinitely

Solutions:

  • Verify pattern is correct
  • Check if table is actually being created
  • Query INFORMATION_SCHEMA manually to verify pattern
  • Implement timeout logic
  • Check permissions on INFORMATION_SCHEMA

Performance Considerations​

Database Load​

  • Each poll executes a query
  • Faster polling = more queries
  • Consider database load when setting poll time
  • Use appropriate poll interval for your scenario

Long-Running Monitoring​

  • For very long monitoring, use credentials mode
  • Monitor connection health
  • Consider implementing heartbeat checks
  • Plan for connection timeouts

Multiple Monitors​

  • Running multiple Wait Table nodes increases load
  • Consider consolidating patterns if possible
  • Use appropriate poll times
  • Monitor database performance

Integration Patterns​

Trigger-Based Automation​

Wait Table (external system creates table)
  ↓
Query (load new data)
  ↓
Transform Data
  ↓
Insert (into target table)
  ↓
Non Query (DROP source table)
  ↓
Send Notification

Scheduled with Wait​

Scheduler (runs every hour)
  ↓
Wait Table (wait for specific table, max 55 minutes)
  ↓
If table found: Process
If timeout: Log and exit

Multi-Stage Pipeline​

System A: Creates table Import_Stage1
  ↓
Wait Table (Import_Stage1)
  ↓
Process and create Import_Stage2
  ↓
Wait Table (Import_Stage2)
  ↓
Final Processing

Troubleshooting​

Debug Pattern Matching​

-- Test your pattern in a Query node first
SELECT TABLE_SCHEMA, TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '{{yourPattern}}'
  AND TABLE_SCHEMA = '{{yourSchema}}'
ORDER BY TABLE_NAME

Monitor Polling Activity​

// Add logging between polls
let pollCount = 0;
const startTime = new Date();

// This would be internal to the node
// Log externally by running queries periodically
console.log(`Poll ${pollCount}: No new table yet`);
console.log(`Elapsed time: ${new Date() - startTime}ms`);

Verify Permissions​

-- Check permissions on INFORMATION_SCHEMA
SELECT * FROM INFORMATION_SCHEMA.TABLES
WHERE 1 = 0
-- If this fails, you don't have permission

See Also​

  • Wait Column - Wait for new columns in tables
  • Wait Row - Wait for new rows in tables
  • Connect - Establish database connections
  • Query - Execute SELECT queries