Skip to main content

Download Object

Downloads an object from an S3 bucket to a local file on your system.

Common Properties​

  • Name - The custom name of the node.
  • Color - The custom color of the node.
  • Delay Before (sec) - Waits in seconds before executing the node.
  • Delay After (sec) - Waits in seconds after executing node.
  • Continue On Error - Automation will continue regardless of any error. The default value is false.
info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs​

  • Client Id - The client connection ID from the Connect node. Optional if using credentials directly.
  • Bucket Name - The name of the S3 bucket containing the object.
  • Object Name - The key/name of the object to download, including any prefixes (folder path).
  • File Path - The local file system path where the downloaded file will be saved. Include the filename.

Options​

  • End Point - S3 endpoint URL. Required only if using credentials directly instead of Client ID.
  • Access Key Id - AWS Access Key ID credential. Optional - use this instead of Client ID for direct authentication.
  • Secret Key Access - AWS Secret Access Key credential. Optional - use this instead of Client ID for direct authentication.

How It Works​

The Download Object node retrieves a file from S3 and saves it to your local file system. When executed, the node:

  1. Retrieves the S3 client using either the Client ID or creates a new client from credentials
  2. Validates that the bucket name, object name, and file path are provided and not empty
  3. Sends a request to S3 to download the specified object
  4. Saves the downloaded content to the specified local file path
  5. Completes successfully once the file is saved

Requirements​

  • Either a valid Client ID from a Connect node, or Access Key ID and Secret Access Key credentials
  • A valid S3 bucket with the object to download
  • Appropriate S3 permissions to read objects (s3:GetObject)
  • Write permissions on the local file system path
  • Sufficient disk space for the downloaded file

Error Handling​

The node will return specific errors in the following cases:

  • Empty or invalid bucket name
  • Empty or invalid object name
  • Empty or invalid file path
  • Object does not exist in the bucket
  • Invalid Client ID or credentials
  • Bucket does not exist
  • Insufficient permissions to download objects
  • Local file path is not writable
  • Insufficient disk space
  • Network or connection errors

Usage Notes​

  • The file is downloaded to the local file system where the robot is running
  • If the target file already exists, it will be overwritten without warning
  • The parent directory must exist before downloading
  • File paths should use the correct separator for your operating system
  • Large files are automatically handled with efficient streaming
  • The download is atomic - the file appears only after the complete download
  • Object metadata is not downloaded, only the file content

Best Practices​

  • Create the target directory before downloading if it doesn't exist
  • Use absolute file paths for reliability
  • Check available disk space before downloading large files
  • Implement retry logic for large file downloads
  • Validate the downloaded file after completion (size, checksum)
  • Clean up downloaded files after processing to save disk space
  • Use unique filenames to avoid overwriting existing files
  • Handle errors gracefully with appropriate logging

Example​

To download a PDF invoice from S3:

Inputs:

  • Client Id: (from Connect node)
  • Bucket Name: company-invoices
  • Object Name: 2024/march/invoice-2024-001.pdf
  • File Path: /home/user/downloads/invoice-2024-001.pdf

Result: The object 2024/march/invoice-2024-001.pdf will be downloaded from S3 and saved to /home/user/downloads/invoice-2024-001.pdf.

Download with Dynamic Filename​

To download files with timestamps:

Inputs:

  • Client Id: (from Connect node)
  • Bucket Name: daily-reports
  • Object Name: sales/report.csv
  • File Path: /reports/sales-report-2024-03-15.csv (constructed dynamically)

Use JavaScript to build the file path:

const today = new Date().toISOString().split('T')[0];
const filePath = `/reports/sales-report-${today}.csv`;

Batch Download Example​

To download multiple files, use a Loop node:

  1. List Objects - Get all objects in a bucket
  2. Loop - For each object:
    • Download Object
    • Object Name: (from loop item)
    • File Path: (construct local path)

Example:

// In the loop, construct the local path
const localPath = `/downloads/${objectName.replace(/\//g, '-')}`;

Direct Credentials Example​

Inputs:

  • Bucket Name: my-backups
  • Object Name: daily-backup-2024-03-15.zip
  • File Path: /restore/backup.zip

Options:

  • End Point: s3.us-east-1.amazonaws.com
  • Access Key Id: (your AWS Access Key ID credential)
  • Secret Key Access: (your AWS Secret Access Key credential)

Common Use Cases​

Backup Restoration Download backup files from S3 for disaster recovery or data restoration.

Document Retrieval Download invoices, contracts, or reports for processing or archival.

Media Download Download images, videos, or audio files for local processing.

Data Import Download CSV or Excel files for data analysis or database import.

Log Analysis Download application logs from S3 for debugging or compliance review.

Batch Processing Download files for batch processing tasks like image conversion or data transformation.

Download to Temporary Directory​

For temporary processing:

File Path: /tmp/downloaded-file.pdf

Process the file and delete it afterward to save space.

Handling Folder Structures​

When downloading objects with slashes in the name:

Object Name: documents/2024/Q1/report.pdf File Path: /local/documents/2024/Q1/report.pdf

Ensure the directory /local/documents/2024/Q1/ exists before downloading.

Creating Directories​

Before downloading, create the directory structure:

  1. Extract Directory from file path using JavaScript
  2. Create Directory using a File System node or Bash command
  3. Download Object to the file path

Example:

const filePath = '/downloads/reports/2024/report.pdf';
const directory = filePath.substring(0, filePath.lastIndexOf('/'));
// Create directory if it doesn't exist

Common Errors​

Error: "NoSuchKey: The specified key does not exist"

  • Solution: Verify the object name is correct, including the full path with prefixes

Error: "NoSuchBucket: The specified bucket does not exist"

  • Solution: Verify the bucket name is correct and the bucket exists

Error: "Access Denied"

  • Solution: Ensure your credentials have the s3:GetObject permission

Error: "Failed to download object: no such file or directory"

  • Solution: Create the parent directory before attempting to download

Error: "Permission denied"

  • Solution: Ensure the robot has write permissions to the target directory

Performance Tips​

  • Download files during off-peak hours for large transfers
  • Use regional endpoints matching your bucket region for faster downloads
  • Consider compression if downloading many small files
  • Implement parallel downloads for multiple files (use multiple Download Object nodes)
  • Monitor network bandwidth to avoid overwhelming your connection