Triggers & Integrations
Post-Execution Triggers allow you to push your scraped data to external systems automatically. This category covers integrations with PostgreSQL, MySQL, MongoDB, AWS S3, Google Cloud, Kafka, RabbitMQ, Email (SMTP), and REST/GraphQL Webhooks.
When asking for help here, please provide:
- Trigger Type — e.g.,
aws_s3,postgresql,rest_api. - Variable Mapping — How are you passing data? (e.g., “I am trying to pass
{dataset_name}into the JSON body of my webhook.”) - The Error Log — The exact error from the Execution Logs (e.g.,
SMTP Error: Authentication failedorPostgreSQL Error: relation does not exist).
Supported Integrations at a Glance:
| Category | Services |
|---|---|
| Cloud Storage | AWS S3, Google Cloud Storage, Azure Blob |
| Databases | PostgreSQL, MySQL, MongoDB |
| Message Queues | RabbitMQ, Kafka, AWS SQS |
| File Transfer | SFTP, FTPS, WebDAV |
| Spreadsheets | Google Sheets, Local Excel (.xlsx) |
| SMTP with optional attachments | |
| File Formats | JSON → CSV, Parquet, Excel, ZIP/GZIP |
| APIs | REST, GraphQL, SOAP |
Security Note: NEVER paste your actual passwords, API keys, or connection strings in the forum. Replace them with
[REDACTED]before posting.