Amazon SQS

Amazon SQS (opens new window) is a web service offered by Amazon Web Services that provides fully managed message queues for microservices, distributed systems, and serverless applications.

API version

The Amazon SQS connector uses the Amazon SQS REST API (opens new window).

How to connect to SQS on Workato

The SQS connector allows two types of authentication:

• IAM role
• Access key

Workato recommends using IAM role authentication.

Input fields

FIll out the following input fields:

Connection name

Give this connection a unique name that identifies which SQS instance it is connected to.

Authorization type

IAM role or Access key.

Access key ID (Access key only)

The user's ID.

Secret access key (Access key only)

The user's secret.

IAM role ARN (IAM role only)

The IAM role ARN. Learn how to retrieve the IAM role ARN. Workato generates a unique external ID (for example, workato-user-84762). This value is different for every Workato user and must be provided when creating an IAM role in SQS.

Region

Provide the region for this SQS account.

Version

The SQS version. Version 2012-11-05 is available.

Service

Your SQS queue service, defined in the queue ARN. If your queue ARN is arn:aws:sqs-fips:us-east-1:123456789:bennett_standard_q, use sqs-fips as the service.

Create IAM role and ARN retrieval

Refer to the IAM role-based authentication for AWS page for instructions on how to create an IAM role for Workato and retrieve your Amazon resource name (ARN).

Permissions

You may use the AmazonSQSFullAccess policy for this IAM role. At a minimum, the SQS connector requires these permissions:

• sqs:ReceiveMessage
• sqs:DeleteMessage
• sqs:ListQueues
• sqs:GetQueueAttributes
• sqs:SendMessage

Workato uses list queues to test and establish the connection. Suppose you only have access to a single queue or a few specific queues. In that case, the connection still works and you may perform operations on those queues by manually adding the queue name in the actions.

Terminology

The terminology (opens new window) for basic Amazon SQS objects follows:

Queues

There are two types of queues in SQS:

• Standard Queues - The default queue type which supports at-least-once message delivery. Standard queues provide best-effort ordering which ensures that messages are generally delivered in the same order they're sent.
• FIFO Queues - FIFO (First-In-First-Out) queues enhance messaging between applications when the order of operations and events is critical, or duplicates can't be tolerated.

The Workato SQS connector only supports standard queues.

Messages

Amazon SQS lets you include structured metadata, including timestamps, geospatial data, signatures, and identifiers with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body but are sent alongside it. Workato supports both message bodies and message attributes.

When a consumer receives and processes a message from a queue, the message remains in the queue. Amazon SQS doesn't automatically delete the message. Because Amazon SQS is a distributed system, there's no guarantee that the consumer receives the message (for example, due to a connectivity issue or to an issue in the consumer application). the Workato SQS connector allows you to delete messages automatically when you retrieve them in triggers. Alternatively, you may use the Delete messages action to delete messages after you have processed them in your recipe.

Short and long polling

Due to SQS's distributed nature, a single request to SQS may not retrieve all messages in the queue. For example, a request to retrieve 10 messages may only retrieve 9 messages when there could be hundreds of messages in the queue. This is the result of SQS sampling a subset of servers holding messages to return them.

To mitigate this, Workato uses long polling with a wait timeout of 5 seconds. When Workato attempts to retrieve messages in triggers or actions, Workato waits for up to 5 seconds for messages to arrive at sampled servers. Configure this timeout for actions using the input Wait time (seconds).

Visibility timeout

Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, during which Amazon SQS prevents other consumers from receiving and processing the message.

The Workato connector has a visibility timeout default of 12 hours instead of SQS's 30-second default. This minimizes the possibility of jobs having the same messages. You may configure this timeout using the Visibility Timeout input field.

Dead letter queues

Messages in an SQS queue must be deleted to be considered processed. If messages are left in a queue for a set amount of time (configurable in SQS) or retrieved too many times, they are then stored in a dead-letter queue for debugging.

Workato does not actively support messages stored in dead-letter queues but recommends you set alarms to notify you of such messages stored there. Learn more. (opens new window)

Troubleshooting

Duplicate messages across jobs

In some cases, jobs process too slowly and messages retrieved by Workato have passed their visibility timeout and are eligible to be retrieved again. In such cases, you may see duplicate messages across different jobs.

To prevent this, increase the visibility timeout or delete messages as they're retrieved.