# SQS Poller Design

The SQS poller is designed to be a resilient concurrent poller that pulls messages
out of SQS to be processed by consumer code. It indends to poll the queue 
sufficiently fast to saturate your worker, but uses backpressure to help prevent
over-committing to work.

## Goroutines

This design compreses several groups of goroutines each doing independent work:

1. **Pollers** (1 or more) run an infinite loop of polling SQS and forwarding 
   messages to the messages channel.
2. **Acking** Batch finished message IDs into DeleteMessageBatch calls.
3. (coming in EDGEPLAT-1900) **Stats Hub** (always 1 goroutine) gets stats from the pollers about how 
   long was spent polling and how many messages were received and decides 
   whether to add or remove pollers.

## Scaling

(Coming in EDGEPLAT-1900)

For each loop of the poller, it sends stats to the stats hub about the 
following things:

1. How long spent waiting for ReceiveMessage
2. How many messages were received
3. How long spent enqueueing these messages (hint about blocking)
4. If we got any rate-limit errors from AWS.

With this information the stats hub looks at whether the ReceiveMessage
returned early, or waited the entire duration. Furthermore, it checks
primarily on whether it returned 0, 1, or more than 1 message.

If we got more than 1 message or especially the maximum 10 messages, then
it is a signal that we should be adding more poller goroutines.

However, to prevent over-fetch scenarios where we pull a lot of messages off 
the queue and don't have enough CPU to process them, we monitor time spent
blocking as well and will not scale up if we're blocking a lot on enqueueing.

## Acking

An important part of working with SQS is making sure we delete messages once
processing work is complete. If we get a non-error response from the 
dispatcher, we will delete that messages' ID off the queue so it does not 
get re-delivered again after the delivery window.

SQS provides a batch delete operation, so we will optimistically delete 
things in batches, but also have a min frequency of deletions set so that
in general, deletions still happen within the receive window.

## Shutdown

We try to take the most safe route to shutdown, even though it may take longer.
Essentially, we're trying to shut down with no messages that have been pulled
from SQS without being processed. An ideal shutdown sequence looks like this:

1. Stop all pollers to stop filling channels with messages.
2. (client-determined) Wait for all in-flight calls to finish
3. Perform left-over DeleteMessage if any.

However, we know this could take a while so we offer one way to speed it up: 
if you call `CancelOutgoingRequests` it will cancel all their request contexts 
and therefore allow you to shutdown faster.

If you do this un-graceful shutdown, then it is possible that some messages 
are going to be re-delivered after the VisibilityTimeout to another poller. 
This is probably not a big deal for most use cases, but it does underscore
the need for idempotency.

An implementation of semi-graceful shutdown with a lifecycle timeout can be seen in
full_example_test.go