What is an SNS Fan-Out pattern?
Fan-out is a message pattern where messages are broadcast in a one to many arrangement.
It is particularly useful when we need to design solutions where we may need to scale the number of subscribers of a message as required.
For example, let's say you're building an image processing service where all the requests for images are placed on a queue and are picked up by the handlers to process and store the image attributes to a database.
But if we need to scale this functionality, we need to add more queues to the solution since a single queue may not be sufficient to handle.
But adding more and more queues and then sending messages to multiple queues can be inefficient, data loss, perfomance.
In this case, we can implement a Fan-Out pattern.
For example, we can add an SNS topic that writes to multiple SQS queues and handlers process from these queues.
The producer service publishes the image data to the SNS topic instead of having to individually place the messages into SQS queues.
This decouples the SQS queues from the producer service, and we can add as many SQS queues as needed without having to worry about performance, data loss etc.
Some advantages / features of an SNS Fan Out Pattern
✅ This approach is fully decoupled, you don't need to worry about data loss between the processing
✅ SQS queues offer data persistence, delayed processing and retries
✅ We can also have the ability to add more SQS queues as subscribers over time
✅ We need to add an SQS queue access policy to allow SNS to write to the queue
✅ Cross Delivery - this pattern works with SQS queues in other regions
AWS Documentation — https://docs.aws.amazon.com/sns/latest/dg/sns-common-scenarios.html
