Importance of Asynchronous Invocation for Lambda Destinations
This invocation allows us to speed up the process of events if there is no urgency for results. With Asynchronous Invocation, all the events which are queued are handed off to Lambda. Further, the Lambda is configured in such a way that it handles errors, sends invocation records (events) to the set targets, etc.
The problem while doing asynchronous invocation is that it was really hard to see if the event had succeeded or failed. And if it did, then how can we retrieve it? Initially, for asynchronous invocations events, DLQ (Dead Letter Queue) was the only way possible, but it has certain limitations as it has limited targets like SQS and SNS. Secondly, it only allows for sending failure events to the target and does not allow for sending success events. To overcome these issues, AWS came up with a feature called Lambda Destinations.
What are Lambda Destinations?
Lambda Destinations is a feature that helps to send the results of asynchronous invocation to multiple targets. Unlike DLQ, in Lambda Destinations, it is possible to define destinations for processing of successful and failed events. As mentioned, this feature supports multiple targets like SQS, SNS, Lambda, and Eventbridge Bus.
The idea behind Lambda Destination is that the Lambda function is invoked asynchronously by the S3 event. If it’s successful, then the records are sent to successful event destination like SQS. And if it’s a failure, they are sent to failed event destinations.
Note: – Here in this example, S3 has been added as an Event source which will trigger Lambda Asynchronously, and the Lambda Destination feature is being used to send the Successful/Failure events to Amazon SQS.
Working Architectur
Steps to Configure
Note: – Create one S3 bucket with the name of your choice and 2 Standard SQS Queues with names lambda-success and lambda-failure.
1. Create a Lambda Function with runtime of your choice.
2. Once done, add the S3 bucket which was created initially as a trigger to the Lambda function.
Note: – It is necessary to add S3 as trigger because it will send the events like when an object is uploaded or deleted from the bucket and accordingly Lambda function will be invoked.
3. To add Destinations to the lambda function, click on + Add Destination.
4. In Destination Configuration page, select Asynchronous Invocation from Source.
5. First select On Failure from Conditions.
6. Select SQS Queue from Destination Type and select the lambda-failure SQS queue which was created initially.
7. Click Save once configuration is done.
Note: – In the background AWS will add permissions to the Lambda function which will help us to send results to destinations.
8. Follow the same procedure for On Success events as well. Select On Success from Conditions.
9. Select SQS Queue from Destination Type and select the lambda-success SQS queue which was created initially.
10. Confirm the Destination once they are added from Configuration
Working
1. For event to be sent to successful target (lambda-success SQS queue) keep Lambda Code as follows
2. Upload an image to S3 bucket which was created initially.
3. Once uploaded, Lambda function will get invoked as S3 is configured as trigger.
4. The image details and print statement from code will be sent to lambda-success SQS queue.
5. From AWS console open SQS and confirm the message count from Messages available. From the image below it is confirmed that the message was sent to the Success
6. Open the queue and poll for the message. The message sent can be seen in the image below.
7. Similarly, for failed event keep lambda code as follows which will raise the exception and send the data in failure queue.
8. Again, upload an image to S3 bucket which was created initially.
9. This time, the image details and print statement from code will be sent to lambda-failure SQS queue as we have raised the exception in the lambda code.
10. Switch to SQS tab and confirm the message count from Messages available. From the image below it is confirmed that the message was sent to the Failure
Conclusion
Lambda Destination helps to send successful and failure events easily to destinations. It helps to debug easily in case of data failure or to check whether the data has been correctly sent or not. It supports multiple destination types and helps to retrieve detailed data.
Happy Reading 😊
Chaitanya Karadkhedkar
