Importance of Asynchronous Invocation for Lambda Destinations
This invocation allows us to speed up the process of events if there is no urgency for results. With Asynchronous Invocation, all the queued events are handed off to Lambda. Further, Lambda is configured to handle errors, send invocation records (events) to the set targets, etc.
The problem while doing asynchronous invocation was that it was hard to see if the event succeeded or failed. And if it did, then how can we retrieve it? Initially, DLQ (Dead Letter Queue) was the only way possible for asynchronous invocations events, but it has certain limitations as it has limited targets like SQS and SNS. Secondly, it only allows for sending failure events to the target and does not allow for sending success events. To overcome these issues, AWS developed a Lambda Destinations feature.
What are Lambda Destinations?
Lambda Destinations is a feature that helps to send the results of asynchronous invocation to multiple targets. Unlike DLQ, in Lambda Destinations, it is possible to define destinations for processing successful and failed events. As mentioned, this feature supports multiple targets like SQS, SNS, Lambda, and Eventbridge Bus.
The idea behind Lambda Destination is that the Lambda function is invoked asynchronously by the S3 event. If successful, the records are sent to successful event destinations like SQS. And if it’s a failure, they are sent to failed event destinations.
Note: In this example, S3 has been added as an Event source that will trigger Lambda Asynchronously. The Lambda Destination feature sends the Successful/Failure events to Amazon SQS.
Working Architecture
Steps to Configure
Note: – Create one S3 bucket with the name of your choice and 2 Standard SQS Queues with the names lambda-success and lambda-failure.
1. Create a Lambda Function with the runtime of your choice.
2. Once done, add the S3 bucket, created initially as a trigger to the Lambda function.
Note: – It is necessary to add S3 as a trigger because it will send the events like when an object is uploaded or deleted from the bucket, and accordingly Lambda function will be invoked.
3. To add Destinations to the lambda function, click + Add Destination.
4. In the Destination Configuration page, select Asynchronous Invocation from Source.
5. First, select On Failure from Conditions.
6. Select SQS Queue from Destination Type and select the lambda-failure SQS queue, which was created initially.
7. Click Save once the configuration is done.
Note: – In the background, AWS will add permissions to the Lambda function, which will help us to send results to destinations.
8. Follow the same procedure for On Success events as well. Select On Success from Conditions.
9. Select SQS Queue from Destination Type and select the lambda-success SQS queue, which was created initially.
10. Confirm the Destination once they are added from the Configuration.
Working
1. For event to be sent to a successful target (lambda-success SQS queue), keep Lambda Code as follows
2. Upload an image to S3 bucket which was created initially.
3. Once uploaded, Lambda function will get invoked as S3 is configured as a trigger.
4. The image details and print statement from code will be sent to lambda-success SQS queue.
5. From AWS console open SQS and confirm the message count from the Messages available. From the image below, it is confirmed that the message was sent to the Success.
6. Open the queue and poll for the message. The message sent can be seen in the image below.
7.Similarly, for failed events, keep lambda code as follows, which will raise the exception and send the data to failure queue.
8. Again, upload an image to S3 bucket which was created initially.
9. This time, the image details and print statement from code will be sent to lambda-failure SQS queue as we have raised the exception in the lambda code.
10. Switch to the SQS tab and confirm the message count from the available Messages. From the image below, it is confirmed that the message was sent to the Failure.
Conclusion
Lambda Destination helps to send successful and failed events easily to destinations. It helps debug easily in case of data failure or to check whether the data has been correctly sent. It supports multiple destination types and helps to retrieve detailed data.
Happy Reading 😊
Chaitanya Karadkhedkar
