Key Considerations for AWS Lambda on production
AWS Lambda is an indispensable component of modern cloud architectures. Its serverless nature, low infrastructure requirements, and pay-as-you-go pricing have made it a cornerstone of production workloads. However, while the benefits of AWS Lambda are undeniable, it is crucial to be aware of certain considerations to ensure its successful implementation.
Monolithic or Individual Lambda
The first choice we often face is how much code should be in one lambda. AWS best practice is against having a monolith lambda See here but if we separate each functionality to separate lambda wouldn't we end up in an explosion of lambdas? From a pricing perspective, it might not be a problem but what about code organization? do developers jump around in repos for relevant code? how do we handle code duplication?
One approach I find useful for me is to have a single repo for related functions but deploy them as separate lambda. In this way, we can keep the common code together and reuse it in all the functions.
Pros -
- From a readability perspective it will feel like a single application.
- It will still be deployed as separate functions so we are not introducing monolithic lambda.
- We can keep the common code together hence less duplicity.
Cons -
- Complex CI/CD, we need to understand which part of the code has changed and act accordingly.
More fragmented infra
Continuing from above, if we break down a service into individual lambdas we need to take care of automated deployment for each lambda.
A decision we need to take here is whether we have one infra code or separate infra code for each lambda. It will be beneficial to separate the infra code for each lambda so that we are not unnecessarily building or updating all lambda.
One problem that often arises here is about which lambda owns a particular infra. We can use tagging to assign ownership.
But overall we are managing from one service to multiple functions so it is a challenge nonetheless.
Cognitive load
Because it is so easy to integrate lambdas with other services we often feel like adding Lego-like blocks and before we realize we end up with several different components that work with each other.
As developers, what we need here is an overall visibility of what event is generated by which service consumed by which lambda which in turn triggers other service that soon becomes a cognitive load.
Create an overall architecture diagram updated and handy we will often need that.
I would personally love to have a tool that can visualize the whole topology.
Testing
AWS Lambda is an AWS offering and it often integrated with other services so a question arises how to test that. There is a whole section in AWS doc about testing. However, one thing that I differ slightly from here is the usage of a simulator. I like usage simulators like localStack. This gives me much higher confidence than mocking out relevant parts. The doc talks about testing by actually deploying on AWS, which is fine but if devs can test locally that will be much faster. Localstack alongside testcontainers can speed up the process.
Tracing
Most of the time Lambdas are part of an overall architecture and not in a standalone way. This means we need tracing, logging, and metrics to work seamlessly when service-to-lambda interactions take place.
for tracing While X-ray is a great tool often organizations do have an existing tracing mechanism or the cost of X-ray becomes an issue.
We need to come up with a unified tracing solution. One option is to depend on a standard library like opentelemetery to programatically propagates tracing info.
Another option is to run the same logic but using AWS lambda Extension API.
In the second, we can make the tracing part language agnostic and run as a separate process from lambda.
Beware about quota
AWS Lambda by default has a quota of a maximum of 1000 concurrent executions in an account.
Initially, this might not be an issue. But as the environment grows we will start to see throttles.
Beware of any scenario that can cause unbounded execution of lambda that might eat up quota from other lambdas.
If the event producer produces a large number of events considering having an upper bound of how many concurrent lambdas can be spawned at max. One example configuration for SQS mapping can be found [here](https://aws.amazon.com/blogs/compute/introducing-maximum-concurrency-of-aws-lambda-functions-when-using-amazon-sqs-as-an-event-source/)
In conclusion, AWS Lambda is a powerful and valuable tool for building scalable and cost-effective cloud architectures. However, its adoption requires careful consideration of various factors. With thoughtful planning and the adoption of best practices, AWS Lambda can significantly enhance scalability, reduce TCO, and drive successful deployments in production environments.