Best Practices for Python with AWS Lambda: An Essential Guide

In today’s tech-driven world, developers leverage Python with AWS Lambda for its simplicity, efficiency, and effectiveness. This article delves into the best practices you can adopt to make the most out of Python with AWS Lambda.

AWS and Deployment

AWS Layer for Python Dependencies

AWS Lambda Layers is a mechanism for distributing libraries, custom runtimes, and other function dependencies. Utilizing layers allows you to manage your dependencies separately from your primary application, simplifying the development process and hastening deployments. Benefits include separation of responsibilities, code reuse, faster deployments, and simplified dependency management.

Upload S3 Source Code Only

When creating a deployment package for AWS Lambda, you typically include your function’s source code and all its dependencies. However, I’d recommend separating the dependencies from your source code and using AWS Lambda layers.

Separating your source code from its dependencies and uploading only the source code to S3 has several benefits:

1. Separate Updates: You can update your function code or its dependencies independently without needing to redeploy everything together.
2. More straightforward Deployment Package: Your deployment package becomes significantly smaller as it only contains your function’s source code.
3. Faster Deployment: Uploading just your source code to S3 and letting Lambda merge it with the dependencies can be faster than creating and uploading a complete deployment package, especially for larger projects.
4. Increased Visibility and Modifiability: You can easily view, modify, and test your source code directly in the AWS Lambda console UI, advancing the development speed.

Keep a structured and consistent versioning practice for both your source code and layers to ensure proper tracking and compatibility between different versions of your application and its dependencies.

AWS Serverless Application Model (SAM) for Exploring AWS Resources

The AWS Serverless Application Model (SAM) is an open-source framework for building serverless applications on AWS. SAM provides shorthand syntax to express functions, APIs, databases, and event source mappings, simplifying how you understand your AWS resources.

Testing

Testing with Moto and Patch Decorator

Testing is a non-negotiable aspect of software development, and it becomes more challenging when dealing with AWS services via the boto3 library in Python Fear not, because the moto library and Python's built-in unittest.mock come to the rescue.

Moto is a fantastic tool that allows you to mock AWS services, letting you write tests without making actual AWS requests. It means you can test your functions in isolation, verifying that they behave as expected under different conditions.

Here is a simple example of using moto to test a function that retrieves a file from an S3 bucket:

import boto3
from moto import mock_s3


@mock_s3
def test_get_s3_file():
    # Set up the mocked S3 resource
    s3 = boto3.resource('s3')
    s3.create_bucket(Bucket='my_bucket')
    s3.Object('my_bucket', 'my_key').put(Body='my_content')
    
    # Your function to test
    def get_s3_file(bucket, key):
        content = s3.Object(bucket, key).get()['Body'].read().decode()
        return content
    content = get_s3_file('my_bucket', 'my_key')
    assert content == 'my_content'

On the other hand, when you need to mock a specific functionality within your code, you can leverage the patch decorator from unittest.mock. The patch() the function enables you to replace parts of your system under test and assert how they have been used.

Let’s say you have a function that calls a function foo and you want to mock foo during your tests:

from unittest.mock import patch


def bar():
    return foo()
@patch('__main__.foo', return_value=42)
def test_bar(mocked_foo):
    assert bar() == 42

In this example, the foo function is patched to return 42 when it's called, allowing you to isolate and test the bar function's behavior. Code and Style Guidelines

Package Management: Prefer pip

While the Python community offers various tools for package management, pip remains the go-to tool for many developers due to its simplicity and compatibility. However, Poetry might be the more suitable choice for larger, more complex projects. Regardless, it’s advised to separate production and testing dependencies for efficient AWS Lambda layer creation.

Avoid Third-Party Libraries

While third-party libraries provide advanced functionality, they introduce added complexity, potential security risks, and increased dependencies. Leveraging Python’s built-in libraries can mitigate these risks and enhance your code’s reliability and efficiency.

File Imports and Code Structuring

AWS Lambda’s execution environment expects the source code to be structured in a certain way, affecting how you structure your imports. The solution is to place your source code in a directory and use relative imports like from .module import function.

You might be tempted to use absolute imports such as from src.module import function, or parent directory imports like from ..module import function. Both approaches may work locally, but they can lead to failures when executing tests via python -m unittest. Python has trouble finding the src module or importing from a parent directory unless it's marked as a package (with an __init__.py file).

Instead, using explicit relative imports like from .module import function is a more reliable approach. However, be aware that this too can cause an error when you run your Lambda function from the AWS console, due to how AWS executes your function from the deployment package's root directory.

To overcome this, consider structuring your deployment package in a specific way:

1. Place your source code in a directory, say src/. You can use relative imports like from .module import function within the src/ directory.
2. Zip the src/ directory and upload it to S3. Your deployment package structure should look like this: zip/src/your_source_code_files.
3. When creating your Lambda function, set the handler to src.your_handler_file.your_handler_function.

This structure allows you to use relative imports in your code and enables AWS Lambda to find the correct module when executing your function.

Follow PEP 8 for Formatting and Naming Conventions

Following PEP 8, the official style guide for Python ensures your code is clean, understandable, and maintainable. Consistency in coding style includes correct indentation, line length, blank lines, import statements, whitespace, naming conventions, and variable assignment.

In summary, these guidelines are not rigid rules but suggestions to keep your Python code with AWS Lambda clean, efficient, and maintainable. By understanding and implementing these best practices, you can streamline your codebase and make it more effective and productive.

More content at PlainEnglish.io.

Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.