How to Handle Lambda Dependencies with Ease

As a Cloud Engineer or DevOps engineer, working with AWS Lambda is a common task in our day-to-day activities. Sometimes, adding third-party libraries or dependencies to Lambda functions can be a bit tedious, especially when we have to manage layers manually. In this guide, I’ll walk you through a simple way to download Lambda dependencies with just your AWS account and the necessary IAM permissions.

This guide assumes you're using your personal AWS account. If you're using an IAM user, make sure you have the necessary permissions to interact with AWS Lambda and S3 services

Step-by-Step Guide to Download AWS Lambda Dependencies

Start by logging into your AWS Management Console. Once logged in, navigate to the Lambda service.

2. Create a Lambda Function

Once you're in the AWS Lambda section, follow these steps:

Click Create function.
Choose Author from scratch.
For simplicity, you can start with a basic "Hello World" example.
Name your function as lambda-dependencies-fetcher.
Select the Python 3.11 runtime.
Click Create function.

Once your function is created, navigate to the Code section of your Lambda function, and paste the following code:

import json
import requests

def lambda_handler(event, context):
    try:
        # Simple GET request to check connectivity to Google (Google's public API)
        response = requests.get('https://www.googleapis.com/discovery/v1/apis')

        if response.status_code == 200:
            return {
                'statusCode': 200,
                'body': json.dumps('Successfully connected to Google using downloaded dependencies!')
            }
        else:
            return {
                'statusCode': response.status_code,
                'body': json.dumps(f'Failed to connect to Google. Status code: {response.status_code}')
            }

    except requests.exceptions.RequestException as e:
        return {
            'statusCode': 500,
            'body': json.dumps(f'Error connecting to Google: {str(e)}')
        }

3. Test Your Lambda Function

Now, head over to the Test section of your Lambda function.

Name your test (e.g., "dev").
Leave the payload as {} (empty braces).
Click Test.

You’ll most likely encounter an ImportError because the required requests library is not bundled with your Lambda by default. Let’s fix that!

Oh, we encountered the import error we were expecting. Let's resolve it together

4. Open AWS CloudShell

To resolve this, open AWS CloudShell.

Check your current directory using the command:

pwd

Create a directory for your Lambda dependencies:

mkdir lambdas

Verify if the directory was created:

ls

If successful, move into the lambdas directory:

cd lambdas

Now, create a file named lambda_function.py:

vi lambda_function.py

Enter insert mode by pressing i on your keyboard, and paste your Lambda code into the file. After pasting, press ESC, then type :wq to save and exit.

5. Install Lambda Dependencies

To install the required dependencies like requests, run the following command:

pip install requests -t .

If you need to install multiple dependencies, for example:

pip install boto3 json datetime pymysql opensearchpy OpenSearch -t .

Alternatively, you can create a requirements.txt file that contains all the dependency names along with their versions.

Once dependencies are installed, verify the installation:

ls -a

6. Package the Lambda Function

Now that we have our code and dependencies, let’s create a ZIP file:

zip -r ../lambda_package.zip .

Verify the contents of the previous directory:

cd ..
ls -a

You should see the lambda_package.zip file.

However, the ZIP file might not have executable permissions. To fix this, use the following command:

chmod +x lambda_package.zip

If it worked, the file name will turn green in the terminal.

7. Create an S3 Bucket (If Not Already Created)

If you don't have an S3 bucket yet, create one using the following command:

aws s3 mb s3://aws-lambdas-source-bucket-1

Replace aws-lambdas-source-bucket-1 with your bucket name.

8. Upload the Lambda ZIP Package to S3

Upload your ZIP package to your S3 bucket:

aws s3 cp lambda_package.zip s3://aws-lambdas-source-bucket-1

9. Update Lambda to Use the ZIP File from S3

Now, go to your Lambda function in the AWS Console. Click on Upload and select the Amazon S3 option. Browse to your S3 bucket, and copy the URL.

Paste this URL into your Lambda function's configuration and deploy the changes.

The dependencies have been added to the code. Now, in the Lambda code section, you'll see that the dependencies are included through the ZIP file.

11. Test the Lambda Function

Finally, go back to the Test section of your Lambda. Run the test, and you should see a 200 OK response. The Lambda function is now successfully using the dependencies you downloaded.

Conclusion

With just an AWS account and the necessary IAM permissions, you can easily download and manage Lambda dependencies without the need for third-party services. This method gives you more control over your Lambda functions, and you can ensure that all required dependencies are packaged and uploaded directly from AWS services.

Congratulations, you've successfully installed the dependencies for your AWS Lambda function!

#AWS #Lambda #CloudEngineer #DevOps #Python #IAM #AWSCloud #S3