Cannot use Requests-Module on AWS Lambda
PythonPython 2.7Amazon Web-ServicesAws LambdaPython Problem Overview
I need to do a rest-call within a python script, that runs once per day. I can't pack the "requests" package into my python-package using the AWS Lambdas. I get the error: "Unable to import module 'lambda_function': No module named lambda_function"
I broke it down to the hello_world predefined script. I can pack it into a zip and upload it. Everything works. As soon as I put "import requests" into the file, I get this error.
Here is what I already did:
- The permissions of the zip and the project folder (including subfolders) are set to
chmod 777
. So permissions shouldn't be a problem. - The script itself is within the root folder. When you open the zip file, you directly see it.
- I installed the requests package into the root-folder of the project using
sudo pip install requests -t PATH_TO_ROOT_FOLDER
The naming of everything looks like this:
- zip-file: lambda_function.zip
- py-file: lambda_function.py
- handler method: lambda_handler(event, context)
- handler-definition in the "webconfig: lambda_function.lambda_handler
The file I want to run in the end looks like this:
import requests
import json
def lambda_handler(event, context):
url = 'xxx.elasticbeanstalk.com/users/login'
headers = {"content-type": "application/json", "Authorization": "Basic Zxxxxxxxxx3NjxxZxxxxzcw==" }
response = requests.put(url, headers=headers, verify=False)
return 'hello lambda_handler'
I'm glad for ANY kind of help. I already used multiple hours on this issue.
Python Solutions
Solution 1 - Python
EDIT: On Oct-21-2019 Botocore removed the vendored version of requests: https://github.com/boto/botocore/pull/1829.
EDIT 2: (March 10, 2020): The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now January 30, 2021. https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/
To use requests module, you can simply import requests
from botocore.vendored
. For example:
from botocore.vendored import requests
def lambda_handler(event, context):
response = requests.get("https://httpbin.org/get", timeout=10)
print(response.json())
you can see this gist to know more modules that can be imported directly in AWS lambda.
Solution 2 - Python
If you're working with Python on AWS Lambda, and need to use requests, you better use urllib3, it is currently supported on AWS Lambda and you can import it directly, check the example on urllib3 site.
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
r.data
# b'User-agent: *\nDisallow: /deny\n'
r.status
# 200
Solution 3 - Python
I finally solved the problem: The structure in my zip file was broken. It is important that the python script and the packed dependencies (as folders) are in the root of the zip file. This solved my problem.
It's a bit depressing if you find such easy errors after hours of try and failure.
Solution 4 - Python
I believe you have lambda_function.py
on the Lambda console. You need to first create the Lambda function deployment package, and then use the console to upload the package.
- You create a directory, for example
project-dir
on your system (locally) - create
lambda_function.py
inproject-dir
, copy the content oflambda_function.py
from lambda console and paste it inproject-dir/lambda_function.py
pip install requests -t /path/to/project-dir
- Zip the content of the
project-dir
directory, which is your deployment package (Zip the directory content, not the directory)
Go to the Lambda console, select upload zip file in code entry type and upload your deployment package. Import requests should work without any error.
Solution 5 - Python
With this command download the folder package
pip install requests -t .
Run this command on your local machine, then zip your working directory, then upload to aws.
Solution 6 - Python
python 3.8 windows 10
lambda is looking for a specific folder structure and we are going to recreate in this manner in the steps below (https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-create):
- make a folder on your desktop called "python," open a cmd terminal: cd desktop
- pip install --target python requests
- right click your python folder and zip it and rename the zip to 'requests.zip' - now if you look inside the zip you should see the python folder.
- aws console > lambda > layers > create layer => name layer/upload requests.zip
- aws console > functions > create function => in the "designer" box select layers and then "add layers." Choose custom layers and select your layer.
- Go back to the function screen by clicking on the lambda symbol in the designer box. Now you can see "function code" again. Click lambda_function.py
Now you can import requests like this:
import json
import requests
def lambda_handler(event, context):
# TODO implement
response = requests.get('your_URL')
return {
'statusCode': 200,
'body': json.dumps(response.json())
}
Solution 7 - Python
Most of the comments somehow correct, but not enough informative for AWS beginners. Here is my long resume what needs to be done for accessing requests functionality:
1. Creates root folder for AWS Lambda function
% mkdir lambda-function
2. Go inside crated root folder
% cd lambda-function
3. Create entry point Python file for AWS Lambda.
% vi lambda_function.py
4. Paste a code into lambda_function.py
import requests
def lambda_handler(event, context):
response = requests.get("https://www.test.com/")
print(response.text)
return response.text
5. Install requests library. Note:package folder created
% pip install --target ./package requests
6. Go inside package
% cd package
7. Zip package
zip -r ../deployment-package.zip .
8. Go into parent folder
% cd ..
9. Zip deployment packge and lambda function file
% zip -g deployment-package.zip lambda_function.py
- In the AWS Lambda functions tap "Upload from" and pick ".zip file". Navigate to your zip package zip file: deployment-package.zip.
- After upload all files will be inside AWS Lambda function.
Solution 8 - Python
-
Copy whatever you have in the
lambda_function
fron AWS lambda console and paste it in a new python script and save it aslambda_function.py
. -
Make a new folder (I name it as package) and save
requests
module in it by running the following code in terminal: pip install -t package requests -
Move
lambda_function.py
into the folder (package). -
Go to the folder and select all content and zip them.
-
Go back to the AWS Lambda console. select the function and under the Function code section, click on 'Action' (on the right side) and select
Upload a .zip file
. -
Upload the folder. lambda_function should be uploaded automatically.
-
Run and Enjoy.