7

For a variety of reasons, I must use a custom layer to import numpy into my Lambda function. I am able to create other layers from which Lambda is able to import libraries without issue, but when attempting to import numpy, I run into the following error:

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

I am really stumped on this issue with numpy and Lambda and would appreciate any help!

The following are some points which may be helpful in determining the issue:

  • The numpy wheel I am creating the custom layer package from is: https://files.pythonhosted.org/packages/87/8c/1c5a2bbfb55e4ff35f9c30933e8816b188de3cc3e2eaf9304906991984a9/numpy-1.26.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

  • Using platform.uname(), I am able to confirm that I am on the correct x86_64 architecture: (system='Linux', node='169.254.84.165', release='4.14.255-327-266.539.amzn2.x86_64', version='#1 SMP Sun Oct 22 13:13:51 UTC 2023', machine='x86_64')

  • The package structure of the zipped numpy package used for the numpy layer is: python/lib/python3.10/site-packages/<extracted numpy wheel contents>

  • According to sys.executable within Lambda, the python executable is at the following path: /var/lang/bin/python3.10

  • According to __file__ from libraries which are able to be imported without issue, the directory in which all dependencies are in is /opt/python/lib/python3.10/site-packages

  • Further, the following is the contents of the /opt/python/lib/python3.10/site-packages directory:

['Bio', 'BioSQL', 'PyYAML-6.0.1.dist-info', '__pycache__', '_distutils_hack', '_yaml', 'bio-1.6.0.dist-info', 'biopython-1.81.dist-info', 'biorun', 'biothings_client', 'biothings_client-0.3.1.dist-info', 'certifi', 'certifi-2023.7.22.dist-info', 'charset_normalizer', 'charset_normalizer-3.3.2.dist-info', 'dateutil', 'distutils-precedence.pth', 'easy_install.py', 'gprofiler', 'gprofiler_official-1.0.0.dist-info', 'idna', 'idna-3.4.dist-info', 'mygene', 'mygene-3.2.2.dist-info', 'numpy', 'numpy-1.26.2.dist-info', 'numpy.libs', 'packaging', 'packaging-23.2.dist-info', 'pandas', 'pandas-2.1.3.dist-info', 'pip', 'pip-23.3.1.dist-info', 'pkg_resources', 'platformdirs', 'platformdirs-4.0.0.dist-info', 'pooch', 'pooch-1.8.0.dist-info', 'python_dateutil-2.8.2.dist-info', 'pytz', 'pytz-2023.3.post1.dist-info', 'requests', 'requests-2.31.0.dist-info', 'setuptools', 'setuptools-50.3.2.dist-info', 'six-1.16.0.dist-info', 'six.py', 'tqdm', 'tqdm-4.66.1.dist-info', 'tzdata', 'tzdata-2023.3.dist-info', 'urllib3', 'urllib3-1.26.18.dist-info', 'yaml']
5
  • 2
    Have you tried using the AWS-provided layer? See: How to import numpy using layer in lambda aws? Commented Nov 15, 2023 at 22:05
  • 1
    @JohnRotenstein Unfortunately, the AWS environment that my Lambda function exists within does not have access to the AWS provided layer Commented Nov 16, 2023 at 14:33
  • @BenBole: How did you solve the problem? Commented Jun 16, 2024 at 8:29
  • 1
    I have the same issue. I created an AWS Lambda layer for rio-tiler which has numpy in it. Commented Oct 3, 2024 at 12:14
  • I have the same issue and my learning user ID doesn't have access to the AWSSDKPandas-Python39 layer as well. Can anyone kindly help? Thanks! Commented Nov 26, 2024 at 19:26

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.