For a variety of reasons, I must use a custom layer to import numpy into my Lambda function. I am able to create other layers from which Lambda is able to import libraries without issue, but when attempting to import numpy, I run into the following error:
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
I am really stumped on this issue with numpy and Lambda and would appreciate any help!
The following are some points which may be helpful in determining the issue:
The numpy wheel I am creating the custom layer package from is:
https://files.pythonhosted.org/packages/87/8c/1c5a2bbfb55e4ff35f9c30933e8816b188de3cc3e2eaf9304906991984a9/numpy-1.26.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whlUsing
platform.uname(), I am able to confirm that I am on the correct x86_64 architecture:(system='Linux', node='169.254.84.165', release='4.14.255-327-266.539.amzn2.x86_64', version='#1 SMP Sun Oct 22 13:13:51 UTC 2023', machine='x86_64')The package structure of the zipped numpy package used for the numpy layer is:
python/lib/python3.10/site-packages/<extracted numpy wheel contents>According to
sys.executablewithin Lambda, the python executable is at the following path:/var/lang/bin/python3.10According to
__file__from libraries which are able to be imported without issue, the directory in which all dependencies are in is/opt/python/lib/python3.10/site-packagesFurther, the following is the contents of the
/opt/python/lib/python3.10/site-packagesdirectory:
['Bio', 'BioSQL', 'PyYAML-6.0.1.dist-info', '__pycache__', '_distutils_hack', '_yaml', 'bio-1.6.0.dist-info', 'biopython-1.81.dist-info', 'biorun', 'biothings_client', 'biothings_client-0.3.1.dist-info', 'certifi', 'certifi-2023.7.22.dist-info', 'charset_normalizer', 'charset_normalizer-3.3.2.dist-info', 'dateutil', 'distutils-precedence.pth', 'easy_install.py', 'gprofiler', 'gprofiler_official-1.0.0.dist-info', 'idna', 'idna-3.4.dist-info', 'mygene', 'mygene-3.2.2.dist-info', 'numpy', 'numpy-1.26.2.dist-info', 'numpy.libs', 'packaging', 'packaging-23.2.dist-info', 'pandas', 'pandas-2.1.3.dist-info', 'pip', 'pip-23.3.1.dist-info', 'pkg_resources', 'platformdirs', 'platformdirs-4.0.0.dist-info', 'pooch', 'pooch-1.8.0.dist-info', 'python_dateutil-2.8.2.dist-info', 'pytz', 'pytz-2023.3.post1.dist-info', 'requests', 'requests-2.31.0.dist-info', 'setuptools', 'setuptools-50.3.2.dist-info', 'six-1.16.0.dist-info', 'six.py', 'tqdm', 'tqdm-4.66.1.dist-info', 'tzdata', 'tzdata-2023.3.dist-info', 'urllib3', 'urllib3-1.26.18.dist-info', 'yaml']