aws cli works perfectly but Python script gets CERTIFICATE_VERIFY_FAILED

Question

I ssh to my EC2 instance. I can run these commands and they work perfectly:

aws sqs list-queues

aws s3 ls

I have a small Python script that pulls data from a database, formats it as XML, and then uploads the file to S3. This upload fails with this error:

Traceback (most recent call last):
  File "./data_test/data_analytics/lexisnexis/async2.py", line 289, in <module>
    insert_parallel(engine, qy, Create_Temp.profile_id, nworkers)
  File "./data_test/data_analytics/lexisnexis/async2.py", line 241, in insert_parallel
    s3upload(bucketname, keyname, f)
  File "./data_test/data_analytics/lexisnexis/async2.py", line 89, in s3upload
    bucket = conn.get_bucket(bucketname)
  File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 506, in get_bucket
    return self.head_bucket(bucket_name, headers=headers)
  File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 525, in head_bucket
    response = self.make_request('HEAD', bucket_name, headers=headers)
  File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 668, in make_request
    retry_handler=retry_handler
  File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1030, in _mexe
    raise ex
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

How can I have a script that dies, even when aws cli works?

To be clear, I'm running the Python script as the same user, from the same EC2 instance, as I run the aws cli commands.

aws --version

aws-cli/1.11.176 Python/2.7.12 Linux/4.9.43-17.38.amzn1.x86_64 botocore/1.7.34

What happens if you run "aws s3api head-bucket --bucket xyz" with your bucket name in place of xyz? — jarmod
– jarmod, Commented Oct 25, 2017 at 0:42

John Hanley · Accepted Answer · 2017-10-24 23:53:47Z

1

The last line of your error messages tells you the problem:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Your issue could be one of the following:

1) There is an error with the certificate with the server that you are connecting to.

2) The certificate chain is incomplete for the server that you are connecting to.

3) You are missing "cacert.pem". Do a Google search on "cacert.pem". This is a common problem and there is a lot of information on downloading and installing this file.

Certificate verification in Python

answered Oct 24, 2017 at 23:53

John Hanley

81.8k7 gold badges115 silver badges181 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

LRK9 Over a year ago

Interesting, but how is it that aws cli has no problem, but boto is too stupid to do whatever aws cli is doing?

Collectives™ on Stack Overflow

aws cli works perfectly but Python script gets CERTIFICATE_VERIFY_FAILED

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related