I am trying to save AWS CLI command into python variable (list). The trick is the following code returns result I want but doesn't save it into variable and return empty list.
import os
bashCommand = 'aws s3api list-buckets --query "Buckets[].Name"'
f = [os.system(bashCommand)]
print(f)
output:
[
"bucket1",
"bucket2",
"bucket3"
]
[0]
desired output:
[
"bucket1",
"bucket2",
"bucket3"
]
("bucket1", "bucket2", "bucket3")
If you are using Python and you wish to list buckets, then it would be better to use the AWS SDK for Python, which is boto3:
import boto3
s3 = boto3.resource('s3')
buckets = [bucket.name for bucket in s3.buckets.all()]
See: S3 — Boto 3 Docs
This is much more sensible that calling out to the AWS CLI (which itself uses boto!).
You really don't need anything fancy , all you have to do is import subprocess and json and use them :)
This was tested using python3.6 and on Linux
output = subprocess.run(["aws", "--region=us-west-2", "s3api", "list-buckets", "--query", "Buckets[].Name"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output_utf8=output.stdout.decode('utf-8')
output_utf8_json = json.loads(output_utf8)
print(output_utf8_json)
for key in output_utf8_json:
print(key)
I use this command to create a Python list of all buckets:
bucket_list = eval(subprocess.check_output('aws s3api list-buckets --query "Buckets[].Name"').translate(None, '\r\n '))
