How do I query AWS DynamoDB in python?

Question

I'm fairly new to NoSQL and using AWS DynamoDB. I'm calling it from AWS Lambda using python 2.7 I'm trying to retrieve a value from an order_number field.
This is what my table looks like(only have one record.):

primary partition key: subscription_id

and my secondary global index: order_number

Is my setup correct? If so given the order_number how do I retrieve the record using python?
I can't figure out the syntax to do it.

I've tried

response = table.get_item( Key = {'order_number': myordernumber} )

But i get:
An error occurred (ValidationException) when calling the GetItem operation: The provided key element does not match the schema: ClientError

Schalton · Accepted Answer · 2019-03-17 02:33:45Z

36

DynamoDB does not automatically index all of the fields of your object. By default you can define a hash key (subscription_id in your case) and, optionally, a range key and those will be indexed. So, you could do this:

response = table.get_item(Key={'subscription_id': mysubid})

and it will work as expected. However, if you want to retrieve an item based on order_number you would have to use a scan operation which looks through all items in your table to find the one(s) with the correct value. This is a very expensive operation. Or you could create a Global Secondary Index in your table that uses order_number as the primary key. If you did that and called the new index order_number-index you could then query for objects that match a specific order number like this:

from boto3.dynamodb.conditions import Key, Attr

response = table.query(
    IndexName='order_number-index',
    KeyConditionExpression=Key('order_number').eq(myordernumber))

DynamoDB is an very fast, scalable, and efficient database but it does require a lot of thought about what fields you might want to search on and how to do that efficiently.

The good news is that now you can add GSI's to an existing table. Previously you would have had to delete your table and start all over again.

edited Mar 17, 2019 at 2:33

Schalton

3,1345 gold badges37 silver badges51 bronze badges

answered Dec 9, 2015 at 14:06

garnaat

46k8 gold badges129 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

MalcolmInTheCenter Over a year ago

thanks for your help, i tried the query code you gave me and its giving me a global name 'Key' is not defined and its referring to that line of code you gave me KeyConditionExpression=Key('order_number').eq(myordernumber))

MalcolmInTheCenter Over a year ago

This is what my function looks like

def getRecord( self, table_name, indexName, gsiKey, gsiValue): 		table = self.dynamodb.Table( table_name ) 		response = table.query(IndexName=indexName,KeyConditionExpression=Key(gsiKey).eq(gsiValue))

MalcolmInTheCenter Over a year ago

Nevermind I figured it out, I was only importing boto3 but needed to do from boto3.dynamodb.conditions import Key, Attr

Dawny33 Over a year ago

^^ . Would be great if this comment is included in the answer :)

user3303554 · Accepted Answer · 2016-07-16 00:03:46Z

16

Make sure you've imported this:

from boto3.dynamodb.conditions import Key, Attr

If you don't have it, you'll get the error for sure. It's in the documentation examples.

Thanks @altoids for the comment above as this is the correct answer for me. I wanted to bring attention to it with a "formal" answer.

answered Jul 16, 2016 at 0:03

user3303554

2,3353 gold badges18 silver badges16 bronze badges

Comments

vahdet · Accepted Answer · 2020-08-05 18:53:04Z

9

To query dynamodb using Index with filter:

import boto3
from boto3.dynamodb.conditions import Key, Attr

dynamodb = boto3.resource('dynamodb', region_name=region)
table = dynamodb.Table('<TableName>')

response = table.query(
    IndexName='<Index>',
    KeyConditionExpression=Key('<key1>').eq('<value>') & Key('<key2>').eq('<value>'),
    FilterExpression=Attr('<attr>').eq('<value>')
)

print(response['Items'])

If filter is not rquired then don't use FilterExpression in query.

edited Aug 5, 2020 at 18:53

vahdet

6,78910 gold badges62 silver badges116 bronze badges

answered Jul 21, 2020 at 10:22

singh30

1,51318 silver badges22 bronze badges

1 Comment

matusko Over a year ago

Thanks for showing me the & operator syntax in the KeyConditionExpression

afelaho · Accepted Answer · 2023-01-26 22:35:14Z

1

So far, this is the cleanest way I've discovered; the query is in JSON format.

dynamodb_client = boto3.client('dynamodb')

def query_items():
    arguments = {
        "TableName": "your_dynamodb_table",
        "IndexName": "order_number-index",
        "KeyConditionExpression": "order_number = :V1",
        "ExpressionAttributeValues": {":V1": {"S": "value"}},
    }
    return dynamodb_client.query(**arguments)

edited Jan 26, 2023 at 22:35

answered Jan 26, 2023 at 5:27

afelaho

4714 silver badges3 bronze badges

Comments

manoj praveen · Accepted Answer · 2021-06-23 23:33:33Z

0

import boto3
from boto3.dynamodb.conditions import Key
dynamodb = boto3.resource('dynamodb', region_name=region_name)
table = dynamodb.Table(tableName)

def queryDynamo(pk, sk):
    response = table.query(
        ProjectionExpression="#pk, #sk, keyA, keyB",
        ExpressionAttributeNames={"#pk": "pk", "#sk": "sk"},
        KeyConditionExpression=
            Key('pk').eq(pk) & Key('sk').eq(sk)
    )
    return response['Items']

answered Jun 23, 2021 at 23:33

manoj praveen

412 bronze badges

Comments

Harry Jones · Accepted Answer · 2022-08-26 14:28:05Z

0

If you use the boto3 dynamodb client, you can do the following (again you would need to use subscription_id as that is the primary key):

dynamodb = boto3.client('dynamodb')
response = dynamodb.query(
    TableName='recurring_charges', 
    KeyConditionExpression="subscription_id = :subscription_id",
    ExpressionAttributeValues={":subscription_id": {"S": "id"}}
)

answered Aug 26, 2022 at 14:28

Harry Jones

3623 silver badges10 bronze badges

1 Comment

hafiz ali Over a year ago

dont we need a index here?

Collectives™ on Stack Overflow

How do I query AWS DynamoDB in python?

6 Answers 6

4 Comments

Comments

1 Comment

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

4 Comments

Comments

1 Comment

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related