DynamoDB Streams + Lambda + NodeJS

Question

Pretty basic question. I have two DynamoDB tables. Table1 contains a list of values, and Table2 will contain an aggregate of values (average, etc).

I am using DynamoDB streams to trigger a lambda function to calculate aggregate values from the data in Table1, then update Table2 with those values. My question is, will the DynamoDB stream contain the ENTIRE table1 or just the new record?

I'm just tweaking the lambda functions to calculate the averages and store into Table2 and I'm just trying to plan the best way to do this. Everything seems to be working. Just stumbling on getting the data from table1. Any other tips regarding forEach for the records will be highly appreciated as well. Thank you!

RoberMP · Accepted Answer · 2018-06-04 12:46:57Z

1

Only the modified field, otherwise it would not make much sense to stream the entire database each time right?. from the docs: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.Tutorial.html

You will receive a message like:

{
    "Records": [
        {
            "eventID": "7de3041dd709b024af6f29e4fa13d34c",
            "eventName": "INSERT",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "us-west-2",
            "dynamodb": {
                "ApproximateCreationDateTime": 1479499740,
                "Keys": {
                    "Timestamp": {
                        "S": "2016-11-18:12:09:36"
                    },
                    "Username": {
                        "S": "John Doe"
                    }
                },
                "NewImage": {
                    "Timestamp": {
                        "S": "2016-11-18:12:09:36"
                    },
                    "Message": {
                        "S": "This is a bark from the Woofer social network"
                    },
                    "Username": {
                        "S": "John Doe"
                    }
                },
                "SequenceNumber": "13021600000000001596893679",
                "SizeBytes": 112,
                "StreamViewType": "NEW_IMAGE"
            },
            "eventSourceARN": "arn:aws:dynamodb:us-east-1:123456789012:table/BarkTable/stream/2016-11-16T20:42:48.104"
        }
    ]
}

Don't know what kind of aggregates you need nor your use case so I can't help with that.

answered Jun 4, 2018 at 12:46

RoberMP

1,35611 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Gene Ellis Over a year ago

Thanks! Table1 is has a collection of bandwidth speeds. Such as upload, download, and ping. Table2 is the average upload, average download and average ping of the values in Table1. Each time a new record comes into Table 1, I want to update the values in Table2 with the new averages.

RoberMP Over a year ago

If you are only calculate averages you just need to keep the count of the results in Table2. New average result will be: (previous average * count + new amount)/ count + 1

Gene Ellis Over a year ago

yep! Got it! Thanks!

Collectives™ on Stack Overflow

DynamoDB Streams + Lambda + NodeJS

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related