DynamoDB in real life

Slide 1

DynamoDB in Real Life Jonathon Hill - Upstate PHP

Slide 2

@compwright compwright.com

Slide 3

Slide 4

For every advantage there is an equal and opposite disadvantage.

Slide 5

Slide 6

Cost-eﬃcient Unlimited scaling Stateless Transactions Parallel operations

Slide 7

Atomic counters Conditional writes Expiration Encryption Streaming

Slide 8

Slide 9

No indexes* No date support Small records Slow console Bad design = $$$

Slide 10

Idiosyncrasies

Slide 11

Query Scan GetItem PutItem UpdateItem DeleteItem TransactGetItems TransactWriteItems BatchGetItem BatchWriteItem …and more

Slide 12

Slide 13

500 InternalServerError 400 ProvisionedThroughputExceededException 400 RequestLimitExceeded 400 ResourceNotFoundException

Slide 14

Slide 15

Slide 16

Slide 17

NULL: null BOOL: boolean S: string N: number B: binary SS: string set NS: number set BS: binary set L: list M: map

Slide 18

Slide 19

Eventually Consistent Strongly Consistent Transactional Read per 4k 1/2 RCU 1 RCU 2 RCU Write per 1k 1 WCU 2 WCU

Slide 20

Dual-key sharded design

Slide 21

Slide 22

Partition key value User ID Possible values Uniformity Many Good Status code Few Bad Timestamp, rounded to day, hour, etc Few Bad Device ID, uniform traﬃc Many Good Device ID, a few “hot” devices Many Bad

Slide 23

Slide 24

Query by key vs. Scan all

Slide 25

Secondary Indexes

Slide 26

Global Secondary Index

Slide 27

Local Secondary Index

Slide 28

Size Shape Velocity

Slide 29

Idempotence and Concurrency

Slide 30

Webhooks Amazon SQS Amazon SNS Amazon Lambda

Slide 31

Slide 32

Real Life

Slide 33

Slide 34

Slide 35

Table Partition Key OrderEvents messageId PendingOrders clientId Failures messageId Sort Key messageId Size RCUs WCUs 3GB 5-100 5-500 56KB 5-500 5-500 68KB 5-50 5-50

Slide 36

Ran out of compute units

Slide 37

Slide 38

Not enough time to retry

Slide 39

Race conditions

Slide 40

Slide 41

Status ConditionExpression created #id <> :id and #did <> :did started #status = :current and #status <> :newStatus queued #status <> :final and #status <> :newStatus finished

Slide 42

Search and filter

Slide 43

Slide 44

More race conditions

Slide 45

Status Step UpdateExpression created 1 started 2 SET #status = :newStatus, #step = :step queued 4 SET #status = :newStatus, #step = :step error 128 SET #status = :newStatus, #step = :step finished +1 SET #status = :newStatus, #step = #step + :step