107. Replay of Ep 42 – The Birth of NoSQL and DynamoDb – Part 4

Summary

Welcome to Mobycast! This is the fourth episode that we’re re-releasing from back when we told Chris’s personal story of starting Viathan, which was a software startup built to create one of the first internet scale databases. In this episode we finish our deep dive into the technical details of how AWS’s Dynamo DB works. If you’re just joining us at Mobycast, thanks for sticking with us through this five part series. By the end, you’ll be ready to join us on any of our other technical deep dives.

Show Details

What’s under the hood of Amazon’s DynamoDB? Jon Christensen and Chris Hickman of Kelsus continue their discussion on DynamoDB, specifically about it’s architecture and components. They utilize a presentation from re:Invent titled, Amazon DynamoDB Under the Hood: How we built a hyper-scale database.

Some of the highlights of the show include:

    • Partition keys and global secondary indexes determine how data is partitioned across a storage node; allows you to scale out, instead of up
    • Understand how a database is built to make architecture/component definitions less abstract
    • DynamoDB has four components:
      1. Request Router: Frontline service that receives and handles requests
      2. Storage Node: Services responsible for persisting and retrieving data
      3. Partition Metadata System: Keeps track of where data is located
      4. Auto Admin: Handles housekeeping aspects to manage system
    • What level of uptime availability do you want to guarantee?
    • Replication: Strongly Consistent vs. Eventually Consistent
    • Walkthrough of Workflow: What happens when, what does it mean when…
    • DynamoDB architecture and components are designed to improve speed and scalability
    • Split Partitions: Longer times that your database is up and the more data you put into it, the more likely you’re going to get a hot partition or partitions that are too big 

Links and Resources

DynamoDB

re:Invent

Amazon DynamoDB Under the Hood: How we built a hyper-scale database

Paxos Algorithm

Amazon S3

Amazon Relational Database Service (RDS)

MongoDB

JSON

Kelsus

Secret Stache Media

Quotes

“Keep in mind that data is partitioned across storage node, and that’s a key feature of being able to scale out, as opposed to scaling up.” Jon Christensen

“Amazon was opening up the kimono…how DynamoDB has been architected and constructed and how it works.” Chris Hickman

“Managed Service – they get to decide how it’s architected…because they also have to keep it up and live up to their SLA.” Chris Hickman

“The longer the time that your database is up and the more data you put into it, the more likely that you’re going to get a hot partition or partitions are just going to get too big.” Chris Hickman

End Song

Residence by Fortune Finder

Coming soon…

Show Buttons
Hide Buttons
>