Last year at re:Invent, Quantum Ledger Database was announced. Amazon QLDB is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log owned by a central trusted authority. In today’s workshop, no Python will be used, we’re writing Java today. Yay for curly braces!
The typical database log is hidden for the user, it is used for system things like recovery. It is not immutable, it can be modified by people on the operations team for example. QLDB took a different view on this, they use an append-only journal. You can insert any document model you want since QLDB is a NoSQL database. The inserted documents are appended to the end of the journal as a block. The block also included system information like the date the block was added. This information is also queryable. Whem you first insert a document, it is assigned version 0. When you update the document, the version is increased. When you delete it, the deletion is recorded as a transaction as well.
Records can not be altered, there is no API the change things in the journal. All actions, including deletes are written to the journal.
When data is stored, the document is cryptographically verified and the block hash is stored in the block.
Documents are stored in Amazon Ion format and are queried using PartiQL pronounced as ‘Party Queue El’ (or not, I’m not your mom)
PartiQL is the query language used in QLDB. To make data easy to move between multiple types of database, one language can be used to query it. It’s SQL compatible language with minimal extensions. As stated above, it is format and datastore independent. Most statement and functions that you already know from SQL are supported in PartiQL.
Amazon Ion aims to be effecient in changing data between applications. There is no schema, so it can grow as the application needs. It supports both a human readable text format and a binary format for efficiency. SQL scalar types are covered like numbers and strings. This thing looks like JSON, and as a MongoDB user, this looks promising.
The Java Driver API is open-source on GitHub. It does all the cool stuff like session pooling, transaction commit handling and automatic retries for recoverable errors. The NodeJS and Python drivers are also availble, but why would you even bother?
QLDB generates a sha256 for each document. Use the API of QLDB to verify the documents that exist in the journal. With this, you can make sure that a document has not been altered. So when someone with the wrong data, who claims that they have the right data, you can ask them to prove that their data is right. And they will fail.
Ingest QLDB streams into Kinesis, process it directly with Lambda. Comparable to DynamoDB streams.