06/10/19

The Various Flavors of Two-Phase Commits — Explained

Internally, NuoDB uses the two-phase commit protocol to manage the durability of user data. NuoDB also supports the X/Open XA protocol for synchronizing global transactions across multiple data stores. XA is also sometimes referred to as two-phase commit. The fundamental principles in both protocols are similar but serve different purposes. Let us explore the difference between these two protocols.

A SINGLE TRANSACTION ACROSS TWO RESOURCES

Let us explore a simple use case. A simple application takes messages from one data source (outgoing_messages) and writes them to a new data source (incoming_messages). This is a fairly common use case if you read messages from a message queue such as Apache ActiveMQ and you write them to a database (such as NuoDB). The following code snippet shows this example in the corresponding SQL form.

SQL> SELECT * FROM outgoing_messages;
 
 ID     MSG    
 --- --------- 
 
  1  message 1 
 
SQL> START TRANSACTION;
SQL> SELECT * FROM outgoing_messages;
 
 ID     MSG    
 --- --------- 
 
  1  message 1 
 
SQL> DELETE FROM outgoing_messages WHERE id=1;
SQL> INSERT INTO incoming_messages(id, msg) VALUES(1, 'message 1');
SQL> commit;

When executed in a single relational database, the two statements (insert and delete) are expected to behave according to the ACID guarantees. They either both succeed or they both fail. Throughout this article, we focus on the A(tomic) guarantee of ACID.

XA abstracts away the statements and transaction lifetime for scenarios where the tables live in different data stores. The following example is a simplified version of an ActiveMQ consumer that receives a message and writes it to NuoDB. Due to space constraints in this article, the code does not contain any setup or failure handling.

​javax.jms.MessageConsumer consumer=xasession.createConsumer(queue);
        MessageListener listener = NEW MessageListener() {
            @Override
            public void onMessage(Message msg) {
                TextMessage msg1=(TextMessage)msg;
 
                NuoXADataSource nuodbDs = NEW NuoXADataSource();
                XAConnection nuodbXAconn = nuodbDs.getXAConnection(DBA_USER, DBA_PASSWORD);
 
                XAResource mqRes = xasession.getXAResource();
                XAResource nuoRes = nuodbXAconn.getXAResource();
 
                nuodbStmt.executeUpdate(String.format("insert into incoming_messages(id, msg) values(1, '%s')", msg1.getText()));
 
                mqRes.END(xid, XAResource.TMSUCCESS);
                nuoRes.END(xid, XAResource.TMSUCCESS);
 
                mqRes.PREPARE(xid);
                nuoRes.PREPARE(xid);
 
                mqRes.commit(xid, FALSE);
                nuoRes.commit(xid, FALSE);
            }
        };
Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin
04/3/19

Distributed Transactional Locks

As I explained in a previous blog post, sometimes MVCC is not sufficient and an operation needs to block out all other concurrent modifications. NuoDB is able to lock three types of lockable resources: tables, schemas, and sequences. A resource can either be locked in SHARED mode (which still allows record modification, but no metadata modification) or EXCLUSIVE which prevents any concurrent modification.

EXCLUSIVE access is required for DDL that visits all records (various index operations for example) and distributed concurrent access is not possible. Exclusive access can also be used by operations that need to work around MVCC write skew anomalies.

SHARED locks need to be fast, consume as little memory as possible, involve no additional nodes in the cluster, and cause no additional messaging.

EXCLUSIVE locks, on the other hand, need to pay the cost.

MVCC uses row locks to serialise updates to a single record. Both transactional locks and row locks participate in the same deadlock detection process, but are otherwise independent. The following table explains the interaction between transactional locks and MVCC row locks. If you are interested in how row locks work, I recommend reading our MVCC blog series.

Row Locks (DML) / Transactional LocksSharedExclusive
selectNo ConflictNo Conflict
insert, update, delete, select for updateNo ConflictConflict
Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin
04/3/19

How Transaction Locks support Zero Overhead Distributed DML

Let us imagine a scenario that needs to prevent MVCC write skews…

One transaction increases the salary of everyone in a department by 10%; another transaction inserts a new employee with a salary X. Since the two transactions do not conflict, MVCC does not prevent either of them from committing. After both resolve, the overall salary in the department could be above the budget. To prevent a similar situation, the application developer might want to have exclusive access to the table.

NuoDB 3.2.2 exposes a new type of lock that guarantees exclusive access to a resource across the distributed cluster. We call them Transactional Locks and they are the underlying mechanism powering the new NuoDB LOCK statement.

Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin
08/2/18

Why A Chairman Ain’t the Master Replica

INTRO

As a distributed, ACID-compliant database, we get a lot of questions about how NuoDB works, and how it compares to more well-known architectures. In particular, people want to understand how NuoDB compares to a master / master architecture, and (related) how we handle lock management. This post will discuss both. We will also discuss when shared-nothing architecture is viable and when conflicting decisions need to coordinated.

In this blog post, we will discuss the concept of Chairmanship, NuoDB’s answer to distributed coordination. We will prove that Chairmanship is both consistent and resilient to failure. But before we go into the details, let us explore the basics of NuoDB architecture.

Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

07/5/18

Quick dive into NuoDB Architecture

Introduction

Traditionally, relational databases were designed for scale-up architectures. Supporting more clients or higher throughput required an upgrade to a larger server. Until recently, this meant that implementing a read & write scale-out architecture either required a NoSQL database and abandoning traditional relational database benefits, or relying on sharding and explicit replication. There were no solutions that could scale-out and still provide complete ACID (Atomicity, Consistency, Isolation, and Durability) -compliant semantics. This tension is what inspired the NewSQL movement and ultimately led to today’s modern “elastic SQL” databases.

NuoDB is an elastic SQL database designed with distributed application deployment challenges in mind. It’s a full SQL solution that provides all the properties of ACID-compliant transactions and standard relational SQL language support. It’s also designed from the start as a distributed system that scales the way a cloud service has to scale, providing high availability and resiliency with no single points of failure. Different from traditional shared-disk or shared nothing architectures, NuoDB’s patented database presents a new kind of peer-to-peer, on-demand independence that yields high availability, low-latency, and a deployment model that is easy to manage.

This article highlights the key concepts and architectural differences that set NuoDB apart from traditional relational databases and even other elastic SQL databases.

Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin