Write-ahead Log

Motivation

A machine stops running while in the middle of performing a data modification. When it restarts, based on atomicity and durability needs, it might decide to redo or undo what it was doing. For this, it must know where it left off.

Solution

To guarantee durability and data integrity, each modification to the system is first written to an append-only log on the disk (known as Write-Ahead Log (WAL) or transaction log or commit log) Each log entry should contain enough information to redo or undo the modification. The log can be read on every restart to recover the previous state by replaying all the log entries.

Each node, in a distributed environment, maintains its own log. WAL is always sequentially appended, which simplifies the handling of the log. Each log entry is given a unique identifier; this identifier helps in implementing certain other operations like log segmentation or log purging.

Applications

• Cassandra: To ensure durability, whenever a node receives a write request, it immediately writes the data to a commit log which is a WAL.

• Kafka implements a distributed Commit Log to persistently store all messages it receives.

• Chubby: For fault tolerance and in the event of a leader crash, all database transactions are stored in a transaction log