EP5: 10 Techniques For Database Replication
In episode 5 of System Design Weekly, we explore 10 essential techniques for database replication. These foundational methods are crucial for every software developer to understand! Learn them today and take your system design skills to the next level.
In the next section, we explore different approaches to code organization. From monorepos to submodules, each method has its advantages and challenges. Choosing the right approach for a given use case isn’t always straightforward, and requires careful consideration.
Finally, in this week’s Tech Spotlight, we take a look at how Jira's engineering team leveraged Protobuf to achieve a 20% faster response time and reduce storage usage by 80%!
🛠️ 10 Techniques for Database Replication
Replicated databases are the reality of modern softwares. Here are 10 techniques that will help you understand the design behind any replicated database ~
Single Leader Replication - One primary database (leader) handles all writes, and changes are replicated to follower databases.
Multi Leader Replication - Multiple leader databases allow writes on any node, with changes synchronized across all leaders.
Leaderless Replication - There is no central leader; all nodes are equal, and updates can happen on any node, with conflict resolution and quorum mechanisms in place.
Database Snapshot - A point-in-time copy of a database that captures its exact state at the moment of the snapshot.
Synchronous Replication - Data is written to both the primary and replica databases simultaneously, ensuring strong consistency at the time of the write.
Asynchronous Replication - Data is written to the primary database first, and then replicated to secondary nodes later, which can result in temporary inconsistency.
Semi-Synchronous Replication - A hybrid approach where the primary waits for acknowledgment from at least one replica before confirming a write, balancing speed and consistency.
Chain Replication - Data is replicated in a chain of nodes, where each node forwards updates to the next node in the chain, ensuring consistency across replicas.
Row-Based Replication - Changes are replicated at the individual row level, capturing only the specific data modified during transactions.
Statement-Based Replication - Replication occurs by capturing and replaying the SQL statements that caused changes, rather than the actual data modifications.
≫ Subscribe to my Medium blog for an upcoming deep dive
🗂️ Techniques to Organize Codebases
There are several ways in which a library can be shared across different projects.
Copy’n’Paste: A simple approach where code is duplicated across projects, offering flexibility at the cost of maintainability.
Multirepo: Each library is kept in its own repository, allowing for clear boundaries but requiring more complex dependency management.
Git Submodules: A method to include one repository inside another, enabling version control of dependencies but introducing extra complexity with updates and merges.
Monorepo: All projects and libraries are housed in a single repository, simplifying dependency management and integration but potentially increasing build times and complexity at scale.
🔦 Tech Spotlight!
This week in Tech Spotlight, we’re featuring an insightful blog from Jira's engineering team that explores the practical advantages of using Protobuf over JSON in Jira.
➥ Using Protobuf to make Jira cloud faster
Protobuf is a data serialization/deserialization protocol commonly used with gRPC, but due to technical challenges at Jira, they couldn’t implement gRPC and instead integrated Protobuf directly over REST.
The results were impressive, delivering significant benefits for Jira Cloud, including:
75% less CPU usage on the database
80% smaller payloads compared to serialized JSON
33x faster deserialization and 4x faster serialization
20% faster server response times
It’s a must-read for anyone interested in performance optimisation!
And that is it for this week! If you enjoyed what you read, consider clicking the ❤️ button below and leaving a comment.
See you next week in yet another release of System Design Weekly! Till then, happy learning!