Problem Statement
Design a client-side local-first synchronization engine (similar to the data stores powering Notion, Linear, or Obsidian). The engine must allow clients to read and write records locally with zero latency, capture modifications as commands in an outbound queue when offline, detect network connectivity changes, and sync changes back to a central database using a Conflict Resolution policy (like Last-Write-Wins) when online.
Asked In Companies
Design Decisions & Patterns Used
Local-first applications store data locally first and sync it to the cloud asynchronously. This architecture ensures the app remains responsive and fully functional offline. To coordinate updates, we encapsulate all database modifications as transaction command payloads and use timestamp-based Last-Write-Wins (LWW) policies to resolve merge conflicts at the server.
We will utilize the following Design Patterns:
- Command Pattern: Wrapping mutations (INSERT, UPDATE) as serializable command payloads to easily queue, retry, or replay them.
- State Pattern: Tracking connection states (ONLINE/OFFLINE) and dynamically modifying the behavior of the synchronization queue.
- CQRS (Command Query Responsibility Segregation): Separating client reads (which query the local index directly) from synchronization writes (which process mutation commands).
Functional Requirements
- Support local reads and writes on the client database.
- Capture all local data modifications in an outbound synchronization queue.
- Allow offline modifications. Keep mutations in the queue until connection is restored.
- Flush the mutation queue to the server when connection is restored.
- Enforce Last-Write-Wins (LWW) conflict resolution at the server: only update records if the incoming mutation has a newer timestamp than the server's record.
Objects Required
NetworkStatus(Enum mapping ONLINE and OFFLINE states)Record(Value object wrapping data entries, versions, and edit timestamps)Mutation(Command object encapsulating a database write operation)LocalDatabase(Client-side engine managing local maps and queues)ServerDatabase(Central database resolving conflicts and storing master data)
NetworkStatus Enum & Record Class
The NetworkStatus enum defines connectivity states, and the Record class models the data payloads with edit timestamps.
public enum NetworkStatus {
ONLINE,
OFFLINE
}
Let's define the Record class:
public class Record {
private final String id;
private String value;
private long version;
private long updatedAt;
public Record(String id, String value, long version, long updatedAt) {
this.id = id;
this.value = value;
this.version = version;
this.updatedAt = updatedAt;
}
public String getId() { return id; }
public synchronized String getValue() { return value; }
public synchronized void setValue(String value) { this.value = value; }
public synchronized long getVersion() { return version; }
public synchronized void setVersion(long version) { this.version = version; }
public synchronized long getUpdatedAt() { return updatedAt; }
public synchronized void setUpdatedAt(long updatedAt) { this.updatedAt = updatedAt; }
public synchronized Record cloneRecord() {
return new Record(id, value, version, updatedAt);
}
}
The constructor sets initial properties. Getter and setter methods are synchronized to ensure thread safety during local reads and background synchronization writes. The cloneRecord() method allows creating copies to prevent memory reference leaks.
Mutation Class (Command)
The Mutation class wraps database write commands, capturing the record snapshot and the exact timestamp the change occurred on the client.
public class Mutation {
private final String recordId;
private final Record recordSnapshot;
private final long clientTimestamp;
public Mutation(String recordId, Record recordSnapshot) {
this.recordId = recordId;
this.recordSnapshot = recordSnapshot.cloneRecord();
this.clientTimestamp = recordSnapshot.getUpdatedAt();
}
public String getRecordId() { return recordId; }
public Record getRecordSnapshot() { return recordSnapshot; }
public long getClientTimestamp() { return clientTimestamp; }
}
The constructor captures a snapshot of the record state at the moment of modification, preserving its data and edit timestamp.
LocalDatabase Class (Client Store)
The LocalDatabase manages client data, queues modifications, and handles synchronization runs.
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class LocalDatabase {
private final Map<String, Record> clientStore;
private final List<Mutation> outboundQueue;
private NetworkStatus networkStatus;
public LocalDatabase() {
this.clientStore = new ConcurrentHashMap<>();
this.outboundQueue = new ArrayList<>();
this.networkStatus = NetworkStatus.ONLINE; // default to online
}
public synchronized void setNetworkStatus(NetworkStatus status) {
this.networkStatus = status;
System.out.println("Client network status changed to: " + status);
}
public Record read(String id) {
Record r = clientStore.get(id);
return r != null ? r.cloneRecord() : null;
}
public synchronized void write(String id, String value) {
Record existing = clientStore.get(id);
long now = System.currentTimeMillis();
Record updated;
if (existing == null) {
updated = new Record(id, value, 1, now);
} else {
updated = new Record(id, value, existing.getVersion() + 1, now);
}
clientStore.put(id, updated);
// Queue mutation command for synchronization
outboundQueue.add(new Mutation(id, updated));
System.out.println("Local write successful: " + id + " = '" + value + "'");
}
public synchronized void sync(ServerDatabase server) {
if (networkStatus == NetworkStatus.OFFLINE) {
System.out.println("Cannot sync: Client is currently OFFLINE. Changes remain queued.");
return;
}
if (outboundQueue.isEmpty()) {
System.out.println("No local changes to sync.");
return;
}
System.out.println("Syncing " + outboundQueue.size() + " local mutations to server...");
List<Mutation> syncedMutations = new ArrayList<>();
for (Mutation mutation : outboundQueue) {
boolean accepted = server.applyMutation(mutation);
if (accepted) {
syncedMutations.add(mutation);
} else {
// Conflict occurred and server had a newer update. Fetch server's latest data.
Record latestServerRecord = server.read(mutation.getRecordId());
if (latestServerRecord != null) {
clientStore.put(mutation.getRecordId(), latestServerRecord.cloneRecord());
System.out.println("Conflict resolved: Client updated to match Server's newer data.");
}
}
}
outboundQueue.removeAll(syncedMutations);
System.out.println("Sync completed. Remaining queued mutations: " + outboundQueue.size());
}
}
Here is an explanation of the core operations in the LocalDatabase class:
- The constructor initializes concurrent maps for local reads, and uses synchronized lists to track outbound mutations.
read()queries the local store directly and returns a clone, ensuring zero-latency reads.write()updates the local store, increments versions, sets timestamps, and enqueues a newMutationcommand.sync()evaluates connectivity. If online, it iterates through the queue and applies mutations to the server. If the server rejects a mutation due to a conflict, the client fetches the server's latest record to update its local store.
ServerDatabase Class (Master Store)
The ServerDatabase acts as the master store. It enforces conflict resolution using a Last-Write-Wins (LWW) policy based on client update timestamps.
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class ServerDatabase {
private final Map<String, Record> masterStore = new ConcurrentHashMap<>();
public Record read(String id) {
Record r = masterStore.get(id);
return r != null ? r.cloneRecord() : null;
}
public synchronized void forceServerWrite(String id, String value) {
long now = System.currentTimeMillis();
masterStore.put(id, new Record(id, value, 1, now));
}
public synchronized boolean applyMutation(Mutation mutation) {
Record incoming = mutation.getRecordSnapshot();
Record existing = masterStore.get(mutation.getRecordId());
if (existing == null) {
// First time this record is created
masterStore.put(mutation.getRecordId(), incoming.cloneRecord());
return true;
}
// Conflict Resolution Check: Last-Write-Wins (LWW)
if (incoming.getUpdatedAt() > existing.getUpdatedAt()) {
System.out.println("[Server] Accepted mutation for: " + mutation.getRecordId() +
" (Client time: " + incoming.getUpdatedAt() + " > Server time: " + existing.getUpdatedAt() + ")");
masterStore.put(mutation.getRecordId(), incoming.cloneRecord());
return true;
} else {
System.out.println("[Server] Rejected outdated mutation for: " + mutation.getRecordId() +
" (Client time: " + incoming.getUpdatedAt() + " <= Server time: " + existing.getUpdatedAt() + ")");
return false;
}
}
}
Here is an explanation of the core operations in the ServerDatabase class:
applyMutation()compares the client timestamp of the incoming mutation against the timestamp of the existing record on the server. If the incoming mutation is newer, it updates the server. If it is older, the mutation is rejected.forceServerWrite()simulates updates from other clients on the server, helping us test conflict resolution scenarios.
Main Driver Class
This class tests our local-first sync engine. It processes writes offline, queues changes, flushes them online, and simulates conflict resolution scenarios.
public class Main {
public static void main(String[] args) throws InterruptedException {
LocalDatabase client = new LocalDatabase();
ServerDatabase server = new ServerDatabase();
System.out.println("--- Scenario 1: Online Writes ---");
client.write("doc-1", "Version 1 content");
client.sync(server);
System.out.println("Server doc-1 content: '" + server.read("doc-1").getValue() + "'");
System.out.println("\n--- Scenario 2: Offline Writes ---");
client.setNetworkStatus(NetworkStatus.OFFLINE);
client.write("doc-1", "Version 2 content (written offline)");
client.write("doc-2", "New document (written offline)");
// Try syncing while offline
client.sync(server);
System.out.println("\n--- Scenario 3: Going Online and Syncing ---");
client.setNetworkStatus(NetworkStatus.ONLINE);
client.sync(server);
System.out.println("Server doc-1 content: '" + server.read("doc-1").getValue() + "'");
System.out.println("Server doc-2 content: '" + server.read("doc-2").getValue() + "'");
System.out.println("\n--- Scenario 4: Conflict Resolution (Last-Write-Wins) ---");
// Simulate client going offline
client.setNetworkStatus(NetworkStatus.OFFLINE);
client.write("doc-1", "Outdated client content");
// Wait briefly and simulate another client writing a newer version directly to the server
Thread.sleep(10);
server.forceServerWrite("doc-1", "Newer server content written by another client");
// Client goes online and tries to sync its outdated mutation
client.setNetworkStatus(NetworkStatus.ONLINE);
client.sync(server);
// Verify that the client's outdated mutation was rejected and the client updated to the server's newer version
System.out.println("Final Client doc-1 content: '" + client.read("doc-1").getValue() + "'");
System.out.println("Final Server doc-1 content: '" + server.read("doc-1").getValue() + "'");
}
}
The main() driver executes online writes, goes offline to modify records, triggers synchronization when online, and asserts that conflicts are correctly resolved using Last-Write-Wins logic.
Comments
Post a Comment