Design an API Gateway / Reverse Proxy

Problem Statement

Design an API Gateway / Reverse Proxy library (similar to Spring Cloud Gateway or Zuul). The gateway should act as a single entry point for microservice architectures. It must support routing requests to backend microservice paths dynamically, piping incoming requests through a pipeline of sequential interceptors (using the Chain of Responsibility pattern) for tasks like request logging and authentication, and aborting the request chain immediately if validations fail.

Asked In Companies

AWS Salesforce VMware Twilio

---

Functional Requirements

• Register routes mapping request path prefixes (e.g., /users) to backend microservice hosts (e.g., http://user-service).
• Pass incoming requests through an extensible pipeline of sequential middleware filters (Chain of Responsibility Pattern).
• Support global pre-processing filters, such as LoggingFilter and AuthenticationFilter.
• Support terminating the filter chain immediately (e.g., returning 401 Unauthorized) if a validation filter fails.
• Execute route mapping to target services, appending paths dynamically to the backend host.

---

Objects Required

• GatewayRequest (Model representing the client's HTTP request details)
• GatewayResponse (Model representing the gateway's response)
• GatewayFilter (Interface for middleware interceptors)
• FilterChain (Controller tracking filter indices and directing execution flows)
• ApiGateway (Core registry and request processor)

---

GatewayRequest & GatewayResponse Classes

These data classes encapsulate the standard HTTP header maps, target paths, status codes, and body payloads.

import java.util.HashMap;
import java.util.Map;

public class GatewayRequest {
    private final String path;
    private final String method;
    private final Map<String, String> headers;
    private final String body;

    public GatewayRequest(String path, String method, String body) {
        this.path = path;
        this.method = method;
        this.body = body;
        this.headers = new HashMap<>();
    }

    public void addHeader(String name, String value) {
        headers.put(name.toLowerCase(), value);
    }

    public String getHeader(String name) {
        return headers.get(name.toLowerCase());
    }

    public String getPath() { return path; }
    public String getMethod() { return method; }
    public String getBody() { return body; }
}

The GatewayRequest constructor sets the path, HTTP method, and body. Headers are normalized to lowercase keys to avoid case-sensitivity issues during checks.

import java.util.HashMap;
import java.util.Map;

public class GatewayResponse {
    private int statusCode;
    private String body;
    private final Map<String, String> headers;

    public GatewayResponse() {
        this.statusCode = 200; // default to OK
        this.headers = new HashMap<>();
    }

    public int getStatusCode() { return statusCode; }
    public void setStatusCode(int statusCode) { this.statusCode = statusCode; }

    public String getBody() { return body; }
    public void setBody(String body) { this.body = body; }

    public void setHeader(String name, String value) {
        headers.put(name, value);
    }
}

The GatewayResponse initializes with a default 200 OK status. Setter methods allow filters to mutate response payloads dynamically.

---

GatewayFilter Interface & FilterChain Class

We apply the **Chain of Responsibility Pattern**. Each filter receives a reference to the active FilterChain, allowing it to decide whether to pass the request to the next interceptor.

public interface GatewayFilter {
    void filter(GatewayRequest request, GatewayResponse response, FilterChain chain);
}

The filter() method allows intercepting operations before and after execution down the chain. Let's write the coordinator class:

import java.util.List;

public class FilterChain {
    private final List<GatewayFilter> filters;
    private int index = 0;
    private final String targetUrl;

    public FilterChain(List<GatewayFilter> filters, String targetUrl) {
        this.filters = filters;
        this.targetUrl = targetUrl;
    }

    public void doFilter(GatewayRequest request, GatewayResponse response) {
        if (index < filters.size()) {
            GatewayFilter activeFilter = filters.get(index);
            index++;
            activeFilter.filter(request, response, this);
        } else {
            // End of chain reached: simulate calling the target backend service
            System.out.println("Forwarding request to target service: " + targetUrl);
            response.setStatusCode(200);
            response.setBody("Response from: " + targetUrl + " for path: " + request.getPath());
        }
    }
}

The doFilter() method sequentially tracks the filter execution index. If there are more filters, it calls filter() on the active instance. If the end of the chain is reached, it mocks calling the target backend service.

---

Concrete Filter Implementations

Let's implement LoggingFilter and AuthenticationFilter.

public class LoggingFilter implements GatewayFilter {
    @Override
    public void filter(GatewayRequest request, GatewayResponse response, FilterChain chain) {
        long startTime = System.currentTimeMillis();
        System.out.println("[LoggingFilter] Pre-processing request: " + request.getMethod() + " " + request.getPath());

        // Delegate to the next filter in the chain
        chain.doFilter(request, response);

        long executionTime = System.currentTimeMillis() - startTime;
        System.out.println("[LoggingFilter] Post-processed request. Execution time: " + executionTime + "ms");
    }
}

The LoggingFilter logs requests, calls chain.doFilter() to pass control to the next filter, and calculates request execution time once the call stack returns.

public class AuthenticationFilter implements GatewayFilter {
    @Override
    public void filter(GatewayRequest request, GatewayResponse response, FilterChain chain) {
        System.out.println("[AuthenticationFilter] Checking credentials...");
        String token = request.getHeader("Authorization");

        if (token != null && token.equals("Bearer valid-secret-token")) {
            System.out.println("[AuthenticationFilter] Token authenticated successfully.");
            chain.doFilter(request, response); // Pass control forward
        } else {
            // Terminate the chain immediately
            System.out.println("[AuthenticationFilter] Auth check failed. Aborting request chain.");
            response.setStatusCode(401);
            response.setBody("401 Unauthorized: Access token is invalid or missing.");
        }
    }
}

The AuthenticationFilter inspects the `Authorization` header. If valid, it passes control forward. If invalid, it sets a `401 Unauthorized` status and skips calling chain.doFilter(), terminating the request chain immediately.

---

ApiGateway Class

The ApiGateway class acts as the core registry, maintaining configured routes and global filter pipelines.

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class ApiGateway {
    private final Map<String, String> routeMap;
    private final List<GatewayFilter> globalFilters;

    public ApiGateway() {
        this.routeMap = new HashMap<>();
        this.globalFilters = new ArrayList<>();
    }

    public void addRoute(String pathPrefix, String targetUrl) {
        routeMap.put(pathPrefix, targetUrl);
        System.out.println("Configured Route: prefix " + pathPrefix + " maps to " + targetUrl);
    }

    public void addFilter(GatewayFilter filter) {
        globalFilters.add(filter);
    }

    public GatewayResponse handleRequest(GatewayRequest request) {
        GatewayResponse response = new GatewayResponse();
        
        // Find target routing URL
        String targetService = null;
        for (Map.Entry<String, String> entry : routeMap.entrySet()) {
            if (request.getPath().startsWith(entry.getKey())) {
                targetService = entry.getValue() + request.getPath().substring(entry.getKey().length());
                break;
            }
        }

        if (targetService == null) {
            response.setStatusCode(404);
            response.setBody("404 Not Found: Route not configured for path: " + request.getPath());
            return response;
        }

        // Initialize and trigger the filter chain
        FilterChain chain = new FilterChain(globalFilters, targetService);
        chain.doFilter(request, response);

        return response;
    }
}

Here is an explanation of the core operations in the ApiGateway class:

• The constructor initializes route tables and filter list variables.
• addRoute() and addFilter() register target systems and dynamic filter plug-ins.
• handleRequest() searches for a matching prefix, compiles the target routing path, instantiates the FilterChain, and triggers execution. It returns a `404` status if no match is found.

---

Main Driver Class

This class tests our API Gateway. It configures paths, adds global filters, processes authenticated calls, and demonstrates the fail-fast execution path for unauthorized calls.

public class Main {
    public static void main(String[] args) {
        ApiGateway gateway = new ApiGateway();

        // Configure routes mapping prefixes to target backend services
        gateway.addRoute("/users", "http://user-microservice");
        gateway.addRoute("/orders", "http://order-microservice");

        // Register global filters in execution order
        gateway.addFilter(new LoggingFilter());
        gateway.addFilter(new AuthenticationFilter());

        System.out.println("\n--- Processing Request 1 (Authenticated) ---");
        GatewayRequest req1 = new GatewayRequest("/users/profile", "GET", "");
        req1.addHeader("Authorization", "Bearer valid-secret-token");

        GatewayResponse res1 = gateway.handleRequest(req1);
        System.out.println("Request 1 Status: " + res1.getStatusCode() + " | Body: " + res1.getBody());

        System.out.println("\n--- Processing Request 2 (Unauthorized) ---");
        GatewayRequest req2 = new GatewayRequest("/orders/123", "POST", "{\"total\": 99.9}");
        req2.addHeader("Authorization", "Bearer wrong-token");

        GatewayResponse res2 = gateway.handleRequest(req2);
        System.out.println("Request 2 Status: " + res2.getStatusCode() + " | Body: " + res2.getBody());

        System.out.println("\n--- Processing Request 3 (Invalid route) ---");
        GatewayRequest req3 = new GatewayRequest("/billing/payment", "GET", "");
        GatewayResponse res3 = gateway.handleRequest(req3);
        System.out.println("Request 3 Status: " + res3.getStatusCode() + " | Body: " + res3.getBody());
    }
}

The main() driver configures routes, sets the filter pipeline order, submits matching query payloads, and displays execution details for both successful and aborted routes.

---

Class Diagram

---

Also See

• Most Commonly Asked Low Level Design Interview Questions
• Top 10 Low Level Design Interview Questions