Written by sumitbhoyar

API Gateway

An API Gateway is a reverse proxy that manages and routes API requests between clients and backend services.

The Big Picture: Where API Gateway Fits

Modern applications rarely run as single monolithic programs. They consist of dozens or hundreds of independent services—user management, payments, inventory, recommendations—each with its own API. Without a gateway, clients must know the location of every service, handle authentication separately for each, and manage complex communication patterns. This creates tight coupling and security risks.

An API Gateway sits at the edge of your system as the single entry point for all external requests. It acts as a concierge: receiving requests, verifying credentials, determining where each request should go, and returning the response. This separation allows backend services to focus on business logic while the gateway handles cross-cutting concerns like security, traffic management, and protocol translation.

In a typical e-commerce platform, the gateway receives requests from web browsers, mobile apps, and partner systems. It routes them to appropriate microservices, enforces rate limits to prevent abuse, and aggregates data from multiple services before returning a unified response to the client.

Core Concepts Explained

Single Entry Point: All external traffic flows through one controlled access point. This simplifies client development—your frontend teams only need to know the gateway’s address, not the location of every backend service.

Intelligent Routing: The gateway examines each request—its URL path, HTTP method, headers—and maps it to the correct service. A request to /api/users/profile goes to the user service, while /api/products/search routes to the catalog service.

Cross-Cutting Concerns: These are responsibilities shared by multiple services. The gateway centralizes them:

  • Authentication: Verifies who is making the request using API keys, JWT tokens, or OAuth
  • Authorization: Checks if the authenticated user has permission for the requested action
  • Rate Limiting: Prevents abuse by restricting how many requests a client can make per minute
  • Logging: Records all requests for monitoring and debugging
  • Caching: Stores responses to reduce backend load and improve latency

Protocol Translation: Backend services might use different communication protocols—some speak HTTP/REST, others use gRPC, GraphQL, or WebSocket. The gateway translates client requests into the appropriate protocol for each service, then translates responses back to a format the client understands.

Request Aggregation: A single user action might require data from multiple services. A mobile app displaying an order summary needs user details, order status, and shipping information. Instead of making three separate requests, the client calls the gateway once. The gateway fans out to multiple services, collects the responses, and returns a single aggregated result.

Backend for Frontend (BFF): Different clients need different data. A mobile app requires lightweight responses with minimal fields, while a web admin panel needs comprehensive data. The gateway can expose tailored endpoints for each client type, optimizing the data shape and reducing payload size.

System Integration: How Everything Connects

API Gateway lives between your external clients and internal infrastructure. It doesn’t replace your services—it orchestrates them.

textExternal Clients (Web, Mobile, IoT, Partners)
         |
         | (HTTPS requests)
         v
    ┌─────────────────┐
    │  API Gateway    │
    │  (Edge Layer)   │
    └─────────────────┘
         |
         | (Routes & transforms)
         v
    ┌─────────────────┐
    │  Load Balancer  │ (Distributes traffic)
    └─────────────────┘
         |
         | (Health-checked routes)
         v
    ┌─────────┬─────────┬─────────┐
    │Service A│Service B│Service C│ (Microservices)
    └─────────┴─────────┴─────────┘
         |         |         |
         v         v         v
    ┌─────────┬─────────┬─────────┐
    │Database │  Cache  │External │
    │   (A)   │  (B)    │   API   │
    └─────────┴─────────┴─────────┘

The gateway integrates with identity providers for authentication, secret management systems for storing API keys, monitoring tools for observability, and configuration management for dynamic routing rules. It reads from distributed caches to serve frequently accessed data and writes audit logs to centralized logging systems.

Request Flow: Step-by-Step Journey

textClient sends request
    ↓
Gateway receives request at public IP
    ↓
Gateway extracts authentication token from header
    ↓
Gateway validates token with identity provider
    ↓
If invalid → Reject with 401 Unauthorized
    ↓
If valid → Apply rate limiting check
    ↓
If rate limit exceeded → Reject with 429 Too Many Requests
    ↓
If under limit → Parse URL path to determine target service
    ↓
Gateway transforms request (adds headers, converts protocol)
    ↓
Gateway forwards to internal load balancer
    ↓
Load balancer routes to healthy service instance
    ↓
Service processes request and returns response
    ↓
Gateway receives response
    ↓
Gateway transforms response if needed
    ↓
Gateway returns response to client

Common Use Cases

Mobile App Backend: A retail mobile app displays a product details screen showing item information, inventory status, user reviews, and personalized recommendations. Without a gateway, the app would make four separate network calls, draining battery and increasing latency. The gateway exposes a single endpoint /api/products/{id}/details that aggregates data from catalog, inventory, reviews, and recommendation services. The app makes one call, the gateway handles the complexity, and the user experiences faster load times.

Microservices Security Gateway: An e-commerce platform has user-facing services (product catalog, search) and sensitive services (payment processing, order management). The gateway enforces authentication uniformly—every request must carry a valid JWT token. It applies different rate limits: 100 requests/minute for product browsing, but only 10 requests/minute for order placement to prevent fraud. Internal services trust requests coming from the gateway and don’t implement authentication themselves, reducing code duplication and security audit surface area.

Partner API Management: A logistics company exposes APIs to shipping partners. Each partner has different capabilities and SLA requirements. The gateway exposes versioned endpoints (/api/v1/shipments, /api/v2/shipments) and routes partners to appropriate versions based on their API key. It applies partner-specific rate limits—large partners get 10,000 requests/hour, small partners get 1,000. It transforms data formats: partners send XML, but internal services use JSON. The gateway handles conversion transparently. Detailed logging tracks every partner request for billing and compliance.

Trade-offs and Limitations

Single Point of Failure: If your gateway goes down, all external access stops. This risk requires high-availability deployment—multiple gateway instances across different availability zones with automatic failover. The gateway becomes a critical infrastructure component that must be monitored and maintained carefully.

Added Latency: Every request passes through an additional network hop. Modern gateways add 1-5 milliseconds, but poor configuration—excessive logging, complex transformations, or synchronous calls to external systems during request processing—can add hundreds of milliseconds. Caching and asynchronous processing mitigate this, but the overhead remains.

Complexity and Operational Overhead: The gateway introduces a new component requiring configuration management, monitoring, upgrades, and security patching. Teams must learn gateway-specific concepts and tools. In small systems with 2-3 services, this overhead may exceed the benefits.

Cost: Managed gateway services charge per million requests. High-traffic applications can incur significant monthly costs. Self-hosted gateways require infrastructure and engineering time. For low-traffic applications, a simpler reverse proxy like NGINX might be more cost-effective.

When NOT to Use an API Gateway:

  • Simple monolithic applications with one or two services—the complexity isn’t justified
  • Internal-only services that don’t face external clients
  • Small microservices architectures (3-5 services) where direct service-to-client communication is manageable
  • When you already have a service mesh handling east-west traffic and don’t need additional north-south complexity

Common Mistakes:

  • Business logic in the gateway: Embedding order calculation or inventory checks makes the gateway a “mini-monolith” that’s hard to maintain
  • Skipping health checks: Not monitoring backend service health causes the gateway to route requests to failed instances
  • Inadequate rate limiting: Setting limits too high provides no protection; too low blocks legitimate traffic
  • Poor error handling: Returning generic 500 errors instead of meaningful messages makes debugging difficult
  • One gateway for everything: Using a single gateway for public APIs, internal tools, and partner integrations creates configuration sprawl

How Technical Architects Should Think About API Gateway

Decision Framework:

Start with simplicity. If you have a monolith or fewer than five services, begin with a basic reverse proxy (NGINX, HAProxy). Add gateway capabilities only when you feel pain—authentication logic duplicated across services, clients overwhelmed by service discovery, or need for sophisticated rate limiting.

Evaluating Need: Ask these questions:

  • Do multiple client types (web, mobile, partners) access your services differently? → Consider BFF pattern
  • Are you repeating authentication, logging, or rate limiting in each service? → Centralize in gateway
  • Do user actions require data from multiple services? → Use aggregation
  • Are services written in different languages or protocols? → Need protocol translation

Sizing and Scaling: The gateway scales horizontally like any stateless service. Plan for 2-3 instances minimum for high availability. Capacity planning: if your services handle 10,000 requests/second, your gateway cluster should handle 15,000 to account for aggregation overhead. Monitor CPU (TLS encryption is expensive), memory (caching), and network throughput.

Gateway vs. Load Balancer vs. Service Mesh: Understand the boundaries:

  • Load Balancer: Distributes traffic across identical servers (layer 4 or 7). Use it in front of your gateway for basic traffic distribution and TLS termination.
  • API Gateway: Manages north-south traffic (external clients to services) with API-specific intelligence—authentication, routing, aggregation.
  • Service Mesh: Manages east-west traffic (service-to-service) inside your cluster with mutual TLS, retries, and fine-grained observability.

They complement, not replace, each other. A common pattern: Internet → Load Balancer → API Gateway → Service Mesh → Services.

Vendor Selection: Choose between managed services (AWS API Gateway, Azure API Management) and self-hosted (Kong, Ambassador, Gloo). Managed services reduce operational burden but may cause vendor lock-in and higher costs at scale. Self-hosted gives control but requires platform engineering expertise. For cloud-native architectures, prefer gateways built on Envoy proxy for better Kubernetes integration.

Design for Evolution: Start with a thin gateway focused on routing and authentication. Add capabilities incrementally as needs arise. Keep configuration in version-controlled files (GitOps). Design your gateway to be replaceable—avoid proprietary features that lock you in. Use open standards: OpenAPI for documentation, OAuth 2.0 for authentication, standard HTTP headers.

Monitoring and Alerting: Treat gateway metrics as critical infrastructure indicators:

  • Request rate, error rate, latency (the “golden signals”)
  • Rate limiting triggers (alerts when legitimate traffic is throttled)
  • Authentication failures (potential attacks)
  • Backend service health (circuit breaker activations)

Set alerts for gateway downtime—this is a pager-worthy event.

The gateway should feel invisible when working correctly. Clients receive fast, secure responses. Backend teams focus on business features, not authentication libraries. As an architect, your goal is making the gateway so reliable and easy to use that teams forget it exists—until they need its powerful capabilities.

Leave a comment