Distributed System Architecture with Kong & Kubernetes

Overview

At Floures Technologies, we've been designing a Distributed System Design (DSD) platform — a microservices architecture with Kong API Gateway, 7 backend services, and supporting infrastructure running on Kubernetes (Docker Desktop).

This post walks through the full system architecture, design decisions, and integration patterns we've adopted.

Architecture Overview

The system consists of three layers:

1. Kong API Gateway — single entry point for all external traffic
2. Backend Services — 7 Go microservices with ClusterIP (internal only)
3. Infrastructure — PostgreSQL, MongoDB, Kafka, Redis via Docker Compose

Components

Kong API Gateway

Single entry point for all external traffic. Deployed as Kong Ingress Controller (KIC) in K8s.

Plugins:

• CORS — cross-origin resource sharing
• Rate Limiting — per-consumer, per-route
• JWT Authentication — token validation
• Prometheus — metrics export
• Request/Response Logging

Services

All services are Go 1.22, deployed as K8s Deployments with ClusterIP (internal only).

Note: Order Service writes to PostgreSQL only. MongoDB read model is populated via CDC (Debezium → Kafka → CDC Consumer).

Infrastructure

Managed via Docker Compose, running alongside K8s.

Request Flow

Client
  → localhost:8000/api/inventory/items
  → Kong Proxy
    → Plugin chain: JWT Auth → Rate Limit → CORS → Logging
    → Route match: /api/inventory/* → inventory-service
  → inventory-service:8080/items (ClusterIP, internal)
  → Response back through Kong

Route Mapping

Service Integration Patterns

Order → Product Catalog

Order Service calls Product Catalog Service during order creation to snapshot product data:

GET /api/products/{sku}
Response: { sku, name, description, price, image_url, inventory_count }

Pattern: Request-Response (Synchronous) Rationale: Product snapshot captured at order time for historical accuracy

Order → Promotion

Order Service calls Promotion Service to validate and apply discounts:

1. POST /api/promotions/validate
   { promotion_code, customer_id, items[] }
   → { valid, discount_amount, promotion_id }

2. POST /api/promotions/usage
   { promotion_id, order_id, customer_id }
   → { success, usage_count }

Pattern: Request-Response (Synchronous) Rationale: Promotion validation must complete before order confirmation

Postgres → MongoDB (CDC Pipeline)

Order data synchronization from Postgres (write model) to MongoDB (read model) using Change Data Capture:

Order Service
    ↓ writes
PostgreSQL (wal_level=logical)
    ↓ WAL (Write-Ahead Log)
Debezium Connector (Kafka Connect :8083)
    ↓ captures INSERT/UPDATE/DELETE
Kafka Topics:
    ├─ dsd.public.orders
    ├─ dsd.public.order_items
    └─ dsd.public.order_promotions
    ↓ consumed by
CDC Consumer (goroutine in order-service)
    ↓ denormalizes & upserts
MongoDB (order_views collection)

Pattern: Change Data Capture (Asynchronous, Eventual Consistency) Latency: ~100ms-1s from Postgres write to MongoDB read availability

Design Decisions

Why Namespace Separation?

Kong runs in its own kong namespace, separate from application namespaces (dsd-dev, dsd-staging, dsd-prod).

Rationale:

1. Cluster-Wide Scope — Kong routes traffic to services across all namespaces. One Kong instance serves multiple environments.
2. Lifecycle Independence — Platform team manages Kong upgrades independently from app deployments.
3. RBAC Separation — Platform/ops team gets admin access to kong namespace; dev team gets access to dsd-* namespaces only.
4. Resource Isolation — Kong gets dedicated CPU/memory quotas separate from app namespaces.

Kubernetes Cluster (Docker Desktop)
├── kube-system          ← K8s core components
├── kong                 ← API Gateway (platform layer)
├── dsd-dev              ← Development environment
├── dsd-staging          ← Staging environment
└── dsd-prod             ← Production environment

Why ClusterIP for All Services?

All backend services use ClusterIP (internal-only), not NodePort or LoadBalancer.

1. Security — Services not directly exposed. Single entry point (Kong) simplifies the security model.
2. Centralized Traffic Management — All policies (CORS, JWT, rate limits) applied at Kong. No need to duplicate auth logic.
3. Service Discovery — K8s DNS resolves inventory-service:8080 within the cluster. No external load balancer per service.
4. Cost Efficiency — One LoadBalancer (Kong) vs. 7 LoadBalancers. Matters significantly in cloud environments.

Why Docker Compose for Infrastructure?

Databases and messaging run in Docker Compose, not K8s.

1. Simplicity — Easier to manage stateful workloads outside K8s. docker compose down -v clears everything.
2. Resource Efficiency — Docker Desktop K8s has limited resources. Running 7 services + Kong + all databases in K8s = resource constrained.
3. Production Parity — In production, you'd use managed services (RDS, DocumentDB, MSK, ElastiCache), not self-hosted. Docker Compose simulates "external managed services."
4. Faster Iteration — Restart Postgres without restarting K8s pods. Clear separation of concerns.

Technology Stack

Service Dependency Map

Client
  ↓
Kong API Gateway
  ↓
  ├─→ inventory-service → PostgreSQL, Redis
  ├─→ order-service → PostgreSQL, MongoDB, Kafka
  │     ├─→ product-catalog-service (snapshot)
  │     ├─→ promotion-service (validate)
  │     └─→ tax-service (external API)
  ├─→ payment-service → PostgreSQL
  ├─→ notification-service → Kafka
  ├─→ query-service → MongoDB
  ├─→ product-catalog-service → PostgreSQL, Redis
  └─→ promotion-service → PostgreSQL

Implementation Status

What's Next

The current focus is Order Service implementation with the CQRS pattern — writing to PostgreSQL and syncing to MongoDB via the CDC pipeline. This is the most complex service in the system, with synchronous dependencies on Product Catalog, Promotion, and Tax services.

Stay tuned for the follow-up post on CQRS implementation patterns with Go and Kafka.