Scaling HAProxy on AKS for Billions of Transactions with Dynamic Autoscaling and Token Management

At our scale, we needed to handle billions of transactions efficiently - routing across multiple environments and internal data centers - while maintaining reliability, security, and dynamic control. To achieve this, we chose HAProxy, a proven, high-performance load balancer known for its lightweight footprint, flexibility, and ability to handle massive concurrency with minimal overhead.

This blog walks you through how we deployed HAProxy in high-availability (HA) mode on Azure Kubernetes Service (AKS) -complete with autoscaling powered by KEDA, seamless integration with Azure Application Gateway, and dynamic token management using Azure File Share, all designed to deliver scalable, self-hosted traffic routing at enterprise scale.

ALSO READ: How We Took Our Kubernetes Autoscaling from Basic to Advanced Mode with Istio Metrics

The Challenge: Scaling HAProxy for Real-World Production Complexities

When running HAProxy on Kubernetes in a production-grade setup, we often face three major challenges:

Scalability

Handling spikes in request traffic without manual intervention.

Dynamic Configuration

Being able to update token or authentication files (used for request validation) dynamically, without restarting or redeploying HAProxy pods.

Integration with Azure Networking

Running HAProxy behind Azure Application Gateway (App Gateway) while maintaining performance, visibility, and security.

Our goal was to create an auto-scaling, token-aware, and persistent HAProxy setup that integrates seamlessly with Azure Application Gateway and KEDA.

Solution Overview

Here’s how the setup looks at a high level:

• HAProxy runs in HA mode on an AKS cluster.
• Application Gateway sits in front, handling SSL termination and forwarding traffic to HAProxy.
• KEDA monitors request counts (via App Gateway metrics) and auto-scales HAProxy pods accordingly.
• Azure File Share provides persistent storage for token files, allowing dynamic updates without redeployment.

Step-By-Step Implementation

1. Create an AKS Cluster with Autoscaling

• Create an AKS cluster with autoscaling enabled.
• Assign system-managed identity of the cluster Reader and Network Contributor roles to the resource group where the VNet/subnet resides.

Create a Service Principal (SP) with Monitoring Reader role for App Gateway metrics.

2. Prepare the Kubernetes Environment

Login to your cluster and set up a namespace for HAProxy:

kubectl create ns haproxy
kubectl config set-context --current --namespace=haproxy

3. Configure Persistent Storage for Token Files

We’ll store token-ID pairs in Azure File Share so they persist even if HAProxy pods restart.

Create the Azure File Share

1. Create a Storage Account and a File Share named haproxy-allowed-ids with a 2 GB quota.

Mount it on your build server:

sudo mkdir -p /mnt/azurefiles/allowed_ids

sudo mount -t cifs //yourstorageaccount.file.core.windows.net/allowed-ids-share /mnt/azurefiles/allowed_ids \
-o vers=3.0,username=yourstorageaccount,password=yourstoragekey,dir_mode=0777,file_mode=0777,serverino

2. Clone the git repo and run arrange_files.sh script

git clone https://gist.github.com/108f3c3444180d9d4a95d3ac7d8dc338.git haproxy_repo

cd haproxy_repo

bash  arrange_files.sh

3. Base64 encode your storage credentials:

 echo -n "KEY" | base64

4. Add them to fileshare/azure_storage_secret.yaml and apply:

kubectl apply -f fileshare/azure_storage_secret.yaml

kubectl apply -f fileshare/pv.yaml
kubectl apply -f fileshare/pvc.yaml

4. Create Token Files Dynamically

Inside the mounted directory:

cd /mnt/azurefiles/allowed_ids

echo "Client-ID" >> token_value.txt

This token file is now automatically visible to all HAProxy pods (thanks to the Azure File Share mount).

5. Deploy HAProxy with Helm

Create a secret for the thirdparty-api-auth token:

kubectl create secret generic thirdparty-api-auth-token-secret \
  --from-literal=token="<THIRDPARTY_API_AUTH_TOKEN>"

1. Deploy HAProxy:

 helm install haproxy haproxy

2. Verify pod logs and ensure token files are mounted:

kubectl logs <haproxy-pod-name>

kubectl exec -it <haproxy-pod-name> -- ls /etc/haproxy/allowed_ids

6. Integrate KEDA for Autoscaling

KEDA will monitor App Gateway metrics (like request count) and automatically adjust HAProxy replicas.

Encode Service Principal credentials:

echo -n "CLIENTID" | base64

echo -n "CLIENTSECRET" | base64

1. Set them in:

kubectl apply -f keda/monitor_secret.yaml

kubectl apply -f keda/az_monitor_triggerauth.yaml

2. Update keda.yaml with:

• Application Gateway Resource URI
• Tenant ID
• Subscription ID
• Resource Group Name

Apply: 

kubectl apply -f keda/keda.yaml

7. Configure Azure Application Gateway

Create the following components:

• Listener on port 443 (with SSL certificate)
• Backend pool → HAProxy service external IP
• Backend settings on port 80
• Health probe that returns 200 OK

Example health endpoint:

https://<haproxy-lb-ip>/health

Finally, map your DNS record to the App Gateway frontend IP.

Example validation call:

curl --location 'https://abc.example.com/wcc/1234567890/message_templates' \
--header 'Content-Type: application/json' \
--header 'Authorization: AUTHTOKEN' \
--header 'Client-ID: SECRET

HAProxy Configuration Highlights

Below are key parts of the HAProxy configuration enabling dynamic token validation and routing.

global
  log stdout format raw daemon
  maxconn 50000
  lua-load /etc/haproxy/validate_token_id.lua
  lua-load /etc/haproxy/read_token.lua
  nbthread 4
  daemon

defaults
  mode http
  option httplog
  timeout connect 5000
  timeout client  50000
  timeout server  50000

frontend http_front
  bind *:8080
  http-request set-var(txn.token) req.fhdr(Authorization)
  http-request set-var(txn.id) req.fhdr(Client-ID)
  http-request set-var(txn.is_token_valid) lua.validate_token
  http-request set-var(txn.is_id_valid) lua.validate_id_for_token

  use_backend health_check_backend if { path_beg /health }
  use_backend token_backend if { path /api/token } { var(txn.is_id_valid) -m str true }
  use_backend backend-wa if { var(txn.is_token_valid) -m str true } { var(txn.is_id_valid) -m str true }

backend health_check_backend
  mode http
  http-request return status 200 content-type "text/plain" string "OK"

Key Benefits of This Setup

Conclusion

With this setup, HAProxy becomes a highly available, auto-scaling, and dynamically configurable API gateway layer for your applications on AKS.

By leveraging Azure Application Gateway, KEDA, and Azure Files, we get:

• Scalable performance,
• Zero-downtime token updates, and
• Seamless integration into Azure’s cloud-native ecosystem.

This architecture forms a robust foundation for secure, scalable, and maintainable ingress control in any enterprise Kubernetes environment.