I've been repeatedly asked how our load balancers scale. So, I have decided to write down your options for endlessly scalable, high-performance load balancing.
With standard active/passive HA (High Availability) deployments, options for horizontal expansion are limited. In contrast, active/active HA deployments open up the possibility of N+1 options for scale and increased redundancy capabilities.
However, we are often asked for Active/Active when it is not necessary. And my usual response to this inquiry is:
"What are you doing that makes you think you'll need more than 88Gbps?
How much do you actually need?"
Often, I find that customers are adamant that they need well over 100Gbps, only to launch into production and see an average of 8Gbps and peaks of 15Gbps.
One of our company values is never mis-selling to our customers, so I like to qualify their actual, realistic throughput requirements the first chance I get.
But, we've recently seen a big increase in demand for large scale object storage systems requiring 100+ Gbps, and even Terra Bits of throughput!
So, what does Loadbalancer.org offer to customers when they need it?
How can you scale your ADC infrastructure?
Depending on which load balancing method you desire (overall requirements), there are two ways to achieve throughputs surpassing 100Gbps.
1. Load balance your load balancers, with DSR mode
One of my colleague spoke about this methodology in 2018, to achieve extremely high SSL terminations per second (TPS) using multiple mid-range appliances, in comparison to a single pair of higher-cost systems. Check out the blog here: Load balancing your load balancers for endless scalability.
Here's what load balancing your load balancers looks like:
This blog highlighted the concept of using a front end pair of ADCs to handle just the incoming requests in high performance Layer 4 DSR mode. Load balancing these requests to a cluster of Layer 7 ADCs that could handle the CPU intensive SSL requirements and have almost unlimited return traffic.
Benefits of load balancing load balancers
- Horizontal scaling by design
- Increased total throughput
- Higher combined TPS/SSL
- Significant cost savings
Disadvantages of load balancing load balancers
The main disadvantage is that this can become difficult to manage alone. Still, it is possible with help from automation tools such as Terraform for deployment, the use of the ADC Portal for appliance management, and third-party reporting from software such as Grafana.
It also has the bottle neck of the first load balancing layer, where incommming traffic requests are restricted to 100 Gb/s. However, outbound traffic can be many times that, because we're using out of path DSR mode.
2. Unlimited N+1 throughput, with Global Server Load Balancing
What is N+1? Well, in the context of this blog, N= the minimum number of load balancers needed to achieve the desired sustained system throughput, with +1 being the minimum required level of redundancy.
Usually, two load balancers are needed to achieve high availability. If you don't understand why, check out this article first: The dangers of not having an HA setup for your load balancer as well as your appliances.
But when you need more throughput, TPS, or redundancy coverage (i.e. 3+:1), you will need additional ADCs.
We normally use GSLB for multi-site resilience, and site affinity
It's really important that you keep the amount of high cost WAN traffic down between data centers. This is easily achievable using GSLB site affinity, specifying topology using subnets, this ensures that local traffic stays in the local data center. And when a site goes down, health checks ensure the GSLB will automatically route you to a working data center.
If you're still not sold on using GSLB for resilience, here are the pros and cons:
And for more detail, compare Layer 4, Layer 7, and GSLB load balancing techniques.
But, how do you achieve endless scalability using GSLB?
NetScaler has some great content on this, using Global Server Load Balancing (GSLB) to scale up to 32 hardware load balancers capable of 6.4 Tbps L7 throughput! Check it out here: High performance load balancing. Although, it's a shame that their new Hybrid Cloud Platform license 'only' allows up to 1000Gbps.
It's important to note that sometimes you don't even need to use layer 7 ADCs at all. If you have huge throughput demands but don't specifically need any clever ADC features then you can use GSLB direct-to-node for unlimited scaling and incredbly low cost!
However, let's assume you've discounted this already, and need the full power of GSLB for N+1 Active/Active clusters of layer 7 ADCs.
First, a quick recap on how GSLB works:
Below is a high-level description of how the GSLB operates. It selects a VIP to send the client traffic to, and then the connection goes through the load balancer via the L7 Virtual Service:
Example use case: Scaling with GSLB for a high-performance workload
Now before I get into this section, I'd like to note that achieving above 1Tbps throughput is rare. However, advancements in AI and data lakes mean high-performance computing is becoming a more frequent topic of conversation—all requiring superior performance.
Challenge
For example, we have a customer deployment working with high-performance object storage that requires the following:
- 1/3 redundancy
- 1Tbps throughput
- 33,000 SSL TPS (transactions-per-second)
Solution
To achieve this, we provided them with 18 Enterprise Max appliances and an excellent service engineer!
The diagram above illustrates a three-site topology, with 18 load balancers across 3 data centers.
- Each site has six load balancers (forming GSLB groups), offering a combined throughput of 540Gbps per site.
- The three GSLB groups (the GSLB pool) collectively provide up to 1620Gbps throughput.
- All load balancers can route client traffic to all backup servers, ensuring continuity, even if one goes offline.
- The maximum throughput capacity should therefore only be reduced by 88Gbps if any single ADC goes offline.
Here's how to set up GSLB on a Enterprise Loadbalancer appliance:
For a comprehensive guide to GSLB, check out our guide:
Results
The benefits realized by the customer were:
- A simple solution for horizontal scaling
- An exponential increase in total throughput
- Dynamically allowing for higher TPS/SSL
- More cost-effective for large-scale deployments
- A future-proofed configuration using GSLB
Summary
The bottom line is, we're here to help you increase your load balancing throughput and scale seamlessly as your organization and project requirements grow.
Our scalable solutions for high throughput and SSL requirements offer a future-proof, cost-effective way to meet the demands of modern high-performance workloads, ensuring levels of high availability and resilience that meet the needs you don't yet have, as well as those you already do.