As the title suggests, I'm going to take a look at the concept of load balancing a cluster of inexpensive load balancers to scale out SSL capabilities horizontaly. Rather than the typical idea of verticaly scaling up SSL offload capabilities by buying ever-larger appliances with ever more expensive hardware and license requirements.
Why do you sometimes need to scale out SSL TPS?
Because supporting a large number of SSL terminations per second is VERY CPU intensive. However, the ongoing traffic and encryption of the packets doesn't need nearly so much processing power. Our customers sometimes panic when they confuse TPS requirements with connection requirements. I won't bore you with the maths, but believe me, you can have millions of SSL connected users with only 100 TPS!
A crazy customer requirement that got me thinking...
The idea itself came out of a conversation, as so many good ideas do. We were discussing a customer SSL offload requirement that could potentially grow to the point of needing 100,000 TPS or more - ouch!
Why on earth did they need so much raw TPS power?
Long story short it was a finance application with millions of tiny connections to an API with very short lifespans i.e. nothing like normal HTTPS traffic.
The customer shared that to achieve that sort of TPS, they'd been quoted $500,000+ for a highly available pair of capable appliances, including the associated licenses to unlock the hardware - even bigger ouch!!
I wondered: couldn't you just use a large cluster of cheaper load balancers, all being load balanced by a pair of fast Layer 4 load balancers, keeping it transparent? Managing such a beast wouldn't be easy. But what if we used an API or central control node?
So the brainchild was born, for better or worse. As a picture speaks louder than words, I'm suggesting along the lines of this:
Creating a monster
The plan is to use an inexpensive pair of load balancers - using Layer 4 DR mode for maximum throughput and transparency - to load balance some more mid to high-end load balancers. This enables us to handle the SSL offload at a much more affordable price than simply buying a big box.
The idea hinges on your ability to manage this effectively, because it could quickly become cumbersome without an API or some other way to perform maintenance tasks across the whole farm of load balancers. With an API and some scripting knowledge, however, this should be fairly easy to overcome. And guess what? The Loadbalancer Enterprise has both an API & CLI already:
[https://www.loadbalancer.org/blog/how-to-automate-load-balancer-deployments/]
Time to get our DevOps on!
Here is an example simple bash script to leverage the API to perform actions on multiple load balancers at once.
#!/bin/bash
# Were we called with at least one arg?
if [ $# -eq 0 ]; then
echo "Not enough paramters: Usage - '$0 command' "
exit 42
fi
# Space separated list of IP addresses to push the command to
SERVERS="192.168.10.11 192.168.10.12 192.168.10.13 192.168.10.14"
# Pull in all args passed to this script as the required command
COMMAND=$*
for SERVER in ${SERVERS} ; do \
echo ssh root@${SERVER} ${COMMAND}
done
exit 0
Does it work on price? I think it does!
2x Enterprise Flex with 1 yr 24/7 Support = $24,990.00 (USD) Excl. Tax
9x Enterprise Max with 1 yr 24/7 Support = $236,205.00 (USD) Excl. Tax
Total = $261,195 (USD) Excl. Tax
Which could potentially handle ~ 150,000 TPS!
Check it out yourself using our online quote form
When else might this work?
I think the other thing to mention is that scaling SSL isn't the only thing that can be a problem. Scaling L7 transformations can get tough, scaling WAFs can get even tougher and approaching either problem with this method would be just as applicable.
What if you hit 100Gb/s on the layer 4 front end ADCs?
No problem, just use GSLB instead. We often come across insanely large object storage deployments that need thousands of Gbits/s! So rather than using a pair of layer 4 load balancers in front of your load balancing cluster. You simply use a pair of GSLBs infront of as many ADCs as you require, which gives you almost limitless horizontal scalability :-).
Why isn't everyone doing this?
Well, it does involve more work than an "off the shelf" solution. But I think you will find that when you look beyond the typical enterprise customer at some of the largest deployments around, this techique is actually used fairly often. Take Facebook for example:
https://www.bizety.com/2017/01/17/facebook-billion-user-load-balancing/
To try or not to try
Large load balancers with massive SSL offload capability are fine. They may suit some enterprises because they are simple to deploy and maintain.
However, when you have a service with requirements that are growing - or you just chucked out those 2048 bit certs and deployed 4096 only to realise that your big pair of boxes are no longer up to the task - it could be worth doing something different. You may value the extra investment of time at the beginning to have a system that only requires adding more Layer 7 load balancers as you grow.
So should you try it? I think it's fair to say "yes" - this is a great way to tackle the problem of scale.
While it does involve having the internal technical resources capable of implementing and maintaining such a solution, the benefits if you are a company with a technical abundance will be great. You'll be saving a truckload of money while justifying your tech team's salaries. And you get the added technical benefit of having an endlessly scalable load balancer designed from the outset to grow with your needs.