This post was co-authored by Vemula Srimithra, NetScaler Senior Software Engineer, and Aman Chaudhary, NetScaler Senior Manager of Engineering
Load balancing for improving your CPU utilization by 5x
NetScaler’s new jumptable-assisted ring hash (JARH) load balancing algorithm can improve your CPU utilization by up to 5x and save on operational costs. We created the JARH algorithm based on the principle of consistent hashing, and it offers stateless persistence for your applications, giving you improved performance with lower operational costs.
Applications and web services come in two forms: stateful or stateless. Stateful transactions are characterized by an application’s intent to store information and use it to make decisions when a client interacts with it — for example, e-commerce shoppers can close their browsers and later resume shopping and check out. Statelessness is ephemeral, often lasting a single transaction. Each new transaction begins without knowledge of past transactions or past data — for example, downloading a file from a public website.
NetScaler can help you load balance applications in a stateful or a stateless way — whichever best suits each deployment. In this blog post, we’ll look at a brief overview of consistent hashing and compare NetScaler’s JARH with another popular algorithm used by F5.
NetScaler consistent hashing algorithm: JARH
With the growth in microservices, stateless application deployments are common. Services are lightweight and can spawn or re-spawn on any node in a distributed system. While there are ample methods to maintain state in microservices environments, there are often challenges such as defining persistent and scalable storage, handling a high number of sticky transactions, and dealing with connection or network failures.
Consistent hashing has long been the primary way to solve these problems. But current load-balancing options face issues such as maintaining uniform load distribution, consistency during failure events, and software performance challenges.
The highly compute-efficient JARH load-balancing algorithm achieves near-perfect consistency when selecting backend application services. It also provides uniform traffic distribution while achieving stateless persistence in a fault-tolerant manner. This ensures that no single backend server gets overloaded, while enabling organizations to save on costs by deploying fewer application delivery controllers (ADCs).
JARH vs. CARP comparing NetScaler and F5 implementations
Many vendors adopt the cache array routing protocol (CARP) for load-balancing consistency in their traffic management solutions. NetScaler has used CARP in the past, but tests have shown JARH to be superior in terms of CPU utilization.
To compare, we looked at the new JARH algorithm and F5’s implementation of CARP using equivalent virtual devices on AWS. We tested each system with 5,000 requests/second and 10,000 requests/second while load balancing 100, 1,000 and 5,000 backend servers. We measured the average CPU utilization (%) of the system for each combination.
As you can see, the JARH algorithm was far more efficient than CARP. In both tests (5,000 and 10,000 requests per second), CPU usage was lower at the outset and, unlike CARP, NetScaler’s JARH algorithm didn’t exhibit a significant increase in CPU utilization as the number of backend servers increased.
As shown in Figure 2, the F5 ADC using the CARP protocol saturates the CPU (82 percent) but the NetScaler ADC using JARH only registers 16 percent utilization.
So, at the low end, with 5,000 requests/second and 100 backend servers, NetScaler performs 3x better. At the high end, with 10,000 requests/second and 5,000 backend servers, NetScaler performs more than 5x more efficiently than the system running CARP.
This means that a single NetScaler ADC can process the same amount of traffic but still have compute capacity left over. So, what does this mean for your production environments?
The number of requests per second processed by applications varies widely. Popular websites such as Wikipedia, Twitter, and GitHub process tens of thousands to hundreds of thousands of requests per second. Moderately sized applications may process a few hundred to a few thousand requests per second. A single NetScaler ADC running JARH can serve many different workloads.
When a small number of backend servers are used for an application, the NetScaler ADC using JARH consumes up to 66 percent fewer CPU resources than ADCs from other vendors. This leaves plenty of additional compute for other ADC functionality, such as:
- Accepting more requests during peak hours or holiday seasons without having to scale up or scale down instances
- Increasing the number of backend servers without worrying about capacity issues
- Accommodating additional compute intensive features and capabilities on the same NetScaler ADC platform
JARH delivers cost savings for distributed applications
Companies with larger, more distributed applications that have several hundred to a few thousand backend servers can experience the benefits of JARH to its full capacity. While F5 BIG-IP or a similar product may be overwhelmed by an increase in the number of backend servers, a single NetScaler ADC can efficiently handle these changes with ease. This efficiency translates into significant cost savings as a single NetScaler ADC can do the job of multiple other products.
With application workloads becoming more dynamic, NetScaler ADCs are built to adapt to the characteristics of the applications they front. They can efficiently handle a broad range of workloads with relative ease. This enables you to ensure that the application end-user experience remains unaffected by the application’s scale and ephemeral nature while also delivering a high-performing application in a cost-effective manner.
Learn more about NetScaler load-balancing methods
To learn more about load-balancing methods based on hashes, check out documentation on hashing methods on NetScaler.
For a comparison of additional NetScaler and F5 capabilities, see NetScaler vs F5.