Benchmarking a new load balancing algorithm

[Dim131]

What should we add or change to make your life better?

We would like to benchmark a simple variant of the Two-Choice load balancing algorithm on YARP. Is there a recommended benchmark or a typical workload that you use?

The variant is the following: For each request,

1. Allocate to the least loaded of two randomly chosen servers (i.e., like Two-Choice) with probability β.
2. Otherwise, allocate to a random server.

Somewhat surprisingly we recently demonstrated in theory and simulations (paper, slides, visualisation) that for suitably chosen β, this outperforms Two-Choice when there is outdated information (RTT is large, there are multiple allocators) or there is noise. The main reason for this is that it allocates to lesser loaded servers less aggressively than Two-Choice, so that it spreads the load more evenly (see figure below).

We would like to see whether this result also implies improvements in real-world load balancers, including YARP. For the evaluation purposes, I have drafted a very simple implementation (see here) and was wondering whether you could give me some pointers for setting up and running a benchmark representative of a typical workload.

Kind regards,
Dimitris

P.S. I have reached out to a few other open source load balancers.

[adityamandaleeka]

@Dim131 this looks really promising, thanks for sharing your research!

I'm not sure how much the existing performance benchmarks we have would be useful for testing this kind of change, to be honest. But since you mentioned you're interested in setting up and running a benchmark, I think that would help us feel confident about the change.

Here are some pointers for the performance benchmarking we have today for YARP:

• Crank, which is the infrastructure used by most of the ASP.NET projects for benchmarking: https://github.com/dotnet/crank
  • In particular, the documentation here is useful for understanding how everything works: https://github.com/dotnet/crank/tree/main/docs
• YML description of the YARP benchmarks: https://github.com/aspnet/Benchmarks/blob/main/scenarios/proxy.benchmarks.yml
  • This is the stuff that describes to crank what we want to run
• Results dashboard: https://aka.ms/aspnet/benchmarks
  • Click on where it says "x of 20" at the bottom, and you'll get a menu of options, select "Proxies", which is page 13 of 20.

[Dim131]

Hi @adityamandaleeka,
Thank you very much for the detailed reply. I will try to run the experiments over the next few weeks and i will probably come back with a few more questions.

For now, I am quickly going through the yml specification for the benchmark. Do we need a 28-core sever or is this simulated? Also should the clients be on the same machine as the server?