feat(streaming): adapt backfill rate limit according to barrier latency

[kwannoel]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

WIP, I want to properly document the shortcomings first.

We implement the barebones tcp congestion control procedure, as indicated here by @chenzl25:

init_rate_limit = 1k

ssthresh: 50k

if rate_limit < ssthresh:
	rate_limit *= 2
else if rate_limit >= ssthresh:
	rate_limit += 1k


(barrier latency > target barrier latency upper bound) ~= Congestion point of network

if barrier latency > target barrier latency upper bound
	ssthresh = rate_limit / 2
	rate_limit = init_rate_limit

• Show grafana monitoring results.
• Add more scenarios: Join with Agg after, UDF high latency.
• Include more description on the alternative solutions.
• Add frontend support for setting adaptive rate limit.

We have a few test scenarios:

Amplification is medium x100

Backpressure can handle this case well. Adaptive Rate Limit does not have any discernible effect.

Amplification is high x500

Backpressure cannot handle this case, barrier latency fluctuates stably, but is relatively high.
Adaptive Rate Limit can sort of handle this case. The Rate Limit becomes 1, and barrier latency keeps low.
But ideally it can maintain at a higher rate instead of just 1..

Amplification is extremely high x10000

TODO

High barrier latency outside of the current job, in a different parallelism

TODO

Improvements

Perhaps fetch metrics from meta directly?

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added test labels as necessary. See details.
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

• My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

[kwannoel]

The default adaptive rate limit strategy doesn't work well when I test it with the high amplification case.

1. The rate limit increases too fast (since it scales rate limit exponentially 2x).
2. We cannot simply get "current barrier latency". The previous barrier may not have been collected yet.
3. Even if we have threshold = 0, and we do linear increments, this linear increment of rate limit can also be too large (1000).

Then we end up having too high a rate limit due to 1 and 2. At that point, when adaptive rate limit kicks in, it's too slow as well, since its barrier latency is high (as rate limit is too high).
After we return to normalcy, threshold becomes 0, and the rate limit fluctuates between 1000, and the initial rate limit (1). At 1000, the rate limit is too high as well in amplification cases. Leading to high barrier latency still.