feat(loadbalancer): Add LoadBalancerType Client Side Weighted Round Robin

[altaiezior]

What type of PR is this?

What this PR does / why we need it:
This PR provides addition of new load balancer type client side weighted round robin. This is a new load balancing extension introduced since envoy 1.32

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/load_balancing_policies/client_side_weighted_round_robin/v3/client_side_weighted_round_robin.proto

Which issue(s) this PR fixes:

Fixes #7305

Release Notes: Yes/No

[altaiezior]

@jukie I have added the implementation and also tested it on my local setup

PS: the repo is so easy to contribute everything just works with the docs given on the site :)

[altaiezior]

Also I wanted to know should I include slow start in client wrr?

So the thing is that I have submitted the proposal in grpc-xds grpc/proposal#498 and also in envoy I have got the proto updated.

It is not implemented yet, but I am trying to pick it up this month if my time allows

[altaiezior]

Also I am unsure of how to test this e2e, so I have just included an AI generated e2e test suite.

The challenge here is that we need multiple replicas with each server respond with a specific header containing rps and cpu_utilisation and then the traffic is distributed by calculating the weight (rps / cpu)

I don't know what the current e2e tests allow and if this type of test case is feasible to write

[codecov]

Codecov Report

❌ Patch coverage is 73.49398% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.81%. Comparing base (5c2075b) to head (53f1473).

Files with missing lines	Patch %	Lines
internal/xds/translator/cluster.go	58.53%	16 Missing and 1 partial ⚠️
internal/gatewayapi/backendtrafficpolicy.go	75.00%	2 Missing and 1 partial ⚠️
internal/gatewayapi/clustersettings.go	92.30%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7407   +/-   ##
=======================================
  Coverage   73.81%   73.81%           
=======================================
  Files         241      241           
  Lines       36608    36688   +80     
=======================================
+ Hits        27021    27082   +61     
- Misses       7681     7698   +17     
- Partials     1906     1908    +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[altaiezior]

Also I wanted to know should I include slow start in client wrr?

So the thing is that I have submitted the proposal in grpc-xds grpc/proposal#498 and also in envoy I have got the proto updated.

It is not implemented yet, but I am trying to pick it up this month if my time allows

I have started the implementation of slow_start_config and locality lb config with WRR as well, if possible I would also want to include them in the gatway implementation.

envoyproxy/envoy#41841

[jukie]

We wait to add features here until they've made it into a full envoy release. The flow would be getting this lb support added for 1.7 and if your envoy changes get merged we can add that support to gateway in 1.8.

Let's keep the scope of this PR to what's currently available and we can always include additional features in a follow-up. Are you able to make the suggested changes or can you join the contributors call next week to discuss?

[altaiezior]

Sure, I just paused because the other changes were also approved, but I understand I will make the respective changes as suggested. Will try to complete them by today / tomorrow @jukie

[altaiezior]

@jukie I have made the respective changes

[jukie]

Overall looks good, just a few more comments @anuragagarwal561994! Thanks for adding this and make sure to run gen-check before pushing.

Sorry for the delayed review on this. I'll prioritize helping you with this next week.

[arkodg]

can we test the feature in the e2e ? i.e. have the backend craft a endpoint-load-metrics response header and use that in LB decision making

[altaiezior]

@arkodg this we have discussed in earlier comment that it will take some time to build and can be done seaprately because this requires changes with the current echo application too.

#7407 (comment)

[arkodg]

thanks, imo, lets track this with a GH issue
some inspiration

cat >"${WORKDIR}/backend.py" <<'PY'
#!/usr/bin/env python3
import os
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse, parse_qs

BACKEND_ID = os.environ.get("ORCA_ID", "backend")
DEFAULT_MEM = os.environ.get("ORCA_MEM_UTIL", "0.5")
DEFAULT_CPU = os.environ.get("ORCA_CPU_UTIL", "0.1")
DEFAULT_EPS = os.environ.get("ORCA_EPS", "0.0")
DEFAULT_RPS_FRACTIONAL = os.environ.get("ORCA_RPS_FRACTIONAL", "1.0")

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        qs = parse_qs(urlparse(self.path).query)
        mem = qs.get("mem", [DEFAULT_MEM])[0]
        cpu = qs.get("cpu", [DEFAULT_CPU])[0]
        eps = qs.get("eps", [DEFAULT_EPS])[0]
        rps_fractional = qs.get("rps_fractional", [DEFAULT_RPS_FRACTIONAL])[0]
        # ORCA native HTTP text encoding.
        orca_header = (
            "TEXT "
            f"cpu_utilization={cpu}, "
            f"mem_utilization={mem}, "
            f"eps={eps}, "
            f"rps_fractional={rps_fractional}"
        )

        body = (
            f"{BACKEND_ID} mem={mem} cpu={cpu} eps={eps} "
            f"rps_fractional={rps_fractional}\n"
        )
        self.send_response(200)
        self.send_header("content-type", "text/plain")
        # This is the ORCA header Envoy consumes.
        self.send_header("endpoint-load-metrics", orca_header)
        self.end_headers()
        self.wfile.write(body.encode("utf-8"))

    def log_message(self, fmt, *args):
        # Keep logs readable in the demo.
        now = time.strftime("%H:%M:%S")
        print(f"[{now}] {BACKEND_ID} " + fmt % args)

if __name__ == "__main__":
    port = int(os.environ.get("PORT", "18080"))
    print(f"starting {BACKEND_ID} on {port}")
    HTTPServer(("127.0.0.1", port), Handler).serve_forever()
PY
chmod +x "${WORKDIR}/backend.py"

[arkodg]

hey @altaiezior thanks for patiently addressing all the comments, the PR looks good !
can you address the final lint issue, and should be good to get this in

[altaiezior]

@zirain I am not able to also run the e2e test case on my local because of the above issues, I have made the changes to include the e2e test case for wrr as well. But these seem to be an issue with the master branch itself, let me know once this is fixed so I can also pull the latest version and fix the things in my local too

[arkodg]

hey @altaiezior can you rebase again, looks like there's another conflict :(
@envoyproxy/gateway-maintainers can we prioritize getting this PR in to avoid more conflicts

[jukie]

@altaiezior could you add a release note?

[jukie]

Can you add full error handling for this and the other options?

[jukie]

v is already guaranteed to be non-nil due to the case check

[jukie]

nit: can probably remove this note

[jukie]

Is this implemented? I don't see handling for it. Not opposed to adding this but there's already fields for header management so it could be good enough to mention or include in the docs follow-up.

[arkodg]

hey i had requested for this, to avoid having the user to do this manually

[altaiezior]

Sure @jukie @arkodg let me try and close them by today / tomorrow

[altaiezior]

@jukie I will be able to pick up these changes only by next week as I am travelling and been catching up with my work lately.