Is there a way to launch gcp nodes with GVNIC enabled?

[flowinh2o]

In order to take advantage of distributed processing on GCP it looks you need to start the instance with GVNIC enabled. In the os I see that the modules are loaded and running but don't think it's working without this enabled. Is this possible.

https://cloud.google.com/compute/docs/gpus/optimize-gpus#high-bandwidth

[concretevitamin]

It appears so. I tried the following patch and could launch a GCP VM with gVNIC, as verified in the console:

--- a/sky/skylet/providers/gcp/config.py
+++ b/sky/skylet/providers/gcp/config.py
@@ -693,6 +693,7 @@ def _configure_subnet(config, compute):
     subnets = _list_subnets(config, compute, filter=f'(name="{usable_vpc_name}")')
     default_subnet = subnets[0]

+    # https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert#request-body
     default_interfaces = [
         {
             "subnetwork": default_subnet["selfLink"],
@@ -702,6 +703,7 @@ def _configure_subnet(config, compute):
                     "type": "ONE_TO_ONE_NAT",
                 }
             ],
+            "nicType": "gVNIC",
         }
     ]

Could you verify if this gives the expected performance? If so, we could look into properly plumbing it through.

[flowinh2o]

Interesting I see that already nicType option defined in that config.py file in the current virtual environment that I am using but when launching it I don't see it enabled on the instance. I was able to verify this by running the following with no output:
gcloud compute instances describe ray-gcp-test-head-09208b0a-compute | grep nicType

For a node that has it enabled it should show "nicType: GVNIC"

I am running skypilot, version 1.0.0-dev0 installed from source.

[concretevitamin]

Have you tried pip install -e ".[gcp]"? Just want to make sure editable -e was used in installation, otherwise local patches may not be picked up.

[flowinh2o]

Sorry for the delay here. I used the -e option and do now see the instance coming up with gVNIC now!

[concretevitamin]

That’s awesome!

[csaroff]

Would love to see this feature as well!

[concretevitamin]

Thanks @csaroff - would you like to try out the one line change to see if it visibly helps with performance? If there's only upsides and no downsides we probably can quickly commit it into main.