Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download & use versioned images #1526

Merged
merged 3 commits into from
May 10, 2024
Merged

Download & use versioned images #1526

merged 3 commits into from
May 10, 2024

Conversation

pykello
Copy link
Contributor

@pykello pykello commented May 1, 2024

In #1524 we changed rhizome to be able to download versioned boot images. In this change, we actually use that infrastructure.

To download a versioned boot image, one can do:

> vmh.download_boot_image("ubuntu-jammy", version: "20240319")

After this has been done, all VMs on that host which want to use the ubuntu-jammy image will use the latest boot image.

This allows us to be able to download new image versions without impacting previous VMs at all. Previous VMs will continue to use the older images.

As follow up items:

  • VmHost setup will include downloading some default images (e.g. ubuntu-jammy) with explicit. vm_setup won't download the images automatically.
  • We will enforce to use versioned images always
  • Add a program to remove unused images

@pykello pykello force-pushed the pykello/image_versioning branch 2 times, most recently from ec842b8 to 4b876e7 Compare May 2, 2024 02:55
@pykello pykello changed the title Change Prog::DownloadBootImage to download versioned images Download & use veresioned images May 2, 2024
@pykello pykello force-pushed the pykello/image_versioning_rhizome branch 3 times, most recently from d54aa7a to cd545b0 Compare May 7, 2024 17:28
Base automatically changed from pykello/image_versioning_rhizome to main May 7, 2024 19:34
@pykello pykello force-pushed the pykello/image_versioning branch 2 times, most recently from 7808104 to c4c45ca Compare May 8, 2024 03:25
@pykello pykello marked this pull request as ready for review May 8, 2024 03:25
@pykello pykello requested review from enescakir and byucesoy May 8, 2024 03:25
prog/vm/nexus.rb Outdated Show resolved Hide resolved
@pykello
Copy link
Contributor Author

pykello commented May 8, 2024

adding @bsatzger for allocator changes

migrate/20240503_boot_image.rb Outdated Show resolved Hide resolved
migrate/20240503_boot_image.rb Outdated Show resolved Hide resolved
prog/download_boot_image.rb Outdated Show resolved Hide resolved
model/vm_host.rb Outdated Show resolved Hide resolved
@pykello pykello requested a review from enescakir May 8, 2024 21:48
@pykello pykello changed the title Download & use veresioned images Download & use versioned images May 8, 2024
Previously, we ran LearnStorage after an image was downloaded to update
StorageDevice::available_storage_gib of storage devices on the host.

However, this caused problems when there were existing storage volumes
on the host: A storage volume's file can be sparse, in which case `df`
will report a lower occupied space for it. This causes problems down the
road.

This PR changes that so we instead deduct the downloaded image size from
available_storage_gib.
Copy link
Member

@enescakir enescakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After all hosts have been associated with boot images, we'll force the allocator to select a host with this image, correct?

@pykello
Copy link
Contributor Author

pykello commented May 9, 2024

After all hosts have been associated with boot images, we'll force the allocator to select a host with this image, correct?

yes

@pykello pykello force-pushed the pykello/image_versioning branch 3 times, most recently from c8f4f5b to 6b43e9a Compare May 9, 2024 23:20
In #1524 we changed rhizome to be able to download versioned boot
images. In this change, we actually use that infrastructure.

To download a versioned boot image, one can do:

```
> vmh.download_boot_image("ubuntu-jammy", version: "20240319")
```

After this has been done, all VMs on that host which want to use the
`ubuntu-jammy` image will use the latest boot image.

This allows us to be able to download new image versions without
impacting previous VMs at all. Previous VMs will continue to use the
older images.

As follow up items:
* VmHost setup will include downloading some default images (e.g.
 `ubuntu-jammy`) with explicit. vm_setup won't download the images
  automatically.
* We will enforce to use versioned images always
* Add a program to remove unused images
@pykello pykello merged commit 8c6c6af into main May 10, 2024
7 checks passed
@pykello pykello deleted the pykello/image_versioning branch May 10, 2024 02:08
@github-actions github-actions bot locked and limited conversation to collaborators May 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants