Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HelmReleases are Upgraded Without Any Valid Trigger #4798

Open
1 task done
mhamzarazzaq97 opened this issue May 16, 2024 · 3 comments
Open
1 task done

HelmReleases are Upgraded Without Any Valid Trigger #4798

mhamzarazzaq97 opened this issue May 16, 2024 · 3 comments

Comments

@mhamzarazzaq97
Copy link

mhamzarazzaq97 commented May 16, 2024

Describe the bug

I have installed multiple HelmRelease Objects along with HelmRepository, GitRepository & Kustomization Objects. Few helm releases are getting upgraded without any reasons. Helm releases upgrade should be triggered only if there is a change in GitOps repository. There is no change in values of the helm releases, no commits in gitops repository, no manual reconcile or upgrade. I am not able to understand what is triggering helm controller to upgrade the helm releases.

Steps to reproduce

This behavior is random. For some helm releases it never happens, but for some of the releases it happens more often. One thing I observed is that, there is one HelmRelease with a pod exiting with OMM killed many times (once in an hour), it is upgrading that particular helm release very often. Screenshot is attached.

Expected behavior

HelmReleases should be upgraded only when there is some change in GitOps repository.

Screenshots and recordings

helm ls output. This helm release is upgraded 120 times although there is no change in gitrepository or values configs.
image

OS / Distro

macOS 14

Flux version

flux-2.0.0

Flux check

► checking prerequisites
✔ Kubernetes 1.29.3 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-2.0.0
✔ bootstrapped: false
► checking controllers
✔ helm-controller: deployment ready
► docker.io/fluxcd/helm-controller:v0.35.0
✔ kustomize-controller: deployment ready
► docker.io/fluxcd/kustomize-controller:v1.0.0
✔ notification-controller: deployment ready
► docker.io/fluxcd/notification-controller:v1.0.0
✔ source-controller: deployment ready
► docker.io/fluxcd/source-controller:v1.0.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta2
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta2
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

Git provider

No response

Container Registry provider

No response

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@souleb
Copy link
Member

souleb commented May 17, 2024

So many things happened since v0.38. Please upgrade your Flux version and check whether it fixes your issue. There have been many changes on the helm-controller side.

@mhamzarazzaq97
Copy link
Author

mhamzarazzaq97 commented May 17, 2024

@souleb this is version of flux cli on my local machine, these are the details with latest flux cli. I am observing this issue on flux 2.0.0
► checking prerequisites
✔ Kubernetes 1.29.3 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-2.0.0
✔ bootstrapped: false
► checking controllers
✔ helm-controller: deployment ready
► docker.io/fluxcd/helm-controller:v0.35.0
✔ kustomize-controller: deployment ready
► docker.io/fluxcd/kustomize-controller:v1.0.0
✔ notification-controller: deployment ready
► docker.io/fluxcd/notification-controller:v1.0.0
✔ source-controller: deployment ready
► docker.io/fluxcd/source-controller:v1.0.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta2
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta2
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

@souleb
Copy link
Member

souleb commented May 17, 2024

There are memory improvements on the last version v2.3.0. Also increase your memory limits to avoid OOM on your pod.

See https://fluxcd.io/flux/installation/ for installation and upgrade procedures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants