Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vmalert: speed up rules reload from object storage #6210

Open
hagen1778 opened this issue Apr 30, 2024 · 0 comments
Open

vmalert: speed up rules reload from object storage #6210

hagen1778 opened this issue Apr 30, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request enterprise Enterprise issue, not included in free open source version TBD To Be Done vmalert

Comments

@hagen1778
Copy link
Collaborator

hagen1778 commented Apr 30, 2024

Is your feature request related to a problem? Please describe

When loading rule files from object storage vmalert performs to subsequent actions:

  1. List all files in configured bucket. Having the list allows prefix matching by file name, to select only specific files to load.
  2. Read files that match filters from the p1.

The most time consuming stage is p2, as it loads each individual file and parses it. Would be great, if vmalert could speed up this stage by checking whether file's update time has changed or whether there were any changes to the bucket at all. In this way, vmalert could detect if something needs to be downloaded from object storage much faster.

Describe the solution you'd like

vmalert should remember file versions or update time of rules loaded from object storage. On next config reload call, vmalert should list all available files in the bucket and download only those which were updated since the last config reload.

@hagen1778 hagen1778 added enhancement New feature or request enterprise Enterprise issue, not included in free open source version vmalert TBD To Be Done labels Apr 30, 2024
@Haleygo Haleygo self-assigned this May 6, 2024
hagen1778 pushed a commit that referenced this issue May 21, 2024
hagen1778 pushed a commit that referenced this issue May 21, 2024
#755)

#6210
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 974b778)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request enterprise Enterprise issue, not included in free open source version TBD To Be Done vmalert
Projects
None yet
Development

No branches or pull requests

2 participants