Strangeness with timestamp()

[lapo-luchini]

Describe the bug

If I use this I get all the results:

vm_app_version{job="vmagent"}

but when I do this I get a varying amount of records:

timestamp(vm_app_version{job="vmagent"})

I use this kind of rule to detect when a remote vmagent stopped responding, but I am now getting false positives.

I noticed that this happened when I upgraded vmagent from 1.93.x to 1.100.1, and not when (previously) I upgraded victoria-metrics di 1.100.1.

To Reproduce

Version

victoria-metrics-20240411-091522-tags-v1.100.1-0-g9ee51e3

Logs

No response

Screenshots

No response

Used command-line flags

No response

Additional information

No response

[lapo-luchini]

Seems to be fixed in v1.101.0… ?
(works with Grafana Explore, but the VMUI interface is broken — might be an issue with the fact that I built it myself though, I didn't check in deeper yet)

[hagen1778]

Hello @lapo-luchini!
timestamp function checks for the last timestamp over specified look-behind window timestamp(foo[5m]). If look-behind window is omitted timestamp(foo), then it uses step param instead. On the screenshot you have step 5s, so result is likely depends on whether you had a successful scrape during last 5s + latency_offset (30s), or not.
Grafana Explore probably doesn't send step param, so it defaults to 5m when omitted. Does it make sense?

[lapo-luchini]

Grafana Explore probably doesn't send step param, so it defaults to 5m when omitted. Does it make sense?

It does! Originally I noticed the problem in vmalert rules, tough, not using vmui.
It had a lot of false negatives in latest updates that didn't happen before.
Might have been a change… but explanation makes sense, I'll add an explicit look-behind window to the rule.