duplicate timeseries coming from the metrics endpoint

[gfa]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that AWX is open source software provided for free and that I might not receive a timely response.
• I am NOT reporting a (potential) security vulnerability. (These should be emailed to security@ansible.com instead.)

Bug Summary

hello

i'm scraping awx for metrics and i've found duplicate timeseries with different timestamps (but same labels), looks to me like a left over from PR 13311

AWX version

24.3.1

Select the relevant components

• UI
• UI (tech preview)
• API
• Docs
• Collection
• CLI
• Other

Installation method

kubernetes

Modifications

no

Ansible version

No response

Operating system

No response

Web browser

No response

Steps to reproduce

list all duplicate metrics

$ curl -s  http://awx.xxx.xxx/api/v2/metrics/ |grep HELP |sort |uniq -d
# HELP subsystem_metrics_pipe_execute_calls Number of calls to pipe_execute
# HELP subsystem_metrics_pipe_execute_seconds Time spent saving metrics to redis
# HELP subsystem_metrics_send_metrics_seconds Time spent sending metrics to other nodes

verify each one of them

$ curl -s  http://awx.xxx.xxx/api/v2/metrics/ |grep subsystem_metrics_pipe_execute_calls | grep -v ^#
subsystem_metrics_pipe_execute_calls{node="awx-task-856bb4b84-h295h"} 83061
subsystem_metrics_pipe_execute_calls{node="awx-task-856bb4b84-h295h"} 83037

$ curl -s  http://awx.xxx.xxx/api/v2/metrics/ |grep subsystem_metrics_pipe_execute_seconds |grep -v ^#
subsystem_metrics_pipe_execute_seconds{node="awx-task-856bb4b84-h295h"} 93.82122787150729
subsystem_metrics_pipe_execute_seconds{node="awx-task-856bb4b84-h295h"} 93.79361654251989

$ curl -s  http://awx.xxx.xxx/api/v2/metrics/ |grep subsystem_metrics_send_metrics_seconds |grep -v ^#
subsystem_metrics_send_metrics_seconds{node="awx-task-856bb4b84-h295h"} 440.74024668122
subsystem_metrics_send_metrics_seconds{node="awx-task-856bb4b84-h295h"} 440.57595454629336

Expected results

no duplicated metrics

Actual results

duplicated metrics

Additional information

this AWX install was done with awx-operator 2.13.1 (AWX 24.0.0) then upgraded to awx-operator 2.16.1 (AWX 24.3.1)