[BUG] Teams alerts spam 5x of the same failure with suppression set.

[christopherekfeldt]

Describe the bug
We have set up x number of Teams alert webhooks with elementary. The job runs hourly to check if any new test failures has arose.
We have set up a workflow which runs with cloud schedule "30 6-19 * * 1-5" # Every hour between 06:30 - 18:30 on workdays.

That job iterates over a list of teams and their filters/webhooks url's etc like this:

`
%{for team in var.elementary_alert_config}

• name: '${var.elementary_image}'
  id: elementary-teams-alert-${team.name}
  waitFor:
  • generate-elementary-report
    args: ['monitor', '--teams-webhook', '${team.webhook_url}', '--filters', '${team.filter}', '--suppression-interval', '${team.suppression}', '--profiles-dir', '.dbt-profiles', '--profile-target', '${var.dbt_config.environment}']
    %{endfor}
    `

A team config could then look like this:

elementary_alert_config = [ { name = "marketing" webhook_url = "<webhook_url_>" filter = "owners:BICA Marketing" suppression = 24 },

With this config set, I expect the Marketing Teams channel to only reveal the failure of a test once during 24 hours, and not spam it 5x for every failure, for every team in every channel.

Expected behavior
A clear and concise description of what you expected to happen.

That it only spits out 1 test failure for each test.

Screenshots
If applicable, add screenshots to help explain your problem.

It always spits out 5x of each failure, daily.

Environment (please complete the following information):

• edr Version: 0.14.1
• dbt-core 1.7.10
• dbt-bigquery 17.6

[haritamar]

Hi @christopherekfeldt !
Thanks for surfacing this, and sorry about the delay.
We will try to look into it as soon as possible.

[haritamar]

Hi @christopherekfeldt !
I'd like to follow-up with a couple of questions:

• Do the commands per team run in parallel or sequentially? In general edr monitor does not support parallel runs and it may cause duplicate alerts.
• Just to clarify - does the 5x spamming happen for the correct team? or every alert is sent 5 times to every team?

Thanks,
Itamar

[christopherekfeldt]

We are running them in parallell, oh okay so that can be the issue then? Every alert is sent 5 times to every team!

[haritamar]

Hi @christopherekfeldt - interesting.
I think the parallel issue is definitely the cause for the 5X, but it doesn't explain why you are getting every alert in all channels - that sounds like a filtering issue.

In any case though - will be great if you can try to make it non-parallel and write here what happens in that case.

[christopherekfeldt]

It worked without running them in parallell. Thanks for the suggestion, i only did it since i thought the filter would seperate them neitherway, and that i wanted the scheduled job to be faster and more effective :)