Extend test description with anomaly_score and training_stddev

[adrianoesch]

Is your feature request related to a problem? Please describe.
We often get alerts in slack with minimal context. I would like to extend the test description with more information, so we can act faster on alerts.

Describe the solution you'd like
I would like to add the anomaly_score and training_stddev in the test description string here. Since these strings are the same for column, table and dimension tests, I would create a separate macro named sth like anomaly_reference_description that can be called in all three description macros.

Would you be willing to contribute this feature?
If this sounds promising to you, I'm happy to contribute a PR.

[haritamar]

Hi @adrianoesch !
Thank you for opening this issue and apologies for the delay.

Just curious, if this issue is still relevant - how would you utilise these values for immediate action on alerts?

[adrianoesch]

hi @haritamar and thanks for getting back. i would react with more urgency to an anomaly with z-score >5 than one that barely makes it over the threshold. also, anomaly thresholds can be quite fiddly and we're still trying to figure out what good thresholds are for some tests. having the z-score with the alerts therefore would make it easier to get a better feeling of what good thresholds might be when flagging alerts as true or false positives. then, the standard deviation is more of a supplementary peace of information that ties the other values together. it makes the other values easier to deduce but in general less valuable than the z-score.

[haritamar]

Thanks for the explanation @adrianoesch
I have a couple of follow-up questions:

1. What do you think about including the expected range as a part of the description? I'm just thinking that for many people this is easier-to-understand than stddev / anomaly score (but I could be wrong).
2. We're considering adding the anomaly graph (same one as in the report) to the alerts. Do you think that if you had that it would cover the need? (Not sure if we can promise that in the near future but just curious)

[adrianoesch]

1. what i'm looking for is a more granular indicator of how strong an anomaly from an alarm is, how much did the value deviate from the expected mean, more than just binary yes/no according to the an "arbitrarily" set sensitivity. as one cannot infer the extent of the deviation from the expected mean from the expected range, i don't think this would help much.
2. having the graph in the alerts would be convenient for sure, but it's currently just a click or two away so i would give this a high prio either.