Triggerer's async thread was blocked

[WytzeBruinsma]

Checks

• I have checked for existing issues.
• This report is about the User-Community Airflow Helm Chart.

Chart Version

8.8.0

Kubernetes Version

Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.4", GitCommit:"fa3d7990104d7c1f16943a67f11b154b71f6a132", GitTreeState:"clean", BuildDate:"2023-07-19T12:20:54Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.7", GitCommit:"55a7e688f9220adca1c99b7903953911dd38b771", GitTreeState:"clean", BuildDate:"2023-11-03T12:18:23Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}

Helm Version

version.BuildInfo{Version:"v3.12.3", GitCommit:"3a31588ad33fe3b89af5a2a54ee1d25bfe6eaa5e", GitTreeState:"clean", GoVersion:"go1.20.7"}

Description

The airflow pod trigger is raising errors and is slowing down Airflow processes. The error is; Triggerer's async thread was blocked for 0.23 seconds, likely by a badly-written trigger. Set PYTHONASYNCIODEBUG=1 to get more information on overrunning coroutines.. I tried resolving this by increasing the resources, but even after removing all the limits and giving it 10 GB RAM and lots of CPU head room it still raises this error. I also check the response times of the Postgres database and couldn't find any thing that could slow down the async process and cause this error. Please let me know what other steps I can do to resolve this error.

Relevant Logs

2024-02-23 01:48:09.268	
[2024-02-23T00:48:09.267+0000] {triggerer_job_runner.py:573} INFO - Triggerer's async thread was blocked for 0.23 seconds, likely by a badly-written trigger. Set PYTHONASYNCIODEBUG=1 to get more information on overrunning coroutines.
2024-02-23 09:00:44.327	
[2024-02-23T08:00:44.325+0000] {triggerer_job_runner.py:573} INFO - Triggerer's async thread was blocked for 0.38 seconds, likely by a badly-written trigger. Set PYTHONASYNCIODEBUG=1 to get more information on overrunning coroutines.

Custom Helm Values

No response