Errors in middlewares break fastapi service

[aguschin]

Not so long ago, we added Prometheus middleware https://github.com/iterative/mlem/blob/main/mlem/contrib/prometheus.py which can be attached to FastAPI service MLEM is building, processing requests and responses data and exposing /metrics endpoint to be scraped by Prometheus.

It happens, that if code in middleware breaks, the server doesn't return the response. Which is a shame, it still needs to return response in any case. We need to fix this.

Example can be found in https://github.com/aguschin/art-expert-telegram-bot/tree/main/fastapi-deployment:

• server.mlem has a config example, you can use it to run the service
• lang_metric.py has middleware metrics code - to reproduce simply make any of them fail (1/0 will suffice)

Setting p2 priority since the feature is not documented and is experimental.

[aminalaee]

I was checking this out, what do you mean it breaks the service?
While testing I can see the response is returned and also the traceback is displayed, which is a good thing since you don't want to mask the errors in the middlewares.

[aguschin]

If that behaves like you say, it's how it should be I think @aminalaee. I didn't get normal (status code 200) responses when I was testing this. Experienced that on multiple deployments. I wonder if this is flaky.