panic on model loading in edgen_rt_llama_cpp

[toschoo]

Description

This line, very sporadically, causes a panic.

Solution

I guess that the problem is the lazy implementation of UnloadingModel but I couldn't prove it yet. The bug is simply too rare. If this is the problem, however, retry should solve the issue.

Remark

The code in Whisper is similar and, whatever solution is found for LLM, it should also be applied there.

[toschoo]

I tried for hour but I cannot reproduce the bug.