mkl_malloc: failed to allocate memory

[samuelbraun04]

Running

model = WhisperModel("large-v3", device="cuda", compute_type="float16")

gives me:

mkl_malloc: failed to allocate memory

I have 32 GB of installed DDR5 memory and I'm running this on a 4080 with 16 GB of VRAM. I'm not really sure how this error keeps coming up, since I have tons of RAM. Anyone else experiencing/experienced the same thing?

[unmotivatedgene]

@samuelbraun04 Mine has started doing this was well occasionally after working fine previously. Did you find a resolution? My guess is torch versions.

[samuelbraun04]

@unmotivatedgene Unfortunately I never figured a fix out, so I thought maybe it was just something I messed up on my end. But it's coming out of nowhere again for me and I haven't changed anything, so now I'm just confused. And I'm literally running this on a 4080 so I really don't get how I could be running out of memory.