Ways to speed up training using CPU?

[lececefifi]

#question
Hi - I am using multi-label classification, and due to the current circumstances, I can only train with CPU. I am required to train on a virtual machine due to data sensitivity, and the CPUs are intel with avx-512 disabled, so trying the IPEX package is out of question (https://huggingface.co/docs/transformers/perf_train_cpu). I am using 1 epoch and default batch of 8. The data instances is 334,835, and it currently takes about 6 hours to complete 1%.

Anyone have other solutions to speed up the training?

Thank you!