[FEATURE] Early stopping when training

[saad-palapa]

Is your feature request related to a problem? Please describe.
When running the training script, there is no way to stop training early when performance plateaus.

This causes 2 problems:

• Wasting resources because we continue training as the model does not improve
• Overfitting since early stopping is a form of regularization

Describe the solution you'd like
I would like to add an early-stopping feature to the training script.

This would involve a few parameters:

• The metric to monitor
• Whether to minimize or maximize that metric
• Number of steps or epochs to wait for an improvement
• The delta in the metric to determine if it is an improvement

Describe alternatives you've considered
Many other CV training platforms offer early stopping. I would like to have early stopping in TIMM because I prefer this repo over others.

[notprime]

Hi @saad-palapa ,

you could easily modify the train.py script adding your own EarlyStopping method, it's straightforward.
You just have to gather validation losses (eval_metrics, line 813), select the metric you want to monitor (top1 for instance), and use it as a stopping criteria.

You can use this simple yet effective implementation of EarlyStopping:
https://github.com/Bjarten/early-stopping-pytorch/blob/master/pytorchtools.py

There you'll find a simple example on how to use it (in the "Train the Model using Early Stopping" section):
https://github.com/Bjarten/early-stopping-pytorch/blob/master/MNIST_Early_Stopping_example.ipynb

The provided training script in TIMM is just a general script, you have to adapt it to your case :D