Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'monitor' is ignored to compute the score #86

Open
AlessandroLovo opened this issue Jan 14, 2024 · 1 comment
Open

'monitor' is ignored to compute the score #86

AlessandroLovo opened this issue Jan 14, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@AlessandroLovo
Copy link
Collaborator

In the function ln.train_model the score of a fold is returned as

score = np.min(history[return_metric])

This means that is monitor != return_metric we early stop according to monitor, but still take the minimum wrt return_metric. So early stopping doesn't make much sense.

A better solution would be

score = history[return_metric][np.argmin(history[monitor])]

This may complicate matters with optimal_chekpoint, as there to have consistency one should have metric == monitor instead of metric == return_metric

@AlessandroLovo AlessandroLovo added the bug Something isn't working label Jan 14, 2024
@AlessandroLovo AlessandroLovo self-assigned this Jan 14, 2024
@AlessandroLovo
Copy link
Collaborator Author

The confusion free scenario is when metric == return_metric == monitor. Outside this condition things could get messy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant