Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training works great, but it doesn't computes correctly losses in validation data #36

Open
AntonioVispi opened this issue Sep 11, 2023 · 0 comments

Comments

@AntonioVispi
Copy link

I trained a Yolov8 model for object detection with my custom dataset, made by videos. Observing at the results the network gave me (both the results.png and the results.csv), I noticed that the train losses are correct: they decrease with the training process, but the validation losses remain constant to zero from the beginning both for the val/box_loss and the val/dfl_loss, while the val/cls_loss is always infinite (it says “inf” in the results.csv).
results

Screenshot

Just a few epochs, to show the issue about val/box_loss, val/dfl_loss, val/cls_loss.

This is the command line I used to train:
!python train.py model=yolov8n.yaml data='My_Path/data/data.yaml' epochs=50 imgsz=800 project='My_Path/Yolov8_models/Yolov8n'
I also did some attempts like interchanging the train and validation datasets or using the train dataset both for training and for validation, but the error still persists, so I guess the problem it’s not on how I defined the validation dataset. Furthermore, I tried to use the trained network on an unseen video and it works just fine, so I think the problem is how it calculates the validation losses specifically (since the train losses are normally calculated), but I can’t find the error.

My suspect is that it computes the losses between ground truth and ground truth, so the loss is exactly zero.

Thank you in advance for your attention to my question, hope you will help me with my issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant