Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why is the optimal result different each time during training? #323

Open
Cao-PeiPei opened this issue Dec 9, 2023 · 4 comments
Open

Comments

@Cao-PeiPei
Copy link

Excuse me, I've encountered an issue. I'm currently training on AutoDL, and I've noticed that the performance of my training differs by 8 points compared to my junior's training of Deformable DETR with the same configuration. I'm considering whether the difference is due to different random seeds. However, even after setting the same random seed, there is still a 5-point difference in performance. I would like to ask how to handle this situation, as I'm concerned that the subsequent experiments might show significant variations in the AP50 values.

@rentainhe
Copy link
Collaborator

How many GPUs do u use for this training

@Cao-PeiPei
Copy link
Author

Sorry, just saw the reply. We are all using a single GPU, and the rest of the configurations are the same.

@rentainhe
Copy link
Collaborator

Hello! Do u use 16 batch-size for training in single GPU?

@Cao-PeiPei
Copy link
Author

NO, we have used batch size=4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants