Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't train successfully with COCO2017 dataset #787

Open
metalgear54 opened this issue Sep 19, 2022 · 2 comments
Open

Can't train successfully with COCO2017 dataset #787

metalgear54 opened this issue Sep 19, 2022 · 2 comments

Comments

@metalgear54
Copy link

I'm using the lateset code to train COCO2017 dataset. But only get very low mAP.
By using the yolov3.weights, I can get mAP=65 in validation dataset and mAP=67 in training dataset using test.py. which can prove that my dataset setup is right. But when i train the dataset with the same yolov3.weights module, the mAP will fall to nearly 0 after 1 or 2 epochs.
Can you analyse why this happen? If I set the leraning rate very litlle such as 1e-6, the mAP still go down each training epoch.

@fddance
Copy link

fddance commented Sep 21, 2022

i have this problem too.I check the label and found it was normal

@Flova
Copy link
Collaborator

Flova commented Sep 30, 2022

Did you try to train for more epochs? Is sounds kind of weird, but maybe the optimizer choice or something else is different to the training setup of the pretrained weights, and this results in a convergence to a different (but maybe somewhat equivalent performance wise) point in the parameter space. This could result in a temporary reduction of performance for a few epochs while we move from one local minimum to the other. Try training for something like 100 epochs with a reasonable learning rate and show us the results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants