Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Questions about re-implementation #20

Open
insanebai opened this issue Apr 29, 2024 · 1 comment
Open

Questions about re-implementation #20

insanebai opened this issue Apr 29, 2024 · 1 comment

Comments

@insanebai
Copy link

Hello author, I conducted experiments according to your file configuration (completely consistent settings) and found that the experimental results are different from the paper results.

Task t1:
AOSE is about 1500 higher
K-map is 4 points lower
UR is 3 points lower
What causes this?

@WangPingA
Copy link

[05/06 10:32:54 d2.evaluation.evaluator]: Total inference time: 1:46:22.203930 (0.704982 s / iter per device, on 1 devices)
[05/06 10:32:54 d2.evaluation.evaluator]: Total inference pure compute time: 1:45:54 (0.701939 s / iter per device, on 1 devices)
Wilderness Impact: {0.1: {50: 0.019775390624999997}, 0.2: {50: 0.026684253246753248}, 0.3: {50: 0.03153739177246422}, 0.4: {50: 0.029262144473296583}, 0.5: {50: 0.025682171277689847}, 0.6: {50: 0.024073368590892794}, 0.7: {50: 0.01802047416673349}, 0.8: {50: 0.01965570856271746}, 0.9: {50: 0.02210940767555562}}
avg_precision: {0.1: {50: 0.09990769012284315}, 0.2: {50: 0.09990769012284315}, 0.3: {50: 0.09990769012284315}, 0.4: {50: 0.09990769012284315}, 0.5: {50: 0.09990769012284315}, 0.6: {50: 0.09990769012284315}, 0.7: {50: 0.09990769012284315}, 0.8: {50: 0.09990769012284315}, 0.9: {50: 0.09990769012284315}}
Absolute OSE (total_num_unk_det_as_known): {50: 9528.0}
total_num_unk 23320
AP50: ['67.6', '47.1', '49.4', '29.2', '21.3', '60.4', '48.1', '78.2', '15.7', '50.8', '17.5', '66.8', '66.8', '56.2', '45.5', '21.1', '55.5', '38.5', '71.9', '50.4', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.9']
Precisions50: ['6.6', '6.1', '5.2', '2.5', '2.3', '11.4', '6.1', '16.3', '2.7', '3.7', '5.5', '11.5', '8.6', '8.5', '6.5', '3.5', '4.0', '6.8', '16.2', '8.8', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '10.0']
Recall50: ['92.8', '70.7', '75.0', '68.6', '44.3', '79.4', '67.8', '91.4', '45.6', '89.3', '38.2', '90.7', '90.2', '80.5', '67.4', '67.5', '86.5', '71.6', '86.0', '73.7', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '6.0']
Current class AP50: 47.89914697917603
Current class Precisions50: 7.141895371900378
Current class Recall50: 73.84517149369682
Known AP50: 47.89914697917603
Known Precisions50: 7.141895371900378
Known Recall50: 73.84517149369682
Unknown AP50: 0.8798663376498874
Unknown Precisions50: 9.980846988721003
Unknown Recall50: 6.0334476843910805

我的结果是这样的,哥哥

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants