Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

大佬请教一下模型训练问题 #157

Open
MrWangChong opened this issue Dec 30, 2024 · 3 comments
Open

大佬请教一下模型训练问题 #157

MrWangChong opened this issue Dec 30, 2024 · 3 comments
Labels
question Further information is requested

Comments

@MrWangChong
Copy link

Epoch: 10/10, Batch:0/1, Loss: 0.0000: 100%|██████████████████████████████████████████████████████████████████████| 1/1 [01:28<00:00, 88.81s/it]
Running Evaluation: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:14<00:00, 14.68s/it]
2024-12-30 15:08:37.588 | DEBUG | text2vec.sentence_model:evaluate:277 - labels: [0, 1, 1, 0, 1, 1, 1, 1, 0, 0]███| 1/1 [00:14<00:00, 14.68s/it]
2024-12-30 15:08:37.600 | DEBUG | text2vec.sentence_model:evaluate:278 - preds: [0.02931239, 0.8442257, 0.9048866, -0.10592551, 0.9284405, 0.93183136, 0.8627026, 0.8772242, -0.050434127, -0.073510975]
2024-12-30 15:08:37.600 | DEBUG | text2vec.sentence_model:evaluate:279 - pearson: 0.9919297894994221, spearman: 0.8432740427115677
2024-12-30 15:08:37.620 | INFO | text2vec.sentence_model:eval_model:231 - {'eval_spearman': 0.8432740427115677, 'eval_pearson': 0.9919297894994221}

训练的时候看着都是正常的,如上面的数据里面的labels和preds,训练评估的数据也没问题。但是训练完成的评估就突然不正常了,如下面的labels和preds,看起来和训练的效果完全不一致,请问一下大佬这是怎么回事?

Epoch: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [17:14<00:00, 103.49s/it]
2024-12-30 15:08:37.740 | INFO | text2vec.cosent_model:train_model:130 - Training model done. Saved to ./outputs/STS-B-model.
2024-12-30 15:08:37.745 | INFO | main:main:87 - Model saved to ./outputs/STS-B-model
2024-12-30 15:08:39.283 | DEBUG | text2vec.sentence_model:init:78 - Use pytorch device: cpu
2024-12-30 15:08:39.313 | DEBUG | main:main:118 - ('A节主变风机出现比较大的异常声音。', 'A节主变水机出现比较大的异常震动。', 0)
2024-12-30 15:08:40.095 | DEBUG | main:main:120 - <class 'numpy.ndarray'>, (26, 768), (768,)
2024-12-30 15:08:41.805 | DEBUG | main:calc_similarity_scores:26 - labels: [0 1 1 0 1 1 1 1 0 0]
2024-12-30 15:08:41.810 | DEBUG | main:calc_similarity_scores:27 - preds: [0.8464496 0.8778059 0.953287 0.75261563 0.9444511 0.91630775 0.94660693 0.8541326 0.7975155 0.7461488 ]
2024-12-30 15:08:41.810 | DEBUG | main:calc_similarity_scores:28 - Spearman: 0.8432740427115677
2024-12-30 15:08:41.810 | DEBUG | main:calc_similarity_scores:29 - spend time: 1.6849, count:52, qps: 30.86247843307947

训练命令就是最原始的:python training_sup_text_matching_model_mydata.py --do_train --do_predict
修改了:save_model_every_epoch: bool = False
训练、验证、测试 数据集都是完全相同的三份我手动输入的数据,数据试过0和1,1-5。都是一样的结果。
也试过训练其它类型的模型,都遇到了这个问题,在训练的时候labels和preds没问题,完成之后就不行了。

@MrWangChong MrWangChong added the question Further information is requested label Dec 30, 2024
@shibing624
Copy link
Owner

啥叫完成之后就不行了?带入保存的模型,做预测就可以了。

@MrWangChong
Copy link
Author

训练时结果是这样的:
labels: [0, 1, 1, 0, 1, 1, 1, 1, 0, 0]
preds: [0.02931239, 0.8442257, 0.9048866, -0.10592551, 0.9284405, 0.93183136, 0.8627026, 0.8772242, -0.050434127, -0.073510975]
预测时结果是这样的:
labels: [0 1 1 0 1 1 1 1 0 0]
preds: [0.8464496 0.8778059 0.953287 0.75261563 0.9444511 0.91630775 0.94660693 0.8541326 0.7975155 0.7461488 ]

@MrWangChong
Copy link
Author

啥叫完成之后就不行了?带入保存的模型,做预测就可以了。

训练时结果是这样的:
labels: [0, 1, 1, 0, 1, 1, 1, 1, 0, 0]
preds: [0.02931239, 0.8442257, 0.9048866, -0.10592551, 0.9284405, 0.93183136, 0.8627026, 0.8772242, -0.050434127, -0.073510975]
预测时结果是这样的:
labels: [0 1 1 0 1 1 1 1 0 0]
preds: [0.8464496 0.8778059 0.953287 0.75261563 0.9444511 0.91630775 0.94660693 0.8541326 0.7975155 0.7461488 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants