Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llava sft embedding报错 #419

Open
ssn1771 opened this issue Jan 8, 2025 · 2 comments
Open

llava sft embedding报错 #419

ssn1771 opened this issue Jan 8, 2025 · 2 comments

Comments

@ssn1771
Copy link

ssn1771 commented Jan 8, 2025

在跑examples/llava的时候，embedding阶段报错
image
image
看起来像是索引越界？ masked_input shape是[1, 128]，embedding weight shape是[32000, 4096]
@jerryli1981
Copy link
Collaborator

您好，请问是原封不动的按照ReadMe里面的流程跑下来的吗？https://github.com/alibaba/Pai-Megatron-Patch/blob/main/examples/llava_mcore/README.md#Megatron-Core%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E6%B5%81%E7%A8%8B

@ssn1771
Copy link
Author

ssn1771 commented Jan 10, 2025

您好，请问是原封不动的按照ReadMe里面的流程跑下来的吗？https://github.com/alibaba/Pai-Megatron-Patch/blob/main/examples/llava_mcore/README.md#Megatron-Core%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E6%B5%81%E7%A8%8B

我跑的llava，您这说的llava-mcore，这两个有什么区别吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jerryli1981 @ssn1771