Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model table fails to print / failure parsing the model #670

Closed
mmagician opened this issue Dec 19, 2023 · 4 comments · Fixed by #673
Closed

Model table fails to print / failure parsing the model #670

mmagician opened this issue Dec 19, 2023 · 4 comments · Fixed by #673
Labels
bug Something isn't working

Comments

@mmagician
Copy link
Contributor

mmagician commented Dec 19, 2023

Describe the bug

Hi.

I was trying a bunch of models from Hugging Face and couldn't parse any of them with the ezkl table command. Maybe I was just unlucky in picking the models... here are a few that failed and their error messages. The 1st and 3rd seem related, it looks like there might be some files missing but I really can't tell.

  1. https://huggingface.co/TaylorAI/gte-tiny/blob/main/onnx/model_quantized.onnx
[E] [2023-12-19 20:12:52, ezkl] - failed: Failed analyse for node #330 "/embeddings/Shape" Shape
Error: Failed analyse for node #330 "/embeddings/Shape" Shape

Caused by:
    0: Infering facts
    1: Applying rule outputs[0].datum_type == TDim
    2: Impossible to unify I64 with TDim.
  1. https://huggingface.co/jinaai/jina-embeddings-v2-small-en/blob/main/model.onnx
[E] [2023-12-19 20:20:47, ezkl] - failed: could not get output shape for node
Error: "could not get output shape for node"
  1. https://huggingface.co/Xenova/glpn-kitti/blob/main/onnx/model_quantized.onnx
[E] [2023-12-19 20:21:30, ezkl] - failed: Failed analyse for node #3260 "/glpn/encoder/Shape" Shape
Error: Failed analyse for node #3260 "/glpn/encoder/Shape" Shape

Caused by:
    0: Infering facts
    1: Applying rule outputs[0].datum_type == TDim
    2: Impossible to unify I64 with TDim.

Mildly related to #181.

Expected behaviors

Some of the above models are quite large, so maybe generating the proof would be too costly, but I should at least be able to parse it to see what's inside.

Steps to reproduce the bug

  1. Download the model from one of the above links
  2. Execute ezkl table -M MODELNAME.onnx
@mmagician mmagician added the bug Something isn't working label Dec 19, 2023
@mmagician
Copy link
Contributor Author

Tagging @erhant, thanks for pointing me to some of these models on Hugging Face.

@alexander-camuto
Copy link
Collaborator

Have opened an issue on tract for 1 & 3. sonos/tract#1293

Will address 2 as that is an ezkl specific error

@alexander-camuto
Copy link
Collaborator

alexander-camuto commented Dec 20, 2023

#673 fixes 2. Once merged you the model should table with the following command (if building from main):

RUST_LOG=debug cargo run --bin ezkl --release -- table -M model.onnx --variables sequence_length=1,batch_size=1

where sequence_length and batch_size should be set to the values you want

@alexander-camuto
Copy link
Collaborator

Fyi @mmagician the tract issues were just resolved. Will merge those into latest PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants