Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ONNX Model Support #2005

Open
zhipeng9802 opened this issue Aug 2, 2024 · 2 comments
Open

ONNX Model Support #2005

zhipeng9802 opened this issue Aug 2, 2024 · 2 comments
Labels
Milestone

Comments

@zhipeng9802
Copy link

Feature request / 功能建议

支持ONNX Model的加载和推理

Motivation / 动机

在仅有CPU的环境下,我不得不考虑使用CPU去运行Embedding & Reranker 模型,经过测试,使用onnx拥有相较于
Sentence Transformer更快的处理速度

Your contribution / 您的贡献

https://huggingface.co/docs/optimum/v1.3.0/en/onnxruntime/modeling_ort#optimum.onnxruntime.ORTModelForSequenceClassification

https://github.com/onnx/onnx

@XprobeBot XprobeBot added this to the v0.14.0 milestone Aug 2, 2024
@qinxuye
Copy link
Contributor

qinxuye commented Aug 3, 2024

Do you have some data to compare ONNX and sentence-transformers?

@zhipeng9802
Copy link
Author

Do you have some data to compare ONNX and sentence-transformers?

I don't have a lot of data, just a simple test using internal industry knowledge data

@XprobeBot XprobeBot modified the milestones: v0.14, v0.15 Sep 3, 2024
@XprobeBot XprobeBot modified the milestones: v0.15, v0.16 Oct 30, 2024
@XprobeBot XprobeBot modified the milestones: v0.16, v1.x Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants