We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Running the following to quantize models using a group size of 32 causes a number of error when running the model:
mlx_lm.convert --hf-path microsoft/phi-4 --quantize --q-bits 2 --q-group-size 32
Attempting to run this with MLXLLM.loadModelContainer will cause the following error:
MLXLLM.loadModelContainer
Error downloading model: mismatchedSize(key: "biases", expectedShape: [100352, 80], actualShape: [100352, 160])
Steps to reproduce:
Expected Result: The model should load. Actual result: You'll get an error, instead of it loading.
The text was updated successfully, but these errors were encountered:
I just tried running it and it ran fine for me on the main branch of mlx-swift-examples. Are you using an outdated version?
Sorry, something went wrong.
No branches or pull requests
Running the following to quantize models using a group size of 32 causes a number of error when running the model:
Attempting to run this with
MLXLLM.loadModelContainer
will cause the following error:Steps to reproduce:
MLXLLM.loadModelContainer
Expected Result: The model should load.
Actual result: You'll get an error, instead of it loading.
The text was updated successfully, but these errors were encountered: