About INT8 and UINT8

[hunterchenghx]

Excause me. I think your quantized prune yolo5 model is trained with uint8 weights and int32 bias.
Could you tell me how to adjust the code so that I could train a model with INT8 weights?
Sometimes, other deployment tools could only support INT8 datatype but not uint8 for a quantized model(like TensorRT8).

[rui-shen-afk]

Hello, is this problem solved somehow?

[bfineran]

Hi @hunterchenghx @rui-shen-afk weight qconfig properties can be modified directly using weight_config_kwargs, however for tensorrt support, the current best pathway is to set tensorrt: True for any QuantizationModifier in your recipe.

sparseml/src/sparseml/pytorch/sparsification/quantization/modifier_quantization.py

Line 142 in d845052

:param tenssorrt: if True sets quantization configuration for compatibility with

Hope this helps, let me know if you have any other questions.