Is it possible use FINN to make predictions without TopK as the last layer?

[bieelpaiao]

Hello everyone, I'm trying to build a CNN for classification with FINN which takes a 64x64x1 input image and returns an output with the probabilities of 4 classes. I saw the notebook examples for multi-class classification and all of them uses a TopK layer in the output but I really need the predictions for each class so TopK will not work for me.

I made a build without TopK and generated an stitched IP to use in Vivado, but now I am confused about what specifically the numbers in output represent.

The sample that I have simulated expects the following output considering streamlined model and execute.onnx function in Python:
[[30.37711 33.0512 81.76213 -8.510721]]

And the values outputed in Vivado are:
6157 6699 16572 -1725

Anyone knows the relation between these numbers? How can I verify if values in Vivado are close to the expected in Python?

[fpjentzsch]

Hi,
if your model graph contains FINN-incompatible nodes at the beginning or end, such as remaining floating-point scale factors that could not be streamlined into a TopK node, FINN will (quietly) not implement them as part of the accelerator and expect that you perform them manually in software (as pre-/post-processing).

You could check how your intermediate .onnx model looks like after the "convert_to_hw" building step. You might see some Mul oder Add node at the end of the graph which could not be converted to a *_hls or *_rtl CustomOp.

[bieelpaiao]

Now I can understand. Indeed, without TopK in last layer my model has a Mul node which is not converted after convert_to_hw step. When I multiply the Vivado outputs by this Mul factor, I can obtain the expected output values. Thank you!

[fpjentzsch]

Glad you were able to solve it.

Just for completeness: PR #1040 might allow FINN to implement such operations in the future if floating-point support would be added.