Skip to content

Latest commit

 

History

History
153 lines (111 loc) · 13.2 KB

README.md

File metadata and controls

153 lines (111 loc) · 13.2 KB

MoveNet quantized

Use case : Pose estimation

Model description

MoveNet is a single pose estimation model targeted for real-time processing implemented in Tensorflow.

The model is quantized in int8 format using tensorflow lite converter.

Network information

Network information Value
Framework TensorFlow Lite
Quantization int8
Provenance https://www.kaggle.com/models/google/movenet
Paper https://storage.googleapis.com/movenet/MoveNet.SinglePose%20Model%20Card.pdf

Networks inputs / outputs

With an image resolution of NxM with K keypoints to detect :

  • For heatmaps models
Input Shape Description
(1, N, M, 3) Single NxM RGB image with UINT8 values between 0 and 255
Output Shape Description
(1, W, H, K) FLOAT values Where WXH is the resolution of the output heatmaps and K is the number of keypoints
  • For the other models
Input Shape Description
(1, N, M, 3) Single NxM RGB image with UINT8 values between 0 and 255
Output Shape Description
(1, Kx3) FLOAT values Where Kx3 are the (x,y,conf) values of each keypoints

Recommended Platforms

Platform Supported Recommended
STM32L0 [] []
STM32L4 [] []
STM32U5 [] []
STM32H7 [] []
STM32MP1 [x] []
STM32MP2 [x] [x]
STM32N6 [x] [x]

Performances

Metrics

Measures are done with default STM32Cube.AI configuration with enabled input / output allocated option.

Reference NPU memory footprint based on COCO Person dataset (see Accuracy for details on dataset)

Model Dataset Format Resolution Series Internal RAM (KiB) External RAM (KiB) Weights Flash (KiB) STM32Cube.AI version STEdgeAI Core version
ST MoveNet Lightning heatmaps COCO-Person Int8 192x192x3 STM32N6 1674 0.0 3036.17 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 192x192x3 STM32N6 1674 0.0 3036.41 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 224x224x3 STM32N6 2058 0.0 3088.56 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 256x256x3 STM32N6 2360 0.0 3141.36 10.0.0 2.0.0

Reference NPU inference time based on COCO Person dataset (see Accuracy for details on dataset)

Model Dataset Format Resolution Board Execution Engine Inference time (ms) Inf / sec STM32Cube.AI version STEdgeAI Core version
ST MoveNet Lightning heatmaps COCO-Person Int8 192x192x3 STM32N6570-DK NPU/MCU 18.44 54.23 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 192x192x3 STM32N6570-DK NPU/MCU 18.49 54.08 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 224x224x3 STM32N6570-DK NPU/MCU 22.33 44.78 10.0.0 2.0.0
MoveNet Lightning heatmaps COCO-Person Int8 256x256x3 STM32N6570-DK NPU/MCU 27.01 37.03 10.0.0 2.0.0

Reference MPU inference time based on COCO Person dataset (see Accuracy for details on dataset)

Model Format Resolution Quantization Board Execution Engine Frequency Inference time (ms) %NPU %GPU %CPU X-LINUX-AI version Framework
ST MoveNet Lightning heatmaps Int8 192x192x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 58.02 ms 3.75 96.25 0 v5.0.0 OpenVX
ST MoveNet Lightning heatmaps Int8 192x192x3 per-tensor STM32MP257F-DK2 NPU/GPU 800 MHz 7.93 ms 84.89 15.11 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 192x192x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 58.17 ms 3.80 96.20 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 192x192x3 per-tensor STM32MP257F-DK2 NPU/GPU 800 MHz 8.00 ms 86.48 13.52 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 224x224x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 81.65 ms 2.77 97.23 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 224x224x3 per-tensor STM32MP257F-DK2 NPU/GPU 800 MHz 11.55 ms 87.04 12.96 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 256x256x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 70.57 ms 3.74 96.26 0 v5.0.0 OpenVX
MoveNet Lightning heatmaps Int8 256x256x3 per-tensor STM32MP257F-DK2 NPU/GPU 800 MHz 12.90 ms 86.33 13.67 0 v5.0.0 OpenVX
MoveNet Lightning Int8 192x192x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 66.97 ms 6.72 93.28 0 v5.0.0 OpenVX
MoveNet Thunder Int8 256x256x3 per-channel** STM32MP257F-DK2 NPU/GPU 800 MHz 187.1 ms 3.96 96.04 0 v5.0.0 OpenVX

** To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization

OKS on COCO Person dataset

Dataset details: link , License CC BY 4.0 , Quotation[1] , Number of classes: 80, Number of images: 118,287

Model Format Resolution OKS
ST MoveNet Lightning heatmaps per-channel Int8 192x192x3 *52.1 %
ST MoveNet Lightning heatmaps per-tensor Int8 192x192x3 *39.31 %
MoveNet Lightning heatmaps per-channel Int8 192x192x3 54.01 %
MoveNet Lightning heatmaps per-tensor Int8 192x192x3 48.49 %
MoveNet Lightning heatmaps per-channel Int8 224x224x3 57.07 %
MoveNet Lightning heatmaps per-tensor Int8 224x224x3 50.93 %
MoveNet Lightning heatmaps per-channel Int8 256x256x3 58.58 %
MoveNet Lightning heatmaps per-tensor Int8 256x256x3 52.86 %
MoveNet Lightning Int8 192x192x3 54.12%
MoveNet Thunder Int8 256x256x3 64.43%

* keypoints = 13

Integration in a simple example and other services support:

Please refer to the stm32ai-modelzoo-services GitHub here

References

[1] “Microsoft COCO: Common Objects in Context”. [Online]. Available: https://cocodataset.org/#download. @article{DBLP:journals/corr/LinMBHPRDZ14, author = {Tsung{-}Yi Lin and Michael Maire and Serge J. Belongie and Lubomir D. Bourdev and Ross B. Girshick and James Hays and Pietro Perona and Deva Ramanan and Piotr Doll{'{a} }r and C. Lawrence Zitnick}, title = {Microsoft {COCO:} Common Objects in Context}, journal = {CoRR}, volume = {abs/1405.0312}, year = {2014}, url = {http://arxiv.org/abs/1405.0312}, archivePrefix = {arXiv}, eprint = {1405.0312}, timestamp = {Mon, 13 Aug 2018 16:48:13 +0200}, biburl = {https://dblp.org/rec/bib/journals/corr/LinMBHPRDZ14}, bibsource = {dblp computer science bibliography, https://dblp.org} }