Hardware Model Benchmarking

This repository holds the scripts to collect perfomance metrics from different IoT hardware while running Machine learning Models

Methodology

Mount the NAS on board

Have the board connected to the purdue network. Then mount the sotrage from Tokyo

mkdir ~/shared
sudo mount -t cifs //128.46.74.12/ParthShared ~/shared -o username=parth

The password is parth123

Install Requirements

Ideally make a virtual env to install the requriments. Also not torch version may differ on the type of board/nvidia drivers you plan to use.

pip install -r requirements.txt

Use this sheet to get which dataset and model you can use

https://docs.google.com/spreadsheets/d/1Gha5fKgtT-MW1U7xAZXRYdubQMwFWqmRj7DwQWVBFn4/edit?usp=sharing

Edit Run Details.

Inside eval.py edit the top variable with the details of the run.

# ------------------------------------------------
# --------- CHange this for every run ------------
# ------------------------------------------------
board_name = Board.RASPBERRY_PI_4

model_name = 'inception_v3'
model = models.inception_v3(weights=Inception_V3_Weights.IMAGENET1K_V1)

dataset_name = 'imagenet_validation'
num_samples = 1000

TORCH_HOME = os.path.expanduser("~/shared")
read_cpu_temp = check_RPI_CPU_temp

# ------------------------------------------------

Here poeple not using RPI please write your know function to read CPU temp. Also if you are using different model / dataset you might need to update the way data is being loaded.

Running Benchmark

python eval.py