ANE Usage: 200% @ 16.1 W

[wangchou]

Is ANE usage 200% correct?

This test model goes to 200% when creating XCode performance report.

Device: MacBook M1 Air
OS: macOS 13.2

import torch
from torch import Tensor
from torch import nn
import coremltools as ct
from coremltools.models.neural_network import quantization_utils

class MyModule(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x: Tensor):
        for i in range(1,500):
            x = x @ x
        return x

module = MyModule()
module.eval()

input_x = torch.ones(1024, 1024)
traced_module = torch.jit.trace(module, input_x)

# convert to coreml model
module = ct.convert(
    traced_module,
    inputs=[ct.TensorType(name="input_x", shape=input_x.shape)],
    outputs=[ct.TensorType(name="output")],
    compute_units=ct.ComputeUnit.ALL,
)

module_fp16 = quantization_utils.quantize_weights(module, nbits=16)
module_fp16.save(f"module_fp16.mlmodel")

[dehydratedpotato]

@wangchou oh yeah that is definitely a bug. Probably from a flaw in the way asitop estimates the usage based on ane power (this is done because ane usage is not directly provided by the system)

[stonerl]

The problem is that ane_max_power = 8.0 is hardcoded.

According to your screenshot, you're using 16 watts. Which lets this calculation fail:

asitop/asitop/asitop.py

Line 244 in 74ebe2c

cpu_metrics_dict["ane_W"] / args.interval / ane_max_power * 100)

So, either the hardcoded value of 8 is wrong and/or the watt usage reported by powermetrics is. It could be both tough. Apple seems to consistently break the values reported by powermetrics with each release.