cyclictest

Cyclictest accurately and repeatedly measures the difference between a thread's intended wake-up time and the time at which it actually wakes up in order to provide statistics about the system's latencies. It can measure latencies in real-time systems caused by the hardware, the firmware, and the operating system.

Deployment

pip3 install jinja2-tools --user
jinja render -e duration=1h -e cpu=16 -e profile_name=$(oc get performanceprofile -o jsonpath='{.items[0].metadata.name}') -e cnf_node_selector=worker-cnf -t ./pod.j2 -o pod.yaml
oc apply -f performanceprofile.yaml
# ... Wait for mcp to complete updating the nodes ...
oc apply -f pod.yaml

Histogram

oc logs cyclictest > cyclictest_logs
csplit --digits=1 --quiet --prefix=histogram_log_ cyclictest_logs "/# Histogram/" "{0}"

histogram_log_0: Contains the cyclictest output
histogram_log_1: Contains the cyclictest histogram

Wiki

https://wiki.linuxfoundation.org/realtime/documentation/howto/tools/cyclictest/start