Metrics for which no data was recorded are not displayed or evaluated for Thresholds

[sniku]

When a custom metric is defined, but no data for that metric is recorded, it's not displayed in the results, and thresholds are not properly evaluated.

It's easiest to show it on an example.

Consider this script.

import { Counter } from 'k6/metrics';
import { sleep } from 'k6';
import http from 'k6/http';

let allErrors = new Counter('error_counter');

export let options = {
  thresholds: {
    error_counter: [
      'count < 10', // 10 of fewer errors are tolerated
    ]
  }
}

export default function () {
  let auth_resp = http.post('https://test-api.k6.io/auth/token/login/', {username: 'test-user', 'password': 'supersecure'});

  if (auth_resp.status >= 400){
    allErrors.add(1, { errorType: 'authError' }); // tagged value creates submetric (useful for making thresholds specific)
  }

  sleep(1);
}

If the test run is successful, the allErrors will contain 0 errors. In that case metric as well as the threshold indicator won't be present in the output.

I expected both to be present, just like in this case, when 1 error was recorded:

The most problematic manifestation of this bug is when threshold should fail, but doesn't because it's not evaluated.

Example:

import { Counter } from 'k6/metrics';
import { sleep } from 'k6';
import http from 'k6/http';

let allErrors = new Counter('error_counter');

export let options = {
  thresholds: {
    error_counter: [
      'count < 10' // This threshold should fail, but it doesn't
    ]
  }
}

export default function () {
  sleep(1);
}

[na--]

A duplicate of #1053, but I'll leave this open, since it's better.

[na--]

FWIW, there's an easy workaround for this issue, when it comes to Counter metrics at least: https://community.k6.io/t/looking-for-max-execution-time-option/613/3

[na--]

#1832 (or something like it) is a prerequisite for the current issue.