Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Univariate drift calculation fails: autodetected range of [nan, nan] is not finite #339

Closed
nnansters opened this issue Nov 17, 2023 · 0 comments · Fixed by #340
Closed
Assignees
Labels
bug Something isn't working triage Needs to be assessed

Comments

@nnansters
Copy link
Contributor

Describe the bug

Running univariate drift detection, in this case JS, fails raising the following exception:

 nannyml.exceptions.CalculatorException: failed while fitting <nannyml.drift.univariate.calculator.UnivariateDriftCalculator object at 0x7fba9ba21c90>.
 autodetected range of [nan, nan] is not finite

To Reproduce

  • Take the car loan binary classification dataset
  • Set values for a feature to np.nan
  • Run UnivariateDriftCalculator with jensen_shannon as a method

Expected behavior

  • The calculator should deal with NaN values without failing.
  • Other methods are still calculated even when first one fails
  • In case of all np.NaN it should return np.NaN and show a warning

Screenshots & scripts

 Traceback (most recent call last):
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/base.py", line 358, in fit
     return self._fit(reference_data, *args, **kwargs)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/usage_logging.py", line 231, in logging_wrapper
     raise runtime_exception
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/usage_logging.py", line 180, in logging_wrapper
     res = func(*args, **kwargs)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/drift/univariate/calculator.py", line 275, in _fit
     self._column_to_models_mapping[column_name] += [
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/drift/univariate/calculator.py", line 282, in <listcomp>
     ).fit(
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/drift/univariate/methods.py", line 101, in fit
     self._fit(reference_data, timestamps)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/drift/univariate/methods.py", line 292, in _fit
     bins = np.histogram_bin_edges(reference_data, bins='doane')
   File "<__array_function__ internals>", line 200, in histogram_bin_edges
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/numpy/lib/histograms.py", line 669, in histogram_bin_edges
     bin_edges, _ = _get_bin_edges(a, bins, range, weights)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/numpy/lib/histograms.py", line 396, in _get_bin_edges
     first_edge, last_edge = _get_outer_edges(a, range)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/numpy/lib/histograms.py", line 323, in _get_outer_edges
     raise ValueError(
 ValueError: autodetected range of [nan, nan] is not finite
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/runner.py", line 177, in run
     calc.fit(reference_data)
   File "/opt/nannyml-runner/.venv/lib/python3.11/site-packages/nannyml/base.py", line 362, in fit
     raise CalculatorException(f"failed while fitting {str(self)}.\n{exc}")
 nannyml.exceptions.CalculatorException: failed while fitting <nannyml.drift.univariate.calculator.UnivariateDriftCalculator object at 0x7fba9ba21c90>.
 autodetected range of [nan, nan] is not finite

@nnansters nnansters added bug Something isn't working triage Needs to be assessed labels Nov 17, 2023
@nnansters nnansters self-assigned this Nov 17, 2023
@michael-nml michael-nml linked a pull request Nov 20, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Needs to be assessed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant