Issue 481 - implement support for missing values with XGBoost #482
Conversation
|
Tagging @styrmis @wrigleyDan for awareness. |
patrick-le-shopify
changed the title
There was a problem hiding this comment.
Thank you for this! I like the approach—I've added just one comment. It was the explain output that in the end clarified the behaviour of the model inference in the plugin for me, so I think it would be worth maintaining the explicit reporting of default values used in that output if possible.
| @@ -259,7 +259,7 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio | |||
| } | |||
| featureString += ":"; | |||
| if (!explain.isMatch()) { | |||
| subs.add(Explanation.noMatch(featureString + " [no match, default value 0.0 used]")); | |||
| subs.add(Explanation.noMatch(featureString + " [no match, default value used]")); | |||
There was a problem hiding this comment.
As the feature vector implementations can return their default value (which may be NaN), could we report this here?
Without this information it would require careful inspection of the source to determine what the default is, where reporting it in the explain output would be more user friendly.
There was a problem hiding this comment.
Good call! I've made changes to add what the default value is to the explanation.
This PR attempts to implement support for missing values with XGBoost (#481).
The main logic change pertains to the initialization of the FeatureVector. As of now, the DenseFeatureVector has a default value of 0.0, and feature scores are filled in with actual values only when they are present. Effectively, features which are missing are given a value of 0.0. This happens "upstream," and by the time XGBoost model (NaiveAdditiveDecisionTree) is invoked, there is no longer any missing feature value.
The implementation here adds a new class called SparseFeatureVector that gives features a default value of Float.NaN, enabling XGBoost model to actually follow branches where the feature is missing.