Add callbacks

[brettshollenberger]

Hey Andrew! Thanks for this library 😄

I wanted to add the callbacks API so I could support an integration with Wandb

I have the Ruby implementation of this callback over here as an example of the use case and have tested the integration in my Wandb console

Some dummy data just to show the integration:

I kept the API consistent w/ the Python implementation but let me know if there's anything else you'd want to see here

[ankane]

Hey Brett, great to hear from you! This looks pretty neat. It looks like the Python API uses the return values of the callbacks in train. I think the best way to do that would be to add TrainingCallback and CallbackContainer classes like Python (code), but I can do that in a follow-up commit if you just want to get the tests passing.

[brettshollenberger]

Good idea! Let me know what you think, should be passing now

[ankane]

Awesome, thanks for adding those classes. Added some comments inline. It looks like the new files need to be required.

require_relative "xgboost/callback_container"
require_relative "xgboost/training_callback"

Also, if you're seeing an error running the tests locally, try running:

bundle exec rake vendor:platform

[ankane]

This should update booster to match Python.

[ankane]

This should break if the return value is falsy.

[ankane]

Please keep the existing code where possible to keep the changeset minimal / easier to review.

[brettshollenberger]

Sorry, disabled rubocop

[ankane]

This should break for falsy values like before_iteration.

[ankane]

This should update booster like before_training.

[ankane]

callback.is_a?(TrainingCallback)

[ankane]

Let's create a separate file for the callback tests.

[brettshollenberger]

Thanks Andrew, had to get rid of some aggressive Rubocop settings complicating the changes. Tests are all passing for me and everything's committed now 😅 Looks like you have to approve the Github workflow run here but I think we should have it this time.

[ankane]

Thanks @brettshollenberger! Merged in the commit above with a few minor changes:

1. I had the logic backwards for before/after_iteration - it should stop if it returns a truthy value
2. Changed callbacks to use positional arguments instead of keyword
3. Left out the params change, as it's not present in the Python library (from what I can tell)

Going to spend a little time getting the overall code more in sync with Python, and then will push a new release.

[brettshollenberger]

@ankane nice, thanks!

With the params thing, it's not exactly 1:1, but I think an easier solution than the Python Wandb callback, which calls xgboost/core#save_config.

https://github.com/wandb/wandb/blob/8698af5862e44baf31af5411b81bea546e069257/wandb/integration/xgboost/xgboost.py#L117

    def before_training(self, model: Booster) -> Booster:
        """Run before training is finished."""
        # Update W&B config
        config = model.save_config()
        wandb.config.update(json.loads(config))

        return model

https://github.com/dmlc/xgboost/blob/3f9bfaf86e6db6a4f54734aa7d164df55aa69ef6/python-package/xgboost/core.py#L2008

    def save_config(self) -> str:
        """Output internal parameter configuration of Booster as a JSON
        string.

        .. versionadded:: 1.0.0

        """
        json_string = ctypes.c_char_p()
        length = c_bst_ulong()
        _check_call(
            _LIB.XGBoosterSaveJsonConfig(
                self.handle, ctypes.byref(length), ctypes.byref(json_string)
            )
        )
        assert json_string.value is not None
        result = json_string.value.decode()  # pylint: disable=no-member
        return result

[ankane]

I think it's better to keep things synced for maintainability in most cases. Added save_config in the commit above.

[brettshollenberger]

Awesome, thank you!

[ankane]

Great, just pushed 0.9.0. Let me know if you need anything else that's missing.