Json formatting option for logs or the ability to add local volumes to conatiners to overwrite logging config

[masmontanas]

Hello,

It would be great if there was an option to format the log data in json -- primary reason being logging/monitoring tools in the k8s ecosystem will automatically tokenize json formatted log data, eliminating the need to set up parsing rules for field-tokenization and new-line delimiters.

Reading through the project, it looks like the logging module's config can be overwritten with a file (exposed as an env variable), though the helm chart does not expose a volumes/volumeMounts config.

We'd be happy to put in this contribution ourselves (either approach) if the team is receptive to a PR, otherwise either option would be great -- though it might be more intuitive to expose an option to select the json formatter in a clear manner through the helm chart (ex: jsonLogging: enabled).

Thanks!

[dabeeeenster]

Yep nice idea - happy to receive a PR on this.

[masmontanas]

Hi, I was thinking of making the following changes instead of just exposing a volume/voluemMount config in the helm chart (which seemed a bit too blunt and cryptic for the intended change).

...
LOGGING_FORMATTER = env.str("LOGGING_FORMAT", default='generic')
LOG_LEVEL = env.str("LOG_LEVEL", default="WARNING")
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"generic": {"format": "%(name)-12s %(levelname)-8s %(message)s"},
"json": {"format": '{"time": "%(asctime)s", "level": "%(levelname)s", "message": "%(message)s"'}
},
"handlers": {
"console": {
"level": LOG_LEVEL,
"class": "logging.StreamHandler",
"formatter": LOGGING_FORMATTER
},
...

Then one new key/config can be introduced in the helm values to just set the environment variable for the logging format, with the default being current behavior ('generic' formatter).

If this approach seems like an ok pattern for the team we can likely have the PR in tomorrow if not the next day.

[masmontanas]

PR submitted. Unfortunately we ran into issue with the axes middleware emitting illegal and were only able to work around the corner-cases by adding a custom formatter. This is working consistently though it's a bit more heavy-handed than we would've liked. My colleague will put in the PR for the changes to the helm chart and project docs in a separate PR.