Adopt log shipping via DevX logs #789
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
akash1810
force-pushed
the
aa/devx-logs
branch
from
f137d73 to
c6fc9b7
Compare
akash1810
commented
Nov 9, 2023
Comment on lines
+18
to
+20
| <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> | ||
| <encoder class="net.logstash.logback.encoder.LogstashEncoder" /> | ||
| </appender> |
There was a problem hiding this comment.
Logs written to stdout are collected by the journal daemon, which in turn is forwarded to Central ELK by devx-logs.
akash1810
commented
Nov 9, 2023
| case Some(stream) => | ||
| val maybeInstanceId = Try(EC2MetadataUtils.getInstanceId).toOption | ||
| val loggingContext = Map( | ||
| "buildId" -> prism.BuildInfo.buildNumber |
There was a problem hiding this comment.
This change drops this log marker.
Since adopting GuCDK's application log shipping capability, this dependency is no longer needed.
Logs written to stdout are picked up by the journal daemon. FluentBit is configured to forward journal logs to Central ELK.
akash1810
force-pushed
the
aa/devx-logs
branch
from
c6fc9b7 to
0f21143
Compare
JuliaBrigitte
approved these changes
Nov 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this change?
Adopts https://github.com/guardian/devx-logs to ship application logs to Central ELK. This change allows us to remove an application dependency (https://github.com/guardian/kinesis-logback-appender), ultimately reducing the volume of code in the repository.
How to test
DevX Logs adds a
ShippedBymarker to each log line. Therefore, to demonstrate this is working, we can deploy and view the logs1:How can we measure success?
Fewer things to maintain in this repository.
Footnotes
The screenshot is limited to "healthcheck" logs, as other application logs contain account IDs. ↩