Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt log shipping via DevX logs #789

Merged
merged 4 commits into from
Nov 13, 2023
Merged

Adopt log shipping via DevX logs #789

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
70016f0
feat: Configure log shipping at the infrastructure level
akash1810 Nov 9, 2023
19a7da0
chore(deps): Remove `com.gu.kinesis-logback-appender`
akash1810 Nov 9, 2023
0f21143
fix: Log to stdout
akash1810 Nov 9, 2023
34bac60
Merge branch 'main' into aa/devx-logs
JuliaBrigitte Nov 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 0 additions & 18 deletions app/AppLoader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
import com.amazonaws.regions.Regions
import conf._
import play.api.{Configuration, _}
import software.amazon.awssdk.regions.internal.util.EC2MetadataUtils
import utils.{AWSCredentialProviders, Logging}

import scala.util.Try

class AppLoader extends ApplicationLoader with Logging {
def load(context: ApplicationLoader.Context): Application = {
LoggerConfigurator(context.environment.classLoader).foreach {
Expand Down Expand Up @@ -34,21 +31,6 @@ class AppLoader extends ApplicationLoader with Logging {
Configuration.empty
)(_.configuration(context.environment.mode).withFallback(_))

val stream: Option[String] =
extraConfig.getOptional[String]("LoggingStream")
stream match {
case Some(stream) =>
val maybeInstanceId = Try(EC2MetadataUtils.getInstanceId).toOption
val loggingContext = Map(
"buildId" -> prism.BuildInfo.buildNumber
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change drops this log marker.

) ++ maybeInstanceId.map("instanceId" -> _)
LogConfiguration.shipping(stream, identity, loggingContext)
case _ =>
log.info(
"Missing stream configuration to enable log shipping to central ELK"
)
}

log.info(s"Loaded config $extraConfig")

val combinedConfig: Configuration =
Expand Down
60 changes: 0 additions & 60 deletions app/conf/LogConfiguration.scala
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ lazy val root = (project in file("."))
filters,
specs2 % "test",
"net.logstash.logback" % "logstash-logback-encoder" % "7.4" exclude ("com.fasterxml.jackson.core", "jackson-databind"),
"com.gu" % "kinesis-logback-appender" % "2.1.1",
"com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.15.3"
),
excludeDependencies ++= Seq(
Expand Down
5 changes: 5 additions & 0 deletions cdk/lib/__snapshots__/prism-ec2-app.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ exports[`The PrismEc2App stack matches the snapshot 1`] = `
"PropagateAtLaunch": true,
"Value": "PROD",
},
{
"Key": "SystemdUnit",
"PropagateAtLaunch": true,
"Value": "prism.service",
},
],
"TargetGroupARNs": [
{
Expand Down
3 changes: 3 additions & 0 deletions cdk/lib/prism-ec2-app.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ export class PrismEc2App extends GuStack {

const pattern = new GuPlayApp(this, {
...PrismEc2App.app,
applicationLogging: {
enabled: true,
},
instanceType: InstanceType.of(InstanceClass.T4G, InstanceSize.MEDIUM),
userData: {
distributable: {
Expand Down
5 changes: 5 additions & 0 deletions conf/logback.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,13 @@
</encoder>
</appender>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
Comment on lines +18 to +20
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logs written to stdout are collected by the journal daemon, which in turn is forwarded to Central ELK by devx-logs.


<root level="INFO">
<appender-ref ref="LOGFILE"/>
<appender-ref ref="STDOUT"/>
</root>

</configuration>