feat: Add the otelcol syslog receiver #2263

dehaansa · 2024-12-11T18:37:47Z

PR Description

While adding the otelcol.exporter.syslog component, I discovered that there was not a clean way to proxy syslog using the loki.source.syslog component as a source. For the purposes of syslog proxying, the otelcol.receiver.syslog component emits logs in the exact syntax expected by the exporter.

This does not add support for enhancing parsing with the numerous stanza operators available in upstream opentelemetry. Most or all of the use cases that would be supported by those can also be implemented through processors like the otelcol.processor.transform component, and the effort to add them in seemed significant for no known customer requirements at this time.

Which issue(s) this PR fixes

Related to #312

Notes to the Reviewer

PR Checklist

CHANGELOG.md updated
Documentation added
Tests updated
Config converters updated

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

…loy into feat/otelcol-syslog-receiver

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

mattdurham · 2024-12-12T14:44:15Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+The `location` argument specifies a Time Zone identifier. The available locations depend on the local IANA Time Zone database.
+See [this wikipedia entry][tz-wiki] for a non-comprehensive list.
+
+The `non_transparent_framing_trailer` argument must be one of `LF`, `NUL`.


Can we add some context around this? Like how can the default be nil? Even if its a string it should be "" but that seems to not be LF or NUL.

Yeah, that's fair. It's a *string, so it's nil if not set. If it is set, then the character chosen is expected to terminate the message. If not set, it's expected that you're using octet counting or that the syslog frame contains only one message.

@clayton-cornell I'd appreciate any feedback you have on this docs change as well as Matt's.

What's in the docs makes sense to me. I don't know the specific nuances of a string data type and a value of nil though. Whats' there seems to make sense as I read it.

mattdurham · 2024-12-12T14:46:46Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+| Name                            | Type     | Description                                                                                                  | Default | Required |
+|---------------------------------|----------|--------------------------------------------------------------------------------------------------------------|---------|----------|
+| `listen_address`                | `string` | The `<host:port>` address to listen to for syslog messages.                                                  |         | yes      |
+| `max_log_size`                  | `int`    | The maximum size of a log entry to read before failing.                                                      | `1MiB`  | no       |


This should be a string.

mattdurham · 2024-12-12T14:48:34Z

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

+| Name               | Type       | Description                                                                                               | Default      | Required |
+|--------------------|------------|-----------------------------------------------------------------------------------------------------------|--------------|----------|
+| `enabled`          | `bool`     | If true, the receiver will pause reading a file and attempt to resend the current batch of logs on error. | `false`      | no       |
+| `initial_interval` | `duration` | The time to wait after first failure to retry.                                                            | `1 second`   | no       |


Durations should be in formatted to 1s or 5m

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

mattdurham · 2024-12-12T14:53:13Z

internal/component/otelcol/receiver/syslog/syslog.go

+}
+
+// Values taken from tcp input Build function
+const defaultMaxLogSize = helper.ByteSize(tcp.DefaultMaxLogSize)


nit: Rename to tcpDefaultMaxLogSize

mattdurham

Looks good aside from a few doc changes.

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

clayton-cornell

The updated description makes sense when I read it.

In reviewing again, I found a couple other minor points I missed in the first pass.

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md

clayton-cornell · 2024-12-12T18:01:18Z

Ooops double posted my review. I was logged out Git right when I submitted the review.

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

clayton-cornell

Approving for docs

mattdurham

Lgtm, looks like a test is failing but will restart

dehaansa added 5 commits December 11, 2024 10:12

Implementation of otelcol syslog receiver wrapper

1aec317

update unmarshal code

69a93f3

Merge branch 'main' into feat/otelcol-syslog-receiver

306a43f

Add changelog entry for syslogreceiver

ad69096

Change field key, add converter

cefb4c4

dehaansa requested review from clayton-cornell and a team as code owners December 11, 2024 18:37

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

dehaansa and others added 5 commits December 11, 2024 14:33

Apply suggestions from code review

587d4a1

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update whitespace attribute descriptions

85b2cd0

Clean up commented out code

b22ebfb

Clean up comments

520eeea

Update autogenerated docs

ad716de

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Dec 11, 2024

dehaansa and others added 4 commits December 11, 2024 14:59

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

d00f726

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

47deaad

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Put a little more maximum delay in syslog test

f79a9ac

clean up validation, try not to use localhost for ci tests

3a438a7

clayton-cornell reviewed Dec 11, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

dehaansa and others added 4 commits December 11, 2024 21:47

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

1072d10

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Fix syslogreceiver converter

27dcf40

Merge branch 'feat/otelcol-syslog-receiver' of github.com:dehaansa/al…

d0e59ac

…loy into feat/otelcol-syslog-receiver

Add test sleep

1d0b32a

clayton-cornell reviewed Dec 12, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Outdated Show resolved Hide resolved

mattdurham reviewed Dec 12, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Show resolved Hide resolved

mattdurham reviewed Dec 12, 2024

View reviewed changes

mattdurham requested changes Dec 12, 2024

View reviewed changes

dehaansa and others added 3 commits December 12, 2024 10:20

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

f2942e4

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Address some PR feedback

ad373b7

Update docs for rfc6587 arguments

70997f7

dehaansa requested a review from mattdurham December 12, 2024 15:59

Add todo to test comment

f278703

clayton-cornell reviewed Dec 12, 2024

View reviewed changes

dehaansa and others added 2 commits December 12, 2024 13:47

Update docs/sources/reference/components/otelcol/otelcol.receiver.sys…

548b173

…log.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Apply suggestions from code review

a9fc448

Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

clayton-cornell approved these changes Dec 12, 2024

View reviewed changes

dehaansa added 3 commits December 24, 2024 22:49

Merge branch 'main' into feat/otelcol-syslog-receiver

d0a0fb8

Merge branch 'main' into feat/otelcol-syslog-receiver

3ee6853

remote timeout from test

14d1788

mattdurham approved these changes Jan 3, 2025

View reviewed changes

dehaansa merged commit 89e17b2 into grafana:main Jan 3, 2025
15 of 16 checks passed

dehaansa deleted the feat/otelcol-syslog-receiver branch January 3, 2025 17:58

aidanleuck mentioned this pull request Jan 4, 2025

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add the otelcol syslog receiver #2263

feat: Add the otelcol syslog receiver #2263

dehaansa commented Dec 11, 2024

mattdurham Dec 12, 2024

dehaansa Dec 12, 2024

dehaansa Dec 12, 2024

clayton-cornell Jan 2, 2025

mattdurham Dec 12, 2024

mattdurham Dec 12, 2024

mattdurham Dec 12, 2024

mattdurham left a comment

clayton-cornell left a comment

clayton-cornell commented Dec 12, 2024

clayton-cornell left a comment

mattdurham left a comment

feat: Add the otelcol syslog receiver #2263

feat: Add the otelcol syslog receiver #2263

Conversation

dehaansa commented Dec 11, 2024

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

mattdurham Dec 12, 2024

Choose a reason for hiding this comment

dehaansa Dec 12, 2024

Choose a reason for hiding this comment

dehaansa Dec 12, 2024

Choose a reason for hiding this comment

clayton-cornell Jan 2, 2025

Choose a reason for hiding this comment

mattdurham Dec 12, 2024

Choose a reason for hiding this comment

mattdurham Dec 12, 2024

Choose a reason for hiding this comment

mattdurham Dec 12, 2024

Choose a reason for hiding this comment

mattdurham left a comment

Choose a reason for hiding this comment

clayton-cornell left a comment

Choose a reason for hiding this comment

clayton-cornell commented Dec 12, 2024

clayton-cornell left a comment

Choose a reason for hiding this comment

mattdurham left a comment

Choose a reason for hiding this comment