Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected Wrapping of resourceLogs in an Array #22261

Open
abienkowski opened this issue Jan 21, 2025 · 1 comment
Open

Unexpected Wrapping of resourceLogs in an Array #22261

abienkowski opened this issue Jan 21, 2025 · 1 comment
Labels
sink: opentelemetry Anything `opentelemetry` sink related type: bug A code related bug.

Comments

@abienkowski
Copy link

abienkowski commented Jan 21, 2025

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When using the remap language to construct a JSON object with the key resourceLogs, the sink of opentelemetry type adds an unexpected array wrapper around the object. This causes the payload to fail validation when sent to systems like SigNoz, which expect resourceLogs as part of a single JSON object and not as an array.

Configuration

sources:
  dummy_logs:
    type: "demo_logs"
    format: "syslog"
    interval: 1

# Parse Syslog logs
# See the Vector Remap Language reference for more info: https://vrl.dev
transforms:

  remap_journald:
   inputs:
     - "dummy_logs"
   type: "remap"
   source: |
     .resourceLogs = [{
       "resource": {
         "attributes": [
           { "key": "service.name", "value": { "stringValue": "vector.service" } }
         ]
       },
       "scopeLogs": [{
         "logRecords": [{
           "body": { "stringValue": "This is a test log messagem from prod12" }
         }]
       }]
     }]

     del(.host)
     del(.message)
     del(.service)
     del(.timestamp)
     del(.source_type)

sinks:
  signoz_optl:
    type: "opentelemetry"
    inputs: ["remap_journald"]
    protocol:
      type: "http"
      uri: "http://127.0.0.1:4318/v1/logs"
      method: "post"
      encoding:
        codec: "json"

Version

0.44.0

Debug Output


Example Data

Additional Context

when I capture the http reuqest with wireshark I see there is JSON object is wrapped in an array

"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x08\x00E\x00\x01\xf0\xd4x@\x00@\x06f\x8d\x7f\x00\x00\x01\x7f\x00\x00\x01\x94\xe2\x10\xde\xc5@a\x94(MN{\x80\x18\x02\x00\xff\xe4\x00\x00\x01\x01\x08\x0a\x9b\x92\x9bQ\x9b\x92\x97iPOST /v1/logs HTTP/1.1\x0d\x0acontent-type: application/json\x0d\x0aaccept-encoding: zstd,gzip,deflate,br\x0d\x0auser-agent: Vector/0.44.0 (x86_64-unknown-linux-gnu 3cdc7c3 2025-01-13 21:26:04.735691656)\x0d\x0ahost: 127.0.0.1:4318\x0d\x0acontent-length: 212\x0d\x0a\x0d\x0a[{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"vector.service"}}]},"scopeLogs":[{"logRecords":[{"body":{"stringValue":"This is a test log messagem from prod12"}}]}]}]}]"

References

No response

@abienkowski abienkowski added the type: bug A code related bug. label Jan 21, 2025
@pront
Copy link
Member

pront commented Jan 22, 2025

Quick check, can you try adding this to your sink?

framing:
  method: newline_delimited

@pront pront added the sink: opentelemetry Anything `opentelemetry` sink related label Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sink: opentelemetry Anything `opentelemetry` sink related type: bug A code related bug.
Projects
None yet
Development

No branches or pull requests

2 participants