From 34fc871f0c756061a6a254f5152aea0262349157 Mon Sep 17 00:00:00 2001 From: mattdurham Date: Tue, 14 Jan 2025 11:35:57 -0500 Subject: [PATCH] Update 1.6.0 to rc1 (#2402) * update changelog for rc (#2360) * update changelog for rc * update changelog for rc * update changelog for rc (#2361) * update version (#2362) * update changelog for rc (#2360) * update changelog for rc * update changelog for rc Signed-off-by: matt durham * fix conversion * Fix changelog main (#2364) * update version * Fix changelog * update the image version to work with the given example (#2358) * docs: fixed kafka config example (#2359) Example shows `loki.source.kafka "local"` pointing to `loki.relabel.kafka.receiver`. This leads to no new label being added. Correct example should have the kafka source pointing directly to `loki.write.local.receiver` * feat(helm): add the ability to deploy extra manifest files (#2347) * feat(helm): add the ability to deploy extra manifest files * docs(helm): run helm-docs * ci(helm): add tests * Update wal queue tls (#2363) * add tls to wal * add alloy config * update version * Add support for TLS doc. * Add changelog. * fix import order * add support and doc for round robin. * fix conversion * Update docs/sources/reference/components/prometheus/prometheus.write.queue.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Add test * fix merge * Update internal/component/prometheus/write/queue/types.go Co-authored-by: William Dumont --------- Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> Co-authored-by: William Dumont * #229 Add OpenTelemetry Collector Server Auth Extensions to Receivers (#2203) * Work on adding auth so far * Cleanup * Made a ton of progress * Fix test fails? * Refactor * Add auth blocks to implementing extensions * Refactor to use feature flag * Comments * Cleanup * Spacing * Update docs * Update CHANGELOG * Last auth extension missing * We also need grpc auth * Fix opencensus docs * Fix extra comment * Update comment with findings * Properly fix merge conflict * Save file * Spelling error * That has been released now * Add auth support to influxdb receiver * Fix failing auth test/MAIL * Comment cleanup * MAIL for documentation * docs MAIL * MAIL * Move from Auth to Authentication * Update triton-go dependency to avoid embedded RSA key (#2380) * Fix examples for filter and transform processors (#2379) * fix examples filter and transform processors * remove unecessary docs about escaping strings and backticks * fix(loki.secretfilter): Fix partial masking for short secrets and support multiple allowlists per rule (#2320) * Fix partial masking bug and support new allowlist format * Add docs and changelog * Update docs * Add comments * Add comments * Minor docs update * Add more tests * Change criteria for partial redaction * Changes to partial masking rules * Fix comment location * Clarify usage of secret types * Clarify usage of secret types * Update docs/sources/reference/components/loki/loki.secretfilter.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Suggestions * Suggestions --------- Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Fix only run on fork guard (#2378) * Fix only run on fork guard The previous guard fails because `github.repository` resolves to the base repository on `pull_request` events. * Fix syntax * Fix relabel processed bug (#2394) * Fix issue where alloy_prometheus_relabel_metrics_processed was not being incremented. * Add unit tests * Update WAL to version that supports v2. (#2397) * Update WAL to version that supports v2. * Update WAL to version that supports v2. * Add samples check. * Clean up Alloy component docs (#2387) * First pass at cleanup, pretty tables, sort lists * Sort content, add badge * Fix link * Set link URL correctly * Still fxing link targets * One more tidy pass * database_observability: report health of component and collectors (#2392) Report unhealthy in case of errors when starting up the collectors or of any collector is stopped during operations. * update for rc.1 (#2401) * Update version. * fix version * fix version --------- Signed-off-by: matt durham Co-authored-by: Adam ABICHOU <55128590+adamab48@users.noreply.github.com> Co-authored-by: Jay Clifford <45856600+Jayclifford345@users.noreply.github.com> Co-authored-by: dbluxo <20679842+dbluxo@users.noreply.github.com> Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> Co-authored-by: William Dumont Co-authored-by: Aidan Leuck <48965785+aidanleuck@users.noreply.github.com> Co-authored-by: Sam DeHaan Co-authored-by: Romain Gaillard Co-authored-by: Jack Baldry Co-authored-by: Cristian Greco --- .github/workflows/deploy-pr-preview.yml | 2 +- CHANGELOG.md | 26 +- .../reference/components/beyla/beyla.ebpf.md | 220 ++++++++--------- .../components/loki/loki.secretfilter.md | 13 +- .../components/loki/loki.source.kafka.md | 2 +- .../components/otelcol/otelcol.auth.basic.md | 2 +- .../components/otelcol/otelcol.auth.bearer.md | 2 +- .../otelcol/otelcol.auth.headers.md | 2 + .../components/otelcol/otelcol.auth.oauth2.md | 2 + .../components/otelcol/otelcol.auth.sigv4.md | 2 + ...telcol.extension.jaeger_remote_sampling.md | 28 +++ .../otelcol/otelcol.processor.filter.md | 28 +-- .../otelcol/otelcol.processor.transform.md | 26 +- .../otelcol/otelcol.receiver.datadog.md | 25 ++ .../otelcol/otelcol.receiver.influxdb.md | 24 ++ .../otelcol/otelcol.receiver.jaeger.md | 33 +++ .../otelcol/otelcol.receiver.opencensus.md | 22 ++ .../otelcol/otelcol.receiver.otlp.md | 31 +++ .../otelcol/otelcol.receiver.zipkin.md | 21 ++ .../prometheus/prometheus.write.queue.md | 41 ++-- example/compose.yaml | 2 +- go.mod | 11 +- go.sum | 38 ++- .../mysql/collector/connection_info.go | 15 ++ .../mysql/collector/query_sample.go | 11 +- .../mysql/collector/schema_table.go | 31 ++- .../database_observability/mysql/component.go | 54 ++++- .../faro/receiver/receiver_otelcol_test.go | 2 +- .../loki/secretfilter/secretfilter.go | 60 +++-- .../loki/secretfilter/secretfilter_test.go | 182 ++++++++++++++ internal/component/otelcol/auth/auth.go | 224 ++++++++++++++++-- internal/component/otelcol/auth/auth_test.go | 182 +++++++++++++- .../component/otelcol/auth/basic/basic.go | 22 +- .../otelcol/auth/basic/basic_test.go | 140 +++++++++-- .../component/otelcol/auth/bearer/bearer.go | 18 +- .../otelcol/auth/bearer/bearer_test.go | 133 +++++++++-- .../component/otelcol/auth/headers/headers.go | 15 +- .../otelcol/auth/headers/headers_test.go | 9 +- .../component/otelcol/auth/oauth2/oauth2.go | 14 +- .../otelcol/auth/oauth2/oauth2_test.go | 7 +- .../component/otelcol/auth/sigv4/sigv4.go | 16 +- .../otelcol/auth/sigv4/sigv4_test.go | 8 +- internal/component/otelcol/config_grpc.go | 78 ++++-- internal/component/otelcol/config_http.go | 75 ++++-- .../exporter/loadbalancing/loadbalancing.go | 58 +++-- .../component/otelcol/exporter/otlp/otlp.go | 7 +- .../otelcol/exporter/otlphttp/otlphttp.go | 7 +- .../jaeger_remote_sampling.go | 44 +++- .../otelcol/receiver/datadog/datadog.go | 9 +- .../otelcol/receiver/influxdb/influxdb.go | 76 +++--- .../otelcol/receiver/jaeger/jaeger.go | 41 +++- .../otelcol/receiver/opencensus/opencensus.go | 8 +- .../component/otelcol/receiver/otlp/otlp.go | 48 +++- .../otelcol/receiver/zipkin/zipkin.go | 8 +- .../component/prometheus/relabel/relabel.go | 2 + .../prometheus/relabel/relabel_test.go | 12 + .../prometheus/write/queue/e2e_test.go | 12 +- .../component/prometheus/write/queue/types.go | 13 + .../prometheus/write/queue/types_test.go | 20 ++ .../converter_loadbalancingexporter.go | 2 +- .../otelcolconvert/converter_otlpexporter.go | 2 +- .../converter_otlphttpexporter.go | 2 +- .../runtime/componenttest/componenttest.go | 10 + operations/helm/charts/alloy/CHANGELOG.md | 1 + operations/helm/charts/alloy/README.md | 1 + .../alloy/ci/extra-manifests-values.yaml | 8 + .../alloy/templates/extra-manifests.yaml | 4 + operations/helm/charts/alloy/values.yaml | 10 + .../alloy/templates/configmap.yaml | 43 ++++ .../templates/controllers/daemonset.yaml | 75 ++++++ .../alloy/templates/extra-manifests.yaml | 9 + .../extra-manifests/alloy/templates/rbac.yaml | 119 ++++++++++ .../alloy/templates/service.yaml | 24 ++ .../alloy/templates/serviceaccount.yaml | 14 ++ 74 files changed, 2143 insertions(+), 445 deletions(-) create mode 100644 internal/component/prometheus/write/queue/types_test.go create mode 100644 operations/helm/charts/alloy/ci/extra-manifests-values.yaml create mode 100644 operations/helm/charts/alloy/templates/extra-manifests.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/configmap.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/controllers/daemonset.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/extra-manifests.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/rbac.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/service.yaml create mode 100644 operations/helm/tests/extra-manifests/alloy/templates/serviceaccount.yaml diff --git a/.github/workflows/deploy-pr-preview.yml b/.github/workflows/deploy-pr-preview.yml index 983b5d57e7..0e76de64fa 100644 --- a/.github/workflows/deploy-pr-preview.yml +++ b/.github/workflows/deploy-pr-preview.yml @@ -11,7 +11,7 @@ on: jobs: deploy-pr-preview: - if: github.repository == 'grafana/alloy' + if: ${{ ! github.event.pull_request.head.repo.fork }} uses: grafana/writers-toolkit/.github/workflows/deploy-preview.yml@main with: sha: ${{ github.event.pull_request.head.sha }} diff --git a/CHANGELOG.md b/CHANGELOG.md index b64a9106cb..3db7330b59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ This document contains a historical list of changes between releases. Only changes that impact end-user behavior are listed; changes to documentation or internal API changes are not present. -v1.6.0-rc.0 +v1.6.0-rc.1 ----------------- ### Breaking changes @@ -21,6 +21,8 @@ v1.6.0-rc.0 ### Features +- Add support for TLS to `prometheus.write.queue`. (@mattdurham) + - Add `otelcol.receiver.syslog` component to receive otel logs in syslog format (@dehaansa) - Add support for metrics in `otelcol.exporter.loadbalancing` (@madaraszg-tulip) @@ -43,12 +45,18 @@ v1.6.0-rc.0 ### Enhancements +- Update `prometheus.write.queue` to support v2 for cpu performance. (@mattdurham) + +- (_Experimental_) Add health reporting to `database_observability.mysql` component (@cristiangreco) + - Add second metrics sample to the support bundle to provide delta information (@dehaansa) - Add all raw configuration files & a copy of the latest remote config to the support bundle (@dehaansa) - Add relevant golang environment variables to the support bundle (@dehaansa) +- Add support for server authentication to otelcol components. (@aidaleuc) + - Update mysqld_exporter from v0.15.0 to v0.16.0 (including 2ef168bf6), most notable changes: (@cristiangreco) - Support MySQL 8.4 replicas syntax - Fetch lock time and cpu time from performance schema @@ -72,12 +80,16 @@ v1.6.0-rc.0 - Update `prometheus.write.queue` library for performance increases in cpu. (@mattdurham) +- Update `loki.secretfilter` to be compatible with the new `[[rules.allowlists]]` gitleaks allowlist format (@romain-gaillard) + - Update `async-profiler` binaries for `pyroscope.java` to 3.0-fa937db (@aleks-p) - Reduced memory allocation in discovery components by up to 30% (@thampiotr) ### Bugfixes +- Fix issue where `alloy_prometheus_relabel_metrics_processed` was not being incremented. (@mattdurham) + - Fixed issue with automemlimit logging bad messages and trying to access cgroup on non-linux builds (@dehaansa) - Fixed issue with reloading configuration and prometheus metrics duplication in `prometheus.write.queue`. (@mattdurham) @@ -88,6 +100,8 @@ v1.6.0-rc.0 - Fixed an issue where the `otelcol.processor.interval` could not be used because the debug metrics were not set to default. (@wildum) +- Fixed an issue where `loki.secretfilter` would crash if the secret was shorter than the `partial_mask` value. (@romain-gaillard) + - Change the log level in the `eventlogmessage` stage of the `loki.process` component from `warn` to `debug`. (@wildum) - Fix a bug in `loki.source.kafka` where the `topics` argument incorrectly used regex matching instead of exact matches. (@wildum) @@ -111,11 +125,11 @@ v1.6.0-rc.0 https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36071 - `otelcol.exporter.datadog`: Stop prefixing `http_server_duration`, `http_server_request_size` and `http_server_response_size` with `otelcol`. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36265 - These metrics can be from SDKs rather than collector. Stop prefixing them to be consistent with + These metrics can be from SDKs rather than collector. Stop prefixing them to be consistent with https://opentelemetry.io/docs/collector/internal-telemetry/#lists-of-internal-metrics - `otelcol.receiver.datadog`: Add json handling for the `api/v2/series` endpoint in the datadogreceiver. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36218 - - `otelcol.processor.span`: Add a new `keep_original_name` configuration argument + - `otelcol.processor.span`: Add a new `keep_original_name` configuration argument to keep the original span name when extracting attributes from the span name. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36397 - `pkg/ottl`: Respect the `depth` option when flattening slices using `flatten`. @@ -123,14 +137,14 @@ v1.6.0-rc.0 https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36198 - `otelcol.exporter.loadbalancing`: Shutdown exporters during collector shutdown. This fixes a memory leak. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36024 - - `otelcol.processor.k8sattributes`: New `wait_for_metadata` and `wait_for_metadata_timeout` configuration arguments, + - `otelcol.processor.k8sattributes`: New `wait_for_metadata` and `wait_for_metadata_timeout` configuration arguments, which block the processor startup until metadata is received from Kubernetes. https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32556 - - `otelcol.processor.k8sattributes`: Enable the `k8sattr.fieldExtractConfigRegex.disallow` for all Alloy instances, + - `otelcol.processor.k8sattributes`: Enable the `k8sattr.fieldExtractConfigRegex.disallow` for all Alloy instances, to retain the behavior of `regex` argument in the `annotation` and `label` blocks. When the feature gate is "deprecated" in the upstream Collector, Alloy users will need to use the transform processor instead. https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25128 - - `otelcol.receiver.vcenter`: The existing code did not honor TLS settings beyond 'insecure'. + - `otelcol.receiver.vcenter`: The existing code did not honor TLS settings beyond 'insecure'. All TLS client config should now be honored. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/36482 - `otelcol.receiver.opencensus`: Do not report error message when OpenCensus receiver is shutdown cleanly. diff --git a/docs/sources/reference/components/beyla/beyla.ebpf.md b/docs/sources/reference/components/beyla/beyla.ebpf.md index 2e19242f02..690f389273 100644 --- a/docs/sources/reference/components/beyla/beyla.ebpf.md +++ b/docs/sources/reference/components/beyla/beyla.ebpf.md @@ -21,7 +21,6 @@ To run this component, {{< param "PRODUCT_NAME" >}} requires administrative (`su In Kubernetes environments, the [AppArmor profile must be `Unconfined`](https://kubernetes.io/docs/tutorials/security/apparmor/#securing-a-pod) for the Deployment or DaemonSet running {{< param "PRODUCT_NAME" >}}. {{< /admonition >}} - ## Usage ```alloy @@ -36,52 +35,70 @@ beyla.ebpf "