Skip to content

Commit

Permalink
Merge pull request #66 from grafana/rename-syntax-submodule
Browse files Browse the repository at this point in the history
Rename github.com/grafana/river submodule to github.com/grafana/alloy/syntax
  • Loading branch information
rfratto authored Mar 25, 2024
2 parents 0b467ee + b13c9b3 commit ad62898
Show file tree
Hide file tree
Showing 342 changed files with 1,158 additions and 1,266 deletions.
40 changes: 20 additions & 20 deletions docs/developer/writing-exporter-flow-components.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Create Prometheus Exporter Flow Components

This guide will walk you through the process of creating a new Prometheus exporter Flow component and best practices for implementing it.
This guide will walk you through the process of creating a new Prometheus exporter Flow component and best practices for implementing it.

It is required that the exporter has an existing [Agent integration](../sources/static/configuration/integrations/_index.md) in order to wrap it as a Flow component. In the future, we will drop this requirement and Flow components will expose the logic of the exporter directly.

Expand All @@ -14,10 +14,10 @@ Use the following exporters as a reference:
`Arguments` struct defines the arguments that can be passed to the component. In most cases, this would be exactly the same as the arguments that the integration for this exporter uses. Some recommendations:

- Use `attr` tag for representing values. Use `attr,optional` tag for optional arguments.
- Use `rivertypes.Secret` type for sensitive arguments (e.g. API keys, passwords, etc). The original integration should have a similar field type called `Secret` from Prometheus.
- Use `alloytypes.Secret` type for sensitive arguments (e.g. API keys, passwords, etc). The original integration should have a similar field type called `Secret` from Prometheus.
- Use `block` tag for representing nested values such slices or structs. For example, the [process_exporter](../../component/prometheus/exporter/process/process.go) `Arguments` struct has `ProcessExporter` param which is a `[]MatcherGroup`. The name of the parameter should be in singular. This will allow the user to define multiple blocks of the same type.

The river config would look like this using `matcher` block multiple times:
The config would look like this using `matcher` block multiple times:

```river
prometheus.exporter.process "example" {
Expand All @@ -27,42 +27,42 @@ prometheus.exporter.process "example" {
}
matcher {
comm = ["firefox"]
}
}
}
```

- Use `label` tag in field of struct represented as block to define named blocks. For example, the [blackbox_exporter](../../component/prometheus/exporter/blackbox/blackbox.go) `BlackboxTarget` struct has a `Name` param which represents the name of the block.
- Use `label` tag in field of struct represented as block to define named blocks. For example, the [blackbox_exporter](../../component/prometheus/exporter/blackbox/blackbox.go) `BlackboxTarget` struct has a `Name` param which represents the name of the block.

The river config would look like this:
The config would look like this:

```river
prometheus.exporter.blackbox "example" {
config_file = "blackbox_modules.yml"
target {
name = "example"
address = "http://example.com"
module = "http_2xx"
}
prometheus.exporter.blackbox "example" {
config_file = "blackbox_modules.yml"
target {
name = "example"
address = "http://example.com"
module = "http_2xx"
}
}
```

- Define `DefaultArguments` as a global variable to define the default arguments for the component.
- Define `DefaultArguments` as a global variable to define the default arguments for the component.

## Functions

- Define `init` function to register the component using `component.Register`.
- Define `init` function to register the component using `component.Register`.
- The `Build` param should be a function that returns a `component.Component` interface.
- The name used in the second parameter of `exporter.New` when defining the `Build` function it's important as it will define the label `job` in the form of `integrations/<name>`.
- Avoid creating components with `Singleton: true` as it will make it impossible to run multiple instances of the exporter.
- Avoid creating components with `Singleton: true` as it will make it impossible to run multiple instances of the exporter.

- If the exporter follows the multi-target pattern, add a function to define Prometheus discovery targets and use `exporter.NewWithTargetBuilder` for the `Build` param of the `component.Register` function.

- If the exporter implements a custom `InstanceKey`, add a function to customize the value of the instance label and use `exporter.NewWithTargetBuilder` for the `Build` param of the `component.Register` function.

- Define the `SetToDefault` function implementing river.Defaulter to specify the default arguments for the component.
- Define the `SetToDefault` function implementing syntax.Defaulter to specify the default arguments for the component.

- Define the `Validate` function implementing river.Validator to specify any validation rules for the component arguments.
- Define the `Validate` function implementing syntax.Validator to specify any validation rules for the component arguments.

- Add a test to validate the unmarshalling covering as many cases as possible.

Expand All @@ -74,4 +74,4 @@ In order to make the component visible for Agent Flow, it needs to be added to [

## Documentation

Writing the documentation for the component is very important. Please, follow the [Writing documentation for Flow components](./writing-flow-component-documentation.md) and take a look at the existing documentation for other exporters.
Writing the documentation for the component is very important. Please, follow the [Writing documentation for Flow components](./writing-flow-component-documentation.md) and take a look at the existing documentation for other exporters.
20 changes: 8 additions & 12 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,12 @@ require (
github.com/grafana/pyroscope/api v0.4.0
github.com/grafana/pyroscope/ebpf v0.4.3
github.com/grafana/regexp v0.0.0-20221123153739-15dc172cd2db
github.com/grafana/river v0.3.1-0.20240123144725-960753160cd1
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20221213150626-862cad8e9538
github.com/grafana/tail v0.0.0-20230510142333-77b18831edf0
github.com/grafana/vmware_exporter v0.0.4-beta
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/hashicorp/consul/api v1.25.1
github.com/hashicorp/go-cleanhttp v0.5.2
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-discover v0.0.0-20230724184603-e89ebd1b2f65
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/golang-lru v1.0.2
Expand All @@ -89,10 +88,9 @@ require (
github.com/json-iterator/go v1.1.12
github.com/klauspost/compress v1.17.4
github.com/lib/pq v1.10.9
github.com/mackerelio/go-osstat v0.2.3
github.com/miekg/dns v1.1.56
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4
github.com/mitchellh/reflectwalk v1.0.2
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
github.com/ncabatoff/process-exporter v0.7.10
github.com/nerdswords/yet-another-cloudwatch-exporter v0.55.0
Expand Down Expand Up @@ -128,9 +126,9 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kafkareceiver v0.87.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver v0.87.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver v0.87.0
github.com/opentracing-contrib/go-grpc v0.0.0-20210225150812-73cb765af46e
github.com/opentracing-contrib/go-grpc v0.0.0-20210225150812-73cb765af46e // indirect
github.com/opentracing-contrib/go-stdlib v1.0.0 // indirect
github.com/opentracing/opentracing-go v1.2.0
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/ory/dockertest/v3 v3.8.1
github.com/oschwald/geoip2-golang v1.9.0
github.com/percona/mongodb_exporter v0.39.1-0.20230706092307-28432707eb65
Expand Down Expand Up @@ -302,7 +300,7 @@ require (
github.com/beevik/ntp v1.3.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/semver v3.5.2-0.20180723201105-3c1074078d32+incompatible // indirect
github.com/blang/semver/v4 v4.0.0 // indirect
github.com/blang/semver/v4 v4.0.0
github.com/boynux/squid-exporter v1.10.5-0.20230618153315-c1fae094e18e
github.com/c2h5oh/datasize v0.0.0-20220606134207-859f65c6625b // indirect
github.com/cenkalti/backoff/v3 v3.0.0 // indirect
Expand Down Expand Up @@ -610,6 +608,7 @@ require (
connectrpc.com/connect v1.14.0
github.com/githubexporter/github-exporter v0.0.0-20231025122338-656e7dc33fe7
github.com/grafana/agent-remote-config v0.0.2
github.com/grafana/alloy/syntax v0.0.0-00010101000000-000000000000
github.com/grafana/jfr-parser/pprof v0.0.0-20240126072739-986e71dc0361
github.com/grafana/jsonparser v0.0.0-20240209175146-098958973a2d
github.com/natefinch/atomic v1.0.1
Expand Down Expand Up @@ -774,7 +773,4 @@ replace github.com/github/smimesign => github.com/grafana/smimesign v0.2.1-0.202
replace github.com/prometheus-community/stackdriver_exporter => github.com/grafana/stackdriver_exporter v0.0.0-20240228143257-3a2c9acef5a2

// Submodules.
// TODO(rfratto): Change all imports of github.com/grafana/river in favor of
// importing github.com/grafana/alloy/syntax and change module and package
// names to remove references of "river".
replace github.com/grafana/river => ./syntax
replace github.com/grafana/alloy/syntax => ./syntax
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1504,8 +1504,6 @@ github.com/lufia/plan9stats v0.0.0-20220913051719-115f729f3c8c h1:VtwQ41oftZwlMn
github.com/lufia/plan9stats v0.0.0-20220913051719-115f729f3c8c/go.mod h1:JKx41uQRwqlTZabZc+kILPrO/3jlKnQ2Z8b7YiVw5cE=
github.com/lyft/protoc-gen-validate v0.0.0-20180911180927-64fcb82c878e/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
github.com/mackerelio/go-osstat v0.2.3 h1:jAMXD5erlDE39kdX2CU7YwCGRcxIO33u/p8+Fhe5dJw=
github.com/mackerelio/go-osstat v0.2.3/go.mod h1:DQbPOnsss9JHIXgBStc/dnhhir3gbd3YH+Dbdi7ptMA=
github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
Expand Down
6 changes: 3 additions & 3 deletions internal/cmd/rivereval/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"fmt"
"os"

"github.com/grafana/river/parser"
"github.com/grafana/river/token/builder"
"github.com/grafana/river/vm"
"github.com/grafana/alloy/syntax/parser"
"github.com/grafana/alloy/syntax/token/builder"
"github.com/grafana/alloy/syntax/vm"
)

func main() {
Expand Down
14 changes: 7 additions & 7 deletions internal/component/all/all_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"testing"

"github.com/grafana/agent/internal/component"
"github.com/grafana/river"
"github.com/grafana/alloy/syntax"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
Expand All @@ -16,7 +16,7 @@ import (
// recursively traverses through its Arguments type to guarantee that no two
// calls to SetDefault result in pointer reuse.
//
// Nested types that also implement river.Defaulter are also checked.
// Nested types that also implement syntax.Defaulter are also checked.
func TestSetDefault_NoPointerReuse(t *testing.T) {
allComponents := component.AllNames()
for _, componentName := range allComponents {
Expand All @@ -37,10 +37,10 @@ func testNoReusePointer(t *testing.T, reg component.Registration) {
args2 = reg.CloneArguments()
)

if args1, ok := args1.(river.Defaulter); ok {
if args1, ok := args1.(syntax.Defaulter); ok {
args1.SetToDefault()
}
if args2, ok := args2.(river.Defaulter); ok {
if args2, ok := args2.(syntax.Defaulter); ok {
args2.SetToDefault()
}

Expand All @@ -60,7 +60,7 @@ func testNoReusePointer(t *testing.T, reg component.Registration) {

assert.Fail(t,
fmt.Sprintf("Detected SetToDefault pointer reuse at %s", fullPath),
"Types implementing river.Defaulter must not reuse pointers across multiple calls. Doing so leads to default values being changed when unmarshaling configuration files. If you're seeing this error, check the path above and ensure that copies are being made of any pointers in all instances of SetToDefault calls where that field is used.",
"Types implementing syntax.Defaulter must not reuse pointers across multiple calls. Doing so leads to default values being changed when unmarshaling configuration files. If you're seeing this error, check the path above and ensure that copies are being made of any pointers in all instances of SetToDefault calls where that field is used.",
)
}
}
Expand Down Expand Up @@ -165,11 +165,11 @@ func pointersMatch(a, b reflect.Value) bool {
}

// initValue initializes nil pointers. If the nil pointer implements
// river.Defaulter, it is also set to default values.
// syntax.Defaulter, it is also set to default values.
func initValue(rv reflect.Value) {
if rv.Kind() == reflect.Pointer && rv.IsNil() {
rv.Set(reflect.New(rv.Type().Elem()))
if defaulter, ok := rv.Interface().(river.Defaulter); ok {
if defaulter, ok := rv.Interface().(syntax.Defaulter); ok {
defaulter.SetToDefault()
}
}
Expand Down
14 changes: 7 additions & 7 deletions internal/component/common/config/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"net/url"
"strings"

"github.com/grafana/river/rivertypes"
"github.com/grafana/alloy/syntax/alloytypes"
"github.com/prometheus/common/config"
)

Expand All @@ -18,7 +18,7 @@ type HTTPClientConfig struct {
BasicAuth *BasicAuth `river:"basic_auth,block,optional"`
Authorization *Authorization `river:"authorization,block,optional"`
OAuth2 *OAuth2Config `river:"oauth2,block,optional"`
BearerToken rivertypes.Secret `river:"bearer_token,attr,optional"`
BearerToken alloytypes.Secret `river:"bearer_token,attr,optional"`
BearerTokenFile string `river:"bearer_token_file,attr,optional"`
ProxyConfig *ProxyConfig `river:",squash"`
TLSConfig TLSConfig `river:"tls_config,block,optional"`
Expand Down Expand Up @@ -110,7 +110,7 @@ var DefaultHTTPClientConfig = HTTPClientConfig{
// BasicAuth configures Basic HTTP authentication credentials.
type BasicAuth struct {
Username string `river:"username,attr,optional"`
Password rivertypes.Secret `river:"password,attr,optional"`
Password alloytypes.Secret `river:"password,attr,optional"`
PasswordFile string `river:"password_file,attr,optional"`
}

Expand Down Expand Up @@ -215,7 +215,7 @@ func (u *URL) Convert() config.URL {
}

type Header struct {
Header map[string][]rivertypes.Secret `river:"proxy_connect_header,attr,optional"`
Header map[string][]alloytypes.Secret `river:"proxy_connect_header,attr,optional"`
}

func (h *Header) Convert() config.Header {
Expand All @@ -239,7 +239,7 @@ func (h *Header) Convert() config.Header {
// Authorization sets up HTTP authorization credentials.
type Authorization struct {
Type string `river:"type,attr,optional"`
Credentials rivertypes.Secret `river:"credentials,attr,optional"`
Credentials alloytypes.Secret `river:"credentials,attr,optional"`
CredentialsFile string `river:"credentials_file,attr,optional"`
}

Expand Down Expand Up @@ -305,7 +305,7 @@ type TLSConfig struct {
CAFile string `river:"ca_file,attr,optional"`
Cert string `river:"cert_pem,attr,optional"`
CertFile string `river:"cert_file,attr,optional"`
Key rivertypes.Secret `river:"key_pem,attr,optional"`
Key alloytypes.Secret `river:"key_pem,attr,optional"`
KeyFile string `river:"key_file,attr,optional"`
ServerName string `river:"server_name,attr,optional"`
InsecureSkipVerify bool `river:"insecure_skip_verify,attr,optional"`
Expand Down Expand Up @@ -359,7 +359,7 @@ func (t *TLSConfig) Validate() error {
// OAuth2Config sets up the OAuth2 client.
type OAuth2Config struct {
ClientID string `river:"client_id,attr,optional"`
ClientSecret rivertypes.Secret `river:"client_secret,attr,optional"`
ClientSecret alloytypes.Secret `river:"client_secret,attr,optional"`
ClientSecretFile string `river:"client_secret_file,attr,optional"`
Scopes []string `river:"scopes,attr,optional"`
TokenURL string `river:"token_url,attr,optional"`
Expand Down
22 changes: 11 additions & 11 deletions internal/component/common/config/types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package config
import (
"testing"

"github.com/grafana/river"
"github.com/grafana/alloy/syntax"
"github.com/stretchr/testify/require"
)

Expand All @@ -25,7 +25,7 @@ func TestHTTPClientConfigBearerToken(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -38,7 +38,7 @@ func TestHTTPClientConfigBearerTokenFile(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -55,7 +55,7 @@ func TestHTTPClientConfigBasicAuthPassword(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -72,7 +72,7 @@ func TestHTTPClientConfigBasicAuthPasswordFile(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -89,7 +89,7 @@ func TestHTTPClientConfigAuthorizationCredentials(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -106,7 +106,7 @@ func TestHTTPClientConfigAuthorizationCredentialsFile(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand Down Expand Up @@ -135,7 +135,7 @@ func TestHTTPClientConfigOath2ClientSecret(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -156,7 +156,7 @@ func TestHTTPClientConfigOath2ClientSecretFile(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
}

Expand All @@ -172,7 +172,7 @@ func TestOath2TLSConvert(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.NoError(t, err)
newCfg := httpClientConfig.Convert()
require.NotNil(t, newCfg)
Expand Down Expand Up @@ -227,6 +227,6 @@ func TestHTTPClientBadConfig(t *testing.T) {
`

var httpClientConfig HTTPClientConfig
err := river.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
err := syntax.Unmarshal([]byte(exampleRiverConfig), &httpClientConfig)
require.ErrorContains(t, err, "at most one of basic_auth password & password_file must be configured")
}
Loading

0 comments on commit ad62898

Please sign in to comment.