[OTLP] add entity processor to the otlp metrics and logs pipeline

[duhminick]

Description of the issue

We want the ability to send entities for customers sending custom metrics via otlp: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-OpenTelemetry-metrics.html

This is currently be limited to just EKS/K8S.

Description of changes

1. Creates an instance of entity processor specifically for OTLP
2. Adds the newly created processor to the OTLP related pipelines by now adding the CloudWatch Logs key to the "list" of destinations
3. Update go.mod to take in this change: Remove internal entity fields for awsemfexporter amazon-contributing/opentelemetry-collector-contrib#271

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

1. Unit test
2. Manual testing in an EC2 instance

Manual Testing

Using the following agent configuration:

"logs": {
    "metrics_collected": {
        "otlp": {
            "grpc_endpoint": "127.0.0.1:9999",
            "http_endpoint": "127.0.0.1:9998"
        }
    }
}
...
"metrics": {
    "metrics_collected": {
        "otlp": {
            "grpc_endpoint": "127.0.0.1:9999",
            "http_endpoint": "127.0.0.1:9998"
        }
    }
}

Output

EMF

Translated YAML config output for EC2

        metrics/hostOtlpMetrics:
            exporters:
                - awscloudwatch
            processors:
                - cumulativetodelta/hostOtlpMetrics
                - ec2tagger
            receivers:
                - otlp/metrics
        metrics/hostOtlpMetrics/cloudwatchlogs:
            exporters:
                - awsemf
            processors:
                - cumulativetodelta/hostOtlpMetrics/cloudwatchlogs
                - batch/hostOtlpMetrics/cloudwatchlogs
            receivers:
                - otlp/metrics
        traces/application_signals:
            exporters:
                - awsxray/application_signals
            processors:
                - resourcedetection
                - awsapplicationsignals
            receivers:
                - otlp/application_signals

Translated YAML config output for K8S

    awsentity/service/otlp:
        cluster_name: application-signals-demo
        entity_type: Service
        kubernetes_mode: EKS
        platform: ec2
...
        metrics/hostOtlpMetrics:
            exporters:
                - awscloudwatch
            processors:
                - awsentity/service/otlp
                - cumulativetodelta/hostOtlpMetrics
            receivers:
                - otlp/metrics
        metrics/hostOtlpMetrics/cloudwatchlogs:
            exporters:
                - awsemf
            processors:
                - awsentity/service/otlp
                - cumulativetodelta/hostOtlpMetrics/cloudwatchlogs
                - batch/hostOtlpMetrics/cloudwatchlogs
            receivers:
                - otlp/metrics
        traces/application_signals:
            exporters:
                - debug/application_signals
                - awsxray/application_signals
            processors:
                - resourcedetection
                - awsapplicationsignals
            receivers:
                - otlp/application_signals

Requirements

Before commit the code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint

[Paramadon]

lgtm

[duhminick]

We've decided to just add this functionality to just K8S/EKS for the time being.

[nathalapooja]

Can you just make sure the mode is set to EC2 incase of EKS and k8sonEC2?

[duhminick]

On EKS, I can confirm it's set to EC2

As for K8sOnEC2, it will be EC2 if the configured mode is EC2. Code ref:

amazon-cloudwatch-agent/translator/util/sdkutil.go

Lines 75 to 77 in bb8a31b

	if configuredMode == config.ModeEC2 {
	return config.ModeK8sEC2
	}

[nathalapooja]

We see server extension on manual testing , why not here?

[duhminick]

I think that's just because of how the translation responsibilities are split up. So it's not really unit-testable (that is unless I did a refactor).

It's set here:

amazon-cloudwatch-agent/translator/translate/otel/translate_otel.go

Lines 91 to 93 in bb8a31b

	if context.CurrentContext().KubernetesMode() != "" {
	pipelines.Translators.Extensions.Set(server.NewTranslator())
	}

[nathalapooja]

as we add the server extension can you query the server endpoint and check whether it is generating the pod to service map with correct values in case of just otlp in config?

[duhminick]

Yep, it works

[duhminick]