Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a method to reference the topic partition #242

Open
be-hase opened this issue Aug 15, 2024 · 2 comments
Open

Provide a method to reference the topic partition #242

be-hase opened this issue Aug 15, 2024 · 2 comments

Comments

@be-hase
Copy link
Member

be-hase commented Aug 15, 2024

There are cases where I need to reference the topic partition for metrics purposes.

While it's possible to force a reference through the MDC injected by LoggingContext, I would prefer a more direct method.

MDC.put(PARTITION_KEY, String.valueOf(request.topicPartition().partition()));

Even just adding getters to the various fields of LoggingContext would be helpful, but it might deviate from the original purpose of LoggingContext.

To elaborate on the use case, when using deferCompletion, the decaton.tasks.error metrics cannot be used, so I want to record the metrics myself. In this case, I need to reference the partition topic.
https://github.com/line/decaton/blob/master/docs/monitoring.adoc#:~:text=in%20a%20topic.-,decaton.tasks.error,-The%20number%20of

(In other words, if decaton.tasks.error could be recorded even when using deferCompletion, that would be fine as well.)

@ocadaruma
Copy link
Contributor

Sorry for the delay of the reaction.

The motivation makes sense.

Why we don't expose partition directly is because we want to make Decaton agnostic to Kafka as much as possible (yeah, I know already there are some places where Kafka-specific thing is exposed though...).

To report per-partition error count even on deferredCompletion is an option though, the problem is how Decaton can know deferred-completion is "failed".

Given these, we are still not sure what's the best way to address the problem.

By the way let me ask if accessing through MDC causes another (e.g. performance) issue, not only the inconvenience.

@be-hase
Copy link
Member Author

be-hase commented Oct 2, 2024

Why we don't expose partition directly

I see.
With the rise of many Kafka-compatible middleware solutions, it certainly makes sense to hide the implementation as much as possible.

By the way let me ask if accessing through MDC causes another (e.g. performance) issue, not only the inconvenience.

Since it's just reading from thread-local storage, there are no performance concerns.

For now, I'm satisfied with the feedback, so it's okay to close the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants