Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(telemetry)_: replace telemetry with prometheus metrics #6256

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

adklempner
Copy link
Contributor

@adklempner adklempner commented Jan 15, 2025

Replace telemetry with local metrics using prometheus. Add parameters to InitializeApplication for enabling waku metrics over prometheus and specifying which port to use.

This commit removes the telemetry functionality with a Prometheus client. Most of the metrics that were collected by telemetry now have their corresponding Prometheus gauges, counters, and histograms.

They still require a telemetry url to be set in order to be enabled. Additionally, the parameter WakuMetricsEnabled needs to be set as true in the request for InitializeApplication in order to start Prometheus at port 9305 (can be changed using WakuMetricsPort).

status-desktop Dogfooding PR: status-im/status-desktop#17020

@status-im-auto
Copy link
Member

status-im-auto commented Jan 15, 2025

Jenkins Builds

Click to see older builds (80)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 7cf1f93 #1 2025-01-15 21:30:58 ~3 min macos 📦zip
✖️ 7cf1f93 #1 2025-01-15 21:31:03 ~4 min tests 📄log
✔️ 7cf1f93 #1 2025-01-15 21:31:28 ~4 min ios 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:17 ~5 min linux 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:30 ~5 min android 📦aar
✔️ 7cf1f93 #1 2025-01-15 21:32:32 ~5 min macos 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:38 ~5 min windows 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:33:32 ~6 min tests-rpc 📄log
✖️ 9c43971 #2 2025-01-16 00:30:24 ~2 min tests 📄log
✔️ 9c43971 #2 2025-01-16 00:31:32 ~3 min windows 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:11 ~4 min macos 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:28 ~4 min linux 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:43 ~5 min ios 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:57 ~5 min macos 📦zip
✔️ 9c43971 #2 2025-01-16 00:33:15 ~5 min android 📦aar
✔️ 9c43971 #2 2025-01-16 00:34:15 ~6 min tests-rpc 📄log
✔️ 7c4ff40 #3 2025-01-16 00:52:34 ~3 min windows 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:52:55 ~4 min macos 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:07 ~4 min ios 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:20 ~4 min linux 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:45 ~5 min macos 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:54:09 ~5 min android 📦aar
✔️ 7c4ff40 #3 2025-01-16 00:54:41 ~6 min tests-rpc 📄log
✖️ 7c4ff40 #3 2025-01-16 01:18:34 ~29 min tests 📄log
✔️ 3763072 #4 2025-01-16 01:28:53 ~3 min windows 📦zip
✔️ 3763072 #4 2025-01-16 01:29:24 ~4 min macos 📦zip
✔️ 3763072 #4 2025-01-16 01:29:40 ~4 min linux 📦zip
✔️ 3763072 #4 2025-01-16 01:29:49 ~4 min ios 📦zip
✔️ 3763072 #4 2025-01-16 01:30:11 ~5 min macos 📦zip
✔️ 3763072 #4 2025-01-16 01:31:02 ~6 min android 📦aar
✔️ 3763072 #4 2025-01-16 01:31:19 ~6 min tests-rpc 📄log
✔️ 3763072 #4 2025-01-16 01:56:45 ~31 min tests 📄log
✔️ 5f8d5f6 #5 2025-01-16 22:52:02 ~3 min windows 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:00 ~4 min linux 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:17 ~5 min ios 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:29 ~5 min macos 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:54:17 ~6 min macos 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:54:32 ~6 min android 📦aar
✔️ 5f8d5f6 #5 2025-01-16 22:54:33 ~6 min tests-rpc 📄log
✖️ 5f8d5f6 #5 2025-01-16 23:18:05 ~29 min tests 📄log
✔️ 07798db #6 2025-01-16 23:05:31 ~3 min windows 📦zip
✔️ 07798db #6 2025-01-16 23:06:31 ~4 min linux 📦zip
✔️ 07798db #6 2025-01-16 23:07:07 ~5 min macos 📦zip
✔️ 07798db #6 2025-01-16 23:07:13 ~5 min ios 📦zip
✔️ 07798db #6 2025-01-16 23:07:28 ~5 min android 📦aar
✔️ 07798db #6 2025-01-16 23:07:35 ~5 min macos 📦zip
✖️ 07798db #6 2025-01-16 23:08:04 ~6 min tests-rpc 📄log
✖️ 07798db #6 2025-01-16 23:48:08 ~29 min tests 📄log
✔️ 534c2df #7 2025-01-21 20:29:02 ~3 min macos 📦zip
✔️ 534c2df #7 2025-01-21 20:30:06 ~5 min ios 📦zip
✔️ 534c2df #7 2025-01-21 20:30:14 ~5 min macos 📦zip
✔️ 534c2df #7 2025-01-21 20:30:35 ~5 min linux 📦zip
✔️ 534c2df #7 2025-01-21 20:31:05 ~6 min android 📦aar
✖️ 534c2df #7 2025-01-21 20:31:22 ~6 min tests-rpc 📄log
✔️ d8f0d5b #8 2025-01-21 20:29:25 ~3 min windows 📦zip
✖️ d8f0d5b #8 2025-01-21 20:55:09 ~29 min tests 📄log
0a81a87 #8 2025-01-21 20:31:12 ~2 min macos 📄log
0a81a87 #9 2025-01-21 20:31:40 ~2 min windows 📄log
0a81a87 #8 2025-01-21 20:31:54 ~1 min ios 📄log
0a81a87 #8 2025-01-21 20:32:55 ~1 min android 📄log
0a81a87 #8 2025-01-21 20:33:22 ~2 min linux 📄log
0a81a87 #8 2025-01-21 20:34:09 ~3 min macos 📄log
✖️ 0a81a87 #8 2025-01-21 20:34:34 ~2 min tests-rpc 📄log
✔️ a3d37c4 #10 2025-01-21 20:43:06 ~3 min windows 📦zip
✔️ a3d37c4 #9 2025-01-21 20:43:29 ~4 min macos 📦zip
✔️ a3d37c4 #9 2025-01-21 20:43:47 ~4 min ios 📦zip
✔️ a3d37c4 #9 2025-01-21 20:44:16 ~5 min linux 📦zip
✔️ a3d37c4 #9 2025-01-21 20:44:24 ~5 min macos 📦zip
✔️ a3d37c4 #9 2025-01-21 20:44:25 ~5 min android 📦aar
✔️ a3d37c4 #9 2025-01-21 20:45:18 ~6 min tests-rpc 📄log
✖️ a3d37c4 #9 2025-01-21 20:58:50 ~3 min tests 📄log
✔️ bdf78e5 #11 2025-01-21 21:41:07 ~3 min windows 📦zip
✔️ bdf78e5 #10 2025-01-21 21:41:39 ~4 min macos 📦zip
✔️ bdf78e5 #10 2025-01-21 21:41:49 ~4 min ios 📦zip
✔️ bdf78e5 #10 2025-01-21 21:42:30 ~5 min linux 📦zip
✔️ bdf78e5 #10 2025-01-21 21:42:39 ~5 min macos 📦zip
✔️ bdf78e5 #10 2025-01-21 21:42:39 ~5 min android 📦aar
✖️ bdf78e5 #10 2025-01-21 21:43:49 ~6 min tests-rpc 📄log
✖️ bdf78e5 #10 2025-01-21 22:07:12 ~29 min tests 📄log
✔️ bdf78e5 #11 2025-01-21 22:51:18 ~29 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ d31d899 #12 2025-01-21 23:34:51 ~2 min tests 📄log
✔️ d31d899 #12 2025-01-21 23:36:01 ~3 min windows 📦zip
✔️ d31d899 #11 2025-01-21 23:36:36 ~4 min macos 📦zip
✔️ d31d899 #11 2025-01-21 23:36:37 ~4 min ios 📦zip
✔️ d31d899 #11 2025-01-21 23:37:04 ~5 min linux 📦zip
✔️ d31d899 #11 2025-01-21 23:37:13 ~5 min macos 📦zip
✔️ d31d899 #11 2025-01-21 23:37:27 ~5 min android 📦aar
✖️ d31d899 #11 2025-01-21 23:38:05 ~5 min tests-rpc 📄log
✔️ 5340c57 #13 2025-01-22 00:18:33 ~3 min windows 📦zip
✔️ 5340c57 #12 2025-01-22 00:18:59 ~4 min macos 📦zip
✔️ 5340c57 #12 2025-01-22 00:19:16 ~4 min ios 📦zip
✔️ 5340c57 #12 2025-01-22 00:19:50 ~5 min macos 📦zip
✔️ 5340c57 #12 2025-01-22 00:19:51 ~5 min linux 📦zip
✔️ 5340c57 #12 2025-01-22 00:20:10 ~5 min android 📦aar
✔️ 5340c57 #12 2025-01-22 00:20:51 ~6 min tests-rpc 📄log
✖️ 5340c57 #13 2025-01-22 00:43:45 ~29 min tests 📄log
✔️ 5340c57 #14 2025-01-22 02:29:45 ~29 min tests 📄log
✔️ 5340c57 #15 2025-01-22 02:58:54 ~29 min tests 📄log

@adklempner adklempner force-pushed the feat/replace-telemetry-prometheus branch 3 times, most recently from 7c4ff40 to 3763072 Compare January 16, 2025 01:24
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 52.00000% with 96 lines in your changes missing coverage. Please review.

Project coverage is 61.84%. Comparing base (3e0b1b2) to head (5340c57).

Files with missing lines Patch % Lines
metrics/wakumetrics/client.go 59.37% 39 Missing ⚠️
wakuv2/waku.go 11.53% 19 Missing and 4 partials ⚠️
protocol/messenger.go 21.42% 10 Missing and 1 partial ⚠️
mobile/status.go 0.00% 9 Missing and 1 partial ⚠️
metrics/metrics.go 0.00% 8 Missing ⚠️
wakuv2/message_publishing.go 0.00% 2 Missing and 1 partial ⚠️
protocol/messenger_pairing_and_syncing.go 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6256      +/-   ##
===========================================
- Coverage    61.92%   61.84%   -0.09%     
===========================================
  Files          843      844       +1     
  Lines       111286   111103     -183     
===========================================
- Hits         68918    68710     -208     
- Misses       34388    34474      +86     
+ Partials      7980     7919      -61     
Flag Coverage Δ
functional 21.68% <3.50%> (+0.10%) ⬆️
unit 60.30% <52.00%> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
metrics/wakumetrics/metrics.go 100.00% <100.00%> (ø)
protocol/requests/initialize_application.go 50.00% <ø> (ø)
wakuv2/common/metrics.go 100.00% <100.00%> (ø)
protocol/messenger_pairing_and_syncing.go 59.00% <0.00%> (ø)
wakuv2/message_publishing.go 75.38% <0.00%> (ø)
metrics/metrics.go 0.00% <0.00%> (ø)
mobile/status.go 9.66% <0.00%> (-0.08%) ⬇️
protocol/messenger.go 64.27% <21.42%> (+0.08%) ⬆️
wakuv2/waku.go 63.81% <11.53%> (-5.00%) ⬇️
metrics/wakumetrics/client.go 59.37% <59.37%> (ø)

... and 38 files with indirect coverage changes

@adklempner adklempner requested a review from a team January 16, 2025 16:39
)

var (
MessagesSentTotal = prometheus.NewHistogramVec(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is SomethingSomethingTotal a histogram? Shouldn't it be just a normal counter? How do you plan to use it as histogram?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated all metrics to use gauge/counter as appropriate

},
)

PeersByOrigin = prometheus.NewGaugeVec(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these currently connected peers or overall number? Cause I'd expect PeersByOrigin to be only increasing as new are discovered (i.e. Counter)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is based on currently connected peers obtained by periodically looking at Waku's peer store

@adklempner adklempner force-pushed the feat/replace-telemetry-prometheus branch 8 times, most recently from bdf78e5 to d31d899 Compare January 21, 2025 23:31
Replace telemetry with local metrics using prometheus client.
Add parameters to InitializeApplication for enabling waku metrics
over prometheus and specifying which port to use.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants