Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exporter is not using exponential backoff/maxRetries on GitHub API-Calls #26

Open
DerTiedemann opened this issue Aug 9, 2021 · 0 comments
Assignees

Comments

@DerTiedemann
Copy link

As the github api exerts a rate limit, and the exporter does not respect that which can cause delays in metric delivery.

Expected Behavior

The Exporter should respect a configurable ratelimit to not exhaust all requests at once.

Current Behavior

On an recurring error, such as wrongly configured creds (wrong permissions) the exporter just depletes all of its requests in a matter of seconds.
For what ever reason this infinite loop does not stop when it hits an error
https://github.com/Spendesk/github-actions-exporter/blob/ec79f05a21c132a0c27f546c4955a159a3721baa/pkg/metrics/get_runners_organization_from_github.go#L29-L46

Possible Solution

Add a ratelimiting RoundTripper to the github client, like this one https://gist.github.com/zdebra/10f0e284c4672e99f0cb767298f20c11
This would also eleviate the need for the caching layer

Steps to Reproduce

  1. Create a GH token with wrong permissons (or anything that reliably produces an error with the gh client)
  2. See it flood the GH API

Context (Environment)

Just me being stoopid and not configureing a proper token, but this could have happend on any error with the github client.
Lets say you alert based on metrics provided my this exporter, any error that would cause the rate limit to be reached would cause alerts.

@moutonjeremy moutonjeremy self-assigned this Aug 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants