Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor integration tests to prepare for proxy-server & agent skew testing #520

Merged
merged 3 commits into from
Sep 22, 2023

Conversation

tallclair
Copy link
Contributor

This PR refactors the running of the test proxy-server & agent to prepare for running them as separate binaries, which will enable version skew testing.

This PR makes it so the server & agent are run with a RunnerInterface accessed through a global Framework. The runner interfaces returns a respective interface for accessing the testable surface of the server & agent.

The TestConcurrentClientRequest test needed to be rewritten, since it relied on internal implementation details. The new test is less deterministic, but should still reliably hit the same case being tested (and others).

First step towards #519

Next steps:

  1. Implement External{ProxyServer,Agent}Runner, which takes an external binary to run for the server/agent, and accesses the measurements through prometheus metrics. Expose test flags to select a binary (or the in-tree version).
  2. Implement a version_skew_test script that handles fetching target tagged versions, compiling, and running the tests.
  3. Set up CI jobs for skew testing
  4. Refactor client operations to enable external client testing via the test-client binary.
  5. Implement ExternalClient, update version_skew_test to run tests with the external client

/assign @jkh52

I recommend reviewing this 1 commit at a time.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 30, 2023
Comment on lines -1017 to +878
hc, cc := clientset.HealthyClientsCount(), clientset.ClientsCount()
t.Logf("got %d clients, %d of them are healthy; expected %d", cc, hc, expectedServerCount)
if csc, err := a.GetConnectedServerCount(); err == nil {
t.Logf("got %d server connections; expected %d", csc, expectedServerCount)
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to reviewers: I dropped ClientsCount here since it is not exposed by prometheus metrics. It wasn't actually being tested anyway, just used in the error message.

@tallclair tallclair changed the title Refactor integration tests to prepare for skew testing Refactor integration tests to prepare for proxy-server & agent skew testing Aug 30, 2023
@tallclair tallclair mentioned this pull request Aug 30, 2023
6 tasks
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 30, 2023
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2023
return 0
}

func newSingleTimeGetter(m *server.DefaultBackendManager) *singleTimeManager {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting rid of this helps me in a Backend test improvement I started, thanks!

@jkh52
Copy link
Contributor

jkh52 commented Sep 22, 2023

/lgtm
/approve

Great contribution!

/hold

Hold in case you want to add that squash directive (some bits like t.Skip, FIXME, etc. show between commits).

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Sep 22, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jkh52, tallclair

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 22, 2023
@tallclair
Copy link
Contributor Author

/hold cancel

I think the t.Skip() is OK to have in an intermediate commit. It helps the commits to be focused, and ensures that tests still pass at each intermediate.

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 22, 2023
@k8s-ci-robot k8s-ci-robot merged commit df7c9f1 into kubernetes-sigs:master Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants