Implement Before and BeforeAll logic execution

[polskikiel]

Description

In Kyma we have some issues with the execution of our tests which applies to every test pod.

1. When we start a test we always need to add a testing-addons to our cluster otherwise, the tests will fail. The number of required things for our tests can increase in time, so it's better to implement it here rather than in many different places later, which will be much harder to maintenance.

2. All of our test pods contain a sidecar which blocks Pod's connections to API server until sidecar is ready. It may fail tests if they don't have a waiter.

Above issues can be addressed in Octopus by implementing a proper logic which will be executed once before test suite execution AND a logic which will be executed before each test.

It should be configurable from the TestSuite level.

Reasons

Many tests have common issues. This implementation allows us to avoid many issues and boilerplate code in the future.

Attachments

[stale]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

[k15r]

Octopus already supports executing Testdefinitions based on labels. An easy solution to the 'before' case that would not require any change on octopus would be to specify a label e.g. kyma-project.io/test.integration-setup=true and kyma-project.io/test.integration-teardown=true

Using those labels would then just be a change in the integration-test scripts. Currently a similar mechanic is used in the upgrade tests. Here we use kyma-project.io/test.before-upgrade to prepare the cluster.

[stale]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

[pbochynski]

We will not work on it.