Skip to content

Commit

Permalink
#204 Renamed DEFAULT to STANDARD
Browse files Browse the repository at this point in the history
  • Loading branch information
baubakg committed Nov 21, 2024
1 parent 0eb891e commit cc96210
Show file tree
Hide file tree
Showing 11 changed files with 61 additions and 26 deletions.
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,14 +130,50 @@ The following dependency needs to be added to your pom file:
</dependency>
```

### Demo
## Demo
We have a standard demo that can be accessed through the [Phased Test Demo](https://github.com/baubakg/phased-test-demo).

## Phases
## Execution Modes
We currently have 4 execution modes:
* STANDARD
* INTERRUPTIVE
* NON-INTERRUPTIVE
* PERMUATIONAL


The execution mode is set by passing the config value "MUTATIONAL.EXECUTION.MODE" at execution time.

Some execution modes have a notion of a "behavior" which add more details to the system as to how the tests should be executed. The behavior is set by passing the behavior within parenthesis.

### STANDARD Execution Mode
This is the default execution mode. By default, we execute the scenario in the order and manner in which it was defined.

### INTERRUPTIVE Execution Mode
The Interruptive execution mode simulates the system being subject to an interruptive event.

The Phased Testing framework was originally devised for Interruptive Events, i.e. you need to stop a system so that you can perform some system change, such as an upgrade, to that system. Once the upgrade is done, we expect that the users can carry on with what they were doing.

The execution of steps in interruptive events is divided into two phases/behaviors depending on their execution relative to the interruptive event. The phase before the event is called “producer”, because the steps executed before the event produce data used after the event has taken place. Similarly, the phase after the event is called “consumer” because the steps rely on data created in the phase before the execution of the event.

| NAME | When Passing | Description |
|----------|-----------------------|-----------------------------------------------------------------------------------------------------------------|
| PRODUCER | INTERUPTIVE(PRODUCER) | The tests will stop before we execute the event. The tests prepare data to be used in the following test phase. |
| CONSUMER | INTERUPTIVE(CONSUMER) | The tests will continue where they left off after the event has finished. The tests consume the data produced in the previous phase. |


### NON-INTERRUPTIVE execution mode
A non-interruptive execution mode is used when we want to inject an event in the middle of the execution of a scenario. Non-Interruptive events allow us to see the effects of parallel events.

This execution mode is a good way of performing chaos testing.

This mode is activated by setting the environment variable "MUTATIONAL.EXECUTION.MODE" to "NON-INTERRUPTIVE".


## LEGACY PHASES - DEPRECATED
Phases are directives at execution time, where we let the system know, in what way we want our tests to interact with an event.

We have four test phases:
* **Producer** In this Interruptive mode, the tests will stop before we execute the event. The tests prepare data to be used in the following test phase. The tests will be interr
* **Producer** In this Interruptive mode, the tests will stop before we execute the event. The tests prepare data to be used in the following test phase.
* **Consumer** In this Interruptive mode, the tests will continue where they left off after the event has finished. The tests consume the data produced in the previous phase.
* **Asynchrounous** In this Non-Interruptive mode, the events are executed in parallel to a step.
* * **Non-Phased** In this state, we have not designated a state, as such, if not unwanted, we execute all tests.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import java.util.List;

public enum ExecutionMode {
DEFAULT(false, new ArrayList<>()),
STANDARD(false, new ArrayList<>()),
//We need to revise this as execution in a suite may not require a precision
NON_INTERRUPTIVE(false, Arrays.asList( "23", "33" )) {

Expand Down Expand Up @@ -75,7 +75,7 @@ public static ExecutionMode fetchCorrespondingMode(String in_stateValue) {
return lt_ptState;
}
}
return DEFAULT;
return STANDARD;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ public void alter(List<XmlSuite> suites) {
.containsKey(ConfigValueHandlerPhased.PROP_PHASED_TEST_DATABROKER.systemName)) {
l_phasedDataBrokerClass = suites.get(0)
.getParameter(ConfigValueHandlerPhased.PROP_PHASED_TEST_DATABROKER.systemName);
} else if (!ExecutionMode.DEFAULT.isSelected()) {
} else if (!ExecutionMode.STANDARD.isSelected()) {
log.info("{} No PhasedDataBroker set. Using the file system path {}/{} instead ",
PhasedTestManager.PHASED_TEST_LOG_PREFIX, PhasedTestManager.STD_STORE_DIR,
PhasedTestManager.STD_STORE_FILE
Expand Down Expand Up @@ -123,7 +123,7 @@ public void transform(ITestAnnotation annotation, Class testClass, Constructor t
}

if (PhasedTestManager.isPhasedTest(l_currentClass)) {
if (ExecutionMode.DEFAULT.isSelected()) {
if (ExecutionMode.STANDARD.isSelected()) {
annotation.setDataProvider(
ConfigValueHandlerPhased.PHASED_TEST_NONPHASED_LEGACY.is("true") ? PhasedDataProvider.SINGLE : PhasedDataProvider.DEFAULT);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ public void alter(List<XmlSuite> suites) {
.containsKey(ConfigValueHandlerPhased.PROP_PHASED_TEST_DATABROKER.systemName)) {
l_phasedDataBrokerClass = suites.get(0)
.getParameter(ConfigValueHandlerPhased.PROP_PHASED_TEST_DATABROKER.systemName);
} else if (!ExecutionMode.DEFAULT.isSelected()) {
} else if (!ExecutionMode.STANDARD.isSelected()) {
log.info("{} No PhasedDataBroker set. Using the file system path {}/{} instead ",
PhasedTestManager.PHASED_TEST_LOG_PREFIX, PhasedTestManager.STD_STORE_DIR,
PhasedTestManager.STD_STORE_FILE
Expand Down Expand Up @@ -434,7 +434,7 @@ public void transform(ITestAnnotation annotation, Class testClass, Constructor t
}

if (PhasedTestManager.isPhasedTest(l_currentClass)) {
if (ExecutionMode.DEFAULT.isSelected()) {
if (ExecutionMode.STANDARD.isSelected()) {
annotation.setDataProvider(
ConfigValueHandlerPhased.PHASED_TEST_NONPHASED_LEGACY.is("true") ? PhasedDataProvider.SINGLE : PhasedDataProvider.DEFAULT);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -648,7 +648,7 @@ public static Object[] fetchProvidersSingle(Method in_method) {
return new Object[] { STD_PHASED_GROUP_SINGLE };
}

if (ExecutionMode.DEFAULT.isSelected() && in_method.getDeclaringClass().getAnnotation(PhasedTest.class)
if (ExecutionMode.STANDARD.isSelected() && in_method.getDeclaringClass().getAnnotation(PhasedTest.class)
.executeInactive()) {
return new Object[] { STD_PHASED_GROUP_SINGLE };
}
Expand All @@ -672,7 +672,7 @@ public static Object[] fetchProvidersSingle(Method in_method) {
public static Object[] fetchProvidersStandard(Method in_method) {
log.debug("Returning provider for method {}", ClassPathParser.fetchFullName(in_method));

if (ExecutionMode.DEFAULT.isSelected() && !in_method.getDeclaringClass().getAnnotation(PhasedTest.class)
if (ExecutionMode.STANDARD.isSelected() && !in_method.getDeclaringClass().getAnnotation(PhasedTest.class)
.executeInactive()) {
return new Object[] { };
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

public enum Phases {
PRODUCER(true, ExecutionMode.INTERRUPTIVE, "PRODUCER"), CONSUMER(true, ExecutionMode.INTERRUPTIVE,
"CONSUMER"), NON_PHASED(false, ExecutionMode.DEFAULT, ""), ASYNCHRONOUS(false,
"CONSUMER"), NON_PHASED(false, ExecutionMode.STANDARD, ""), ASYNCHRONOUS(false,
ExecutionMode.NON_INTERRUPTIVE, ""), PERMUTATIONAL(false, ExecutionMode.PERMUTATIONAL, "");

boolean hasSplittingEvent;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -168,21 +168,21 @@ public void testSetModeInterruptiveConsumerPhased() {

@Test
public void testDefault() {
assertThat("This should be the same as Non-interruptive", ExecutionMode.DEFAULT.isSelected());
assertThat("This should be the same as Non-interruptive", ExecutionMode.STANDARD.isSelected());
assertThat("Even though we pass a bad value we should not throw an exception. It is simply ignored",
ExecutionMode.DEFAULT.isSelected("CONSUMER"));
ExecutionMode.STANDARD.isSelected("CONSUMER"));

assertThat("We should have the correct phase", Phases.getCurrentPhase().equals(Phases.NON_PHASED));

var runMode = ExecutionMode.getCurrentMode().fetchRunValues();
assertThat("The execution mode should be correct", runMode.getExecutionMode(),
Matchers.equalTo(ExecutionMode.DEFAULT));
Matchers.equalTo(ExecutionMode.STANDARD));
assertThat("The execution mode should be correct", runMode.getBehavior(), Matchers.equalTo(""));

assertThat("is is default selected", Phases.NON_PHASED.isSelected());

assertThat("The runMode toString should be correct", runMode.toString(),
Matchers.equalTo("DEFAULT"));
Matchers.equalTo("STANDARD"));

assertThat("We should properly present the execution mode", ExecutionMode.getCurrentModeAsString(),
Matchers.equalTo(runMode.toString()));
Expand Down Expand Up @@ -213,8 +213,8 @@ public void testMutational() {
@Test
public void testIs() {

assertThat("We should have equals", ExecutionMode.getCurrentMode().equals(ExecutionMode.DEFAULT));
assertThat("We should have equals", ExecutionMode.is(ExecutionMode.DEFAULT));
assertThat("We should have equals", ExecutionMode.getCurrentMode().equals(ExecutionMode.STANDARD));
assertThat("We should have equals", ExecutionMode.is(ExecutionMode.STANDARD));

}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ public void testExecutionIndex_InterruptiveProducer() {

assertThat("We should have one steps to executed by default",
MutationManager.fetchExecutionIndex(testClass.getTypeName(), l_phaseGroup,
new RunValues(ExecutionMode.DEFAULT, "")),
new RunValues(ExecutionMode.STANDARD, "")),
Matchers.arrayContaining(0, 3));

assertThat("We should have one steps to executed in Asynchronous",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
import com.adobe.campaign.tests.integro.phased.exceptions.PhasedTestException;
import com.adobe.campaign.tests.integro.phased.mutational.data.permutational.MultipleProducerConsumer;
import com.adobe.campaign.tests.integro.phased.mutational.data.permutational.ShoppingCartDemo;
import com.adobe.campaign.tests.integro.phased.mutational.data.simple1.PhasedChild2;
import com.adobe.campaign.tests.integro.phased.permutational.ScenarioStepDependencies;
import com.adobe.campaign.tests.integro.phased.permutational.ScenarioStepDependencyFactory;
import com.adobe.campaign.tests.integro.phased.utils.ClassPathParser;
Expand Down Expand Up @@ -2900,14 +2899,14 @@ public void testDoesTesthaveStepsInProducer() throws NoSuchMethodException, Secu
.thenReturn(new Object[] { PhasedTestManager.STD_PHASED_GROUP_PREFIX + "1_3" });

assertThat("This method and phase group should not have steps in the producer",
!PhasedTestManager.hasStepsExecutedInProducer(l_itr, new RunValues(ExecutionMode.DEFAULT, "")));
!PhasedTestManager.hasStepsExecutedInProducer(l_itr, new RunValues(ExecutionMode.STANDARD, "")));

Mockito.when(l_itr.getParameters())
.thenReturn(new Object[] { PhasedTestManager.STD_PHASED_GROUP_PREFIX + "0_3" });

assertThat(
"This method and phase group should not have steps in the producer since we are in Producer",
!PhasedTestManager.hasStepsExecutedInProducer(l_itr, new RunValues(ExecutionMode.DEFAULT, "")));
!PhasedTestManager.hasStepsExecutedInProducer(l_itr, new RunValues(ExecutionMode.STANDARD, "")));

}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ public void testNonInterruptivePhaseWithEvents() {
ConfigValueHandlerPhased.PROP_EXECUTION_MODE.activate(ExecutionMode.INTERRUPTIVE.name()+"(jhfdhj)");
assertThat("We should detect that given type is incorrect",!ExecutionMode.getCurrentMode().isTypeValid());

ExecutionMode.DEFAULT.activate();
ExecutionMode.STANDARD.activate();
assertThat("This should be the same as Non-phased", ExecutionMode.getCurrentMode().fetchBehavior(), Matchers.equalTo(""));
assertThat("We should accept an empty type", ExecutionMode.getCurrentMode().isTypeValid());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -433,7 +433,7 @@ public void testNonInterruptive_ParellelHardCoded_SINGLE() {

assertThat("The correct phase must have been selected", ExecutionMode.NON_INTERRUPTIVE.isSelected("33"));
assertThat("The correct phase must have been selected", ExecutionMode.getCurrentMode(),
not(equalTo(ExecutionMode.DEFAULT)));
not(equalTo(ExecutionMode.STANDARD)));

assertThat("We should have 3 successful methods of phased Tests",
(int) tla.getPassedTests().stream().filter(m -> m.getInstance().getClass().equals(l_testClass)).count(),
Expand Down Expand Up @@ -504,7 +504,7 @@ public void testNonInterruptive_ParellelConfigured_SINGLELegacy() {

assertThat("The correct phase must have been selected", ExecutionMode.getCurrentMode(), equalTo(ExecutionMode.NON_INTERRUPTIVE));
assertThat("The correct phase must have been selected", ExecutionMode.getCurrentMode(),
not(equalTo(ExecutionMode.DEFAULT)));
not(equalTo(ExecutionMode.STANDARD)));

assertThat("We should have 3 successful methods of phased Tests",
(int) tla.getPassedTests().stream().filter(m -> m.getInstance().getClass().equals(l_testClass)).count(),
Expand Down Expand Up @@ -576,7 +576,7 @@ public void testNonInterruptive_ParellelConfigured_SINGLE() {

assertThat("The correct phase must have been selected", ExecutionMode.getCurrentMode(), equalTo(ExecutionMode.NON_INTERRUPTIVE));
assertThat("The correct phase must have been selected", ExecutionMode.getCurrentMode(),
not(equalTo(ExecutionMode.DEFAULT)));
not(equalTo(ExecutionMode.STANDARD)));

assertThat("We should have 3 successful methods of phased Tests",
(int) tla.getPassedTests().stream().filter(m -> m.getInstance().getClass().equals(l_testClass)).count(),
Expand Down

0 comments on commit cc96210

Please sign in to comment.