Use custom yaml with kafka 3.8.0 for AMQ streams operator tests

[gtroitsk]

Summary

Latest AMQ streams operator v2.8 supports Kafka 3.8.0 or less while default Kafka version from FW is 3.9.0

This PR brings temporary solution to be able run OperatorOpenShiftAmqStreamsKafkaStreamIT tests until new AMQ streams operator version will be out.

FW OpenShiftStrimziOperatorKafkaWithoutRegistryMessagingIT test also use kafka operator (strimzi-kafka-operator).
Despite the fact that strimzi-kafka-operator support Kafka 3.8.0, we don't want to downgrade version for all other tests across FW and TS.

OperatorOpenShiftAmqStreamsKafkaStreamIT is the single tests in our project using AMQ streams operator

Please select the relevant options.

• Bug fix (non-breaking change which fixes an issue)
• Dependency update
• Refactoring
• Backport
• New scenario (non-breaking change which adds functionality)
• This change requires a documentation update
• This change requires execution against OCP (use run tests phrase in comment)

Checklist:

• Methods and classes used in PR scenarios are meaningful
• Commits are well encapsulated and follow the best practices

[gtroitsk]

run tests

[github-actions]

Following jobs contain at least one flaky test:

• PR - Linux - JVM build - Latest Version

Run summary: https://github.com/quarkus-qe/quarkus-test-suite/actions/runs/12830637255?pr=2277

Flaky tests:

---

io.quarkus.ts.messaging.kafka.reactive.streams.KafkaGratefulShutdownIT.testConnection

• Failure message: Assertion condition defined as a Lambda expression in io.quarkus.test.utils.LogsVerifier Log does not contain any of '[ave]'. Full logs: [__ ____ __ _____ ___ __ ____ ______ , --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ , -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \ , --\___\_\____/_/ |_/_/|_/_/|_|\____/___/ , 14:16:49,227 SRMSG18229: Configured topics for channel 'slow': [slow], 14:16:49,242 SRMSG18214: Key deserializer omitted, using String as default, 14:16:49,242 SRMSG18234: Auto-commit disabled for channel slow, 14:16:49,420 SRMSG18240: 'throttled' commit strategy used for channel 'slow', 14:16:49,421 SRMSG18232: Will commit for group 'kafka-streams-reactive-messaging' every 5000 milliseconds., 14:16:49,421 SRMSG18236: Will mark throttled commit strategy for group 'kafka-streams-reactive-messaging' as unhealthy if records go more than 60000 milliseconds without being processed., 14:16:49,483 SRMSG18248: Key serializer omitted, using String as default, 14:16:49,520 SRMSG18258: Kafka producer kafka-producer-slow-topic, connected to Kafka brokers 'localhost:32778', is configured to write records to 'slow', 14:16:49,576 Delaying application startup until incoming channel 'slow' and outgoing channel 'slow-topic' are ready, 14:16:49,579 SRMSG18257: Kafka consumer kafka-consumer-slow, connected to Kafka brokers 'localhost:32778', belongs to the 'kafka-streams-reactive-messaging' consumer group and is configured to poll records from [slow], 14:16:49,718 [Consumer clientId=kafka-consumer-slow, groupId=kafka-streams-reactive-messaging] Error while fetching metadata with correlation id 2 : {slow=UNKNOWN_TOPIC_OR_PARTITION}, 14:16:49,732 Channels 'slow' and 'slow-topic' are ready, proceeding with application startup, 14:16:49,739 No Topology producer; Kafka Streams will not be started, 14:16:49,889 kafka-streams-reactive-messaging 1.0.0-SNAPSHOT on JVM (powered by Quarkus 999-SNAPSHOT) started in 1.571s. Listening on: http://0.0.0.0:1104, 14:16:49,889 Profile prod activated. , 14:16:49,889 Installed features: [cdi, kafka-client, kafka-streams, messaging, messaging-kafka, resteasy, resteasy-jsonb, smallrye-context-propagation, vertx], 14:16:51,909 SRMSG18209: Sending message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' to Kafka topic 'slow', 14:16:52,013 SRMSG18211: Message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' sent successfully to Kafka topic-partition 'slow-0', with offset 0] ==> expected: <true> but was: <false> within 30 seconds.
• Failed in jobs:
  • PR - Linux - JVM build - Latest Version

Failure stacktrace

org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.test.utils.LogsVerifier Log does not contain any of '[ave]'. Full logs: [__  ____  __  _____   ___  __ ____  ______ ,  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ ,  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   , --\___\_\____/_/ |_/_/|_/_/|_|\____/___/   , 14:16:49,227 SRMSG18229: Configured topics for channel 'slow': [slow], 14:16:49,242 SRMSG18214: Key deserializer omitted, using String as default, 14:16:49,242 SRMSG18234: Auto-commit disabled for channel slow, 14:16:49,420 SRMSG18240: 'throttled' commit strategy used for channel 'slow', 14:16:49,421 SRMSG18232: Will commit for group 'kafka-streams-reactive-messaging' every 5000 milliseconds., 14:16:49,421 SRMSG18236: Will mark throttled commit strategy for group 'kafka-streams-reactive-messaging' as unhealthy if records go more than 60000 milliseconds without being processed., 14:16:49,483 SRMSG18248: Key serializer omitted, using String as default, 14:16:49,520 SRMSG18258: Kafka producer kafka-producer-slow-topic, connected to Kafka brokers 'localhost:32778', is configured to write records to 'slow', 14:16:49,576 Delaying application startup until incoming channel 'slow' and outgoing channel 'slow-topic' are ready, 14:16:49,579 SRMSG18257: Kafka consumer kafka-consumer-slow, connected to Kafka brokers 'localhost:32778', belongs to the 'kafka-streams-reactive-messaging' consumer group and is configured to poll records from [slow], 14:16:49,718 [Consumer clientId=kafka-consumer-slow, groupId=kafka-streams-reactive-messaging] Error while fetching metadata with correlation id 2 : {slow=UNKNOWN_TOPIC_OR_PARTITION}, 14:16:49,732 Channels 'slow' and 'slow-topic' are ready, proceeding with application startup, 14:16:49,739 No Topology producer; Kafka Streams will not be started, 14:16:49,889 kafka-streams-reactive-messaging 1.0.0-SNAPSHOT on JVM (powered by Quarkus 999-SNAPSHOT) started in 1.571s. Listening on: http://0.0.0.0:1104, 14:16:49,889 Profile prod activated. , 14:16:49,889 Installed features: [cdi, kafka-client, kafka-streams, messaging, messaging-kafka, resteasy, resteasy-jsonb, smallrye-context-propagation, vertx], 14:16:51,909 SRMSG18209: Sending message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' to Kafka topic 'slow', 14:16:52,013 SRMSG18211: Message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' sent successfully to Kafka topic-partition 'slow-0', with offset 0] ==> expected: <true> but was: <false> within 30 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
	at io.quarkus.test.utils.AwaitilityUtils.untilAsserted(AwaitilityUtils.java:134)
	at io.quarkus.test.utils.AwaitilityUtils.untilAsserted(AwaitilityUtils.java:124)
	at io.quarkus.test.utils.LogsVerifier.assertContains(LogsVerifier.java:28)
	at io.quarkus.ts.messaging.kafka.reactive.streams.KafkaGratefulShutdownIT.testConnection(KafkaGratefulShutdownIT.java:56)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: org.opentest4j.AssertionFailedError: Log does not contain any of '[ave]'. Full logs: [__  ____  __  _____   ___  __ ____  ______ ,  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ ,  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   , --\___\_\____/_/ |_/_/|_/_/|_|\____/___/   , 14:16:49,227 SRMSG18229: Configured topics for channel 'slow': [slow], 14:16:49,242 SRMSG18214: Key deserializer omitted, using String as default, 14:16:49,242 SRMSG18234: Auto-commit disabled for channel slow, 14:16:49,420 SRMSG18240: 'throttled' commit strategy used for channel 'slow', 14:16:49,421 SRMSG18232: Will commit for group 'kafka-streams-reactive-messaging' every 5000 milliseconds., 14:16:49,421 SRMSG18236: Will mark throttled commit strategy for group 'kafka-streams-reactive-messaging' as unhealthy if records go more than 60000 milliseconds without being processed., 14:16:49,483 SRMSG18248: Key serializer omitted, using String as default, 14:16:49,520 SRMSG18258: Kafka producer kafka-producer-slow-topic, connected to Kafka brokers 'localhost:32778', is configured to write records to 'slow', 14:16:49,576 Delaying application startup until incoming channel 'slow' and outgoing channel 'slow-topic' are ready, 14:16:49,579 SRMSG18257: Kafka consumer kafka-consumer-slow, connected to Kafka brokers 'localhost:32778', belongs to the 'kafka-streams-reactive-messaging' consumer group and is configured to poll records from [slow], 14:16:49,718 [Consumer clientId=kafka-consumer-slow, groupId=kafka-streams-reactive-messaging] Error while fetching metadata with correlation id 2 : {slow=UNKNOWN_TOPIC_OR_PARTITION}, 14:16:49,732 Channels 'slow' and 'slow-topic' are ready, proceeding with application startup, 14:16:49,739 No Topology producer; Kafka Streams will not be started, 14:16:49,889 kafka-streams-reactive-messaging 1.0.0-SNAPSHOT on JVM (powered by Quarkus 999-SNAPSHOT) started in 1.571s. Listening on: http://0.0.0.0:1104, 14:16:49,889 Profile prod activated. , 14:16:49,889 Installed features: [cdi, kafka-client, kafka-streams, messaging, messaging-kafka, resteasy, resteasy-jsonb, smallrye-context-propagation, vertx], 14:16:51,909 SRMSG18209: Sending message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' to Kafka topic 'slow', 14:16:52,013 SRMSG18211: Message org.eclipse.microprofile.reactive.messaging.Message$5@76570ae5 from channel 'slow-topic' sent successfully to Kafka topic-partition 'slow-0', with offset 0] ==> expected: <true> but was: <false>
	at io.quarkus.test.utils.LogsVerifier.lambda$assertContains$0(LogsVerifier.java:30)
	at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
	at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:248)
	at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:235)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

---

[michalvavrik]

Thank you for the fix