Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests Requiring Helper Pods Fail in Air-Gapped OpenShift Cluster with Private Registry Authentication. #731

Open
Yashi5108 opened this issue Dec 30, 2024 · 0 comments

Comments

@Yashi5108
Copy link

Yashi5108 commented Dec 30, 2024

BUG REPORT

What happened:

I’m working on fixing OpenShift failures in LitmusChaos and identified an issue with the set_helper_data flag in the go-runner code. For OpenShift, the set_helper_data flag needs to be set to false. However, when this is done, the SetHelperData function sets ImagePullSecrets to nil. This results in the helper pod failing to pull its image in air-gapped clusters, as the required ImagePullSecrets are missing.

What you expected to happen:

The helper pod should be able to pull the required images successfully in air-gapped environments, even when the set_helper_data flag is set to false.

How to reproduce it (as minimally and precisely as possible):

  1. Set up an air-gapped OpenShift cluster with a private registry that requires authentication.
  2. Attempt to run a LitmusChaos test that requires a helper pod.
  3. The helper pod will fail to pull the image because ImagePullSecrets are set to nil due to the set_helper_data flag being set to false.

Anything else we need to know?:

  1. Solution: Update the SetHelperData function in the go-runner code to ensure that ImagePullSecrets are retained even when the set_helper_data flag is set to false. This will allow the helper pods to successfully pull images in air-gapped clusters.
  2. Temporary Workaround: Until the fix is merged into go-runner, create a custom image with the necessary code changes and use it in the setup to temporarily resolve the issue.

Code Link:

The issue lies in the following go-runner code.

Function to be changed:

func SetHelperData(chaosDetails *types.ChaosDetails, setHelperData string, clients clients.ClientSets) error {
	var pod *core_v1.Pod
	pod, err := GetExperimentPod(chaosDetails.ChaosPodName, chaosDetails.ChaosNamespace, clients)
	if err != nil {
		return err
	}

	// Get Labels
	labels := pod.ObjectMeta.Labels
	for label := range labels {
		if strings.HasSuffix(label, "job-name") || strings.HasSuffix(label, "controller-uid") {
			delete(labels, label)
		}
	}
	chaosDetails.Labels = labels

	switch setHelperData {
	case "false":
		return nil

	default:

		// Get Chaos Pod Annotation
		chaosDetails.Annotations = pod.Annotations

		// Get ImagePullSecrets
		chaosDetails.ImagePullSecrets = pod.Spec.ImagePullSecrets

		// Get Resource Requirements
		chaosDetails.Resources, err = getChaosPodResourceRequirements(pod, chaosDetails.ExperimentName)
		if err != nil {
			return stacktrace.Propagate(err, "could not inherit resource requirements")
		}
		return nil
	}
}
@Yashi5108 Yashi5108 changed the title Tests with helper pods fail in air-gapped mode with private registry requiring authentication. Tests Requiring Helper Pods Fail in Air-Gapped OpenShift Cluster with Private Registry Authentication. Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant