chore: match infra OCP credentialsMode with stackrox CI

[gavin-stackrox]

stackrox/stackrox CI usescredentialsMode: Passthrough to reduce the number of IAM policy bindings. Infra should default to provide the same type of cluster. This PR provides a parameter to override the default.

Testing

• 2 x default (Passthrough) OCP flavors - 7 bindings should be created per cluster

$ INFRA_TOKEN="$STAGE_INFRA_TOKEN" bin/infractl -k -e localhost:8443 create openshift-4
ID: gj-11-08-1

UI defaults for openshift-4-perf-scale: https://localhost:8443/cluster/gj-11-08-spotless-scissors-b

• 1 x Default OCP flavor - 21 bindings per cluster

$ INFRA_TOKEN="$STAGE_INFRA_TOKEN" bin/infractl -k -e localhost:8443 create openshift-4-demo --arg credentials-mode=Mint
ID: gj-11-08-3

[ghost]

A single node development cluster (infra-pr-1057) was allocated in production infra for this PR.

CI will attempt to deploy us.gcr.io/stackrox-infra/infra-server:0.8.6-10-gbe78e79af4 to it.

🔌 You can connect to this cluster with:

gcloud container clusters get-credentials infra-pr-1057 --zone us-central1-a --project acs-team-temp-dev

🛠️ And pull infractl from the deployed dev infra-server with:

nohup kubectl -n infra port-forward svc/infra-server-service 8443:8443 &
make pull-infractl-from-dev-server

🚲 You can then use the dev infra instance e.g.:

bin/infractl -k -e localhost:8443 whoami

⚠️ Any clusters that you start using your dev infra instance should have a lifespan shorter then the development cluster instance. Otherwise they will not be destroyed when the dev infra instance ceases to exist when the development cluster is deleted. ⚠️

Further Development

☕ If you make changes, you can commit and push and CI will take care of updating the development cluster.

🚀 If you only modify configuration (chart/infra-server/configuration) or templates (chart/infra-server/{static,templates}), you can get a faster update with:

make install-local

Logs

Logs for the development infra depending on your @redhat.com authuser:

• authuser=0
• authuser=1
• authuser=2

Or:

kubectl -n infra logs -l app=infra-server --tail=1 -f

[gavin-stackrox]

While the binding count was correct, the clusters did not come up. I'd like to blame this on quay.io outage Nov/8/24 but it might be something else:

2023-11-09T01:01:07.780058667Z time="2023-11-09T00:57:52Z" level=info msg="Cluster operator insights SCAAvailable is False with NotFound: Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: OCM API https://api.openshift.com/api/accounts_mgmt/v1/certificates returned HTTP 404: {\"code\":\"ACCT-MGMT-7\",\"href\":\"/api/accounts_mgmt/v1/errors/7\",\"id\":\"7\",\"kind\":\"Error\",\"operation_id\":\"34a0c0f8-c8cf-4962-af2c-f437ffc86092\",\"reason\":\"The organization (id= 1bpJG2L8SFjFkFDm7pHXsGsu0XU) does not have any certificate of type sca. Enable SCA at https://access.redhat.com/management.\"}"
2023-11-09T01:01:07.780064478Z time="2023-11-09T00:57:52Z" level=info msg="Cluster operator network ManagementStateDegraded is False with : "
2023-11-09T01:01:07.780070633Z time="2023-11-09T00:57:52Z" level=error msg="Cluster initialization failed because one or more operators are not functioning properly.\nThe cluster should be accessible for troubleshooting as detailed in the documentation linked below,\nhttps://docs.openshift.com/container-platform/latest/support/troubleshooting/troubleshooting-installations.html\nThe 'wait-for install-complete' subcommand can then be used to continue the installation"
2023-11-09T01:01:07.780084330Z time="2023-11-09T00:57:52Z" level=error msg="failed to initialize the cluster: Cluster operator image-registry is not available"

[tommartensen]

Even though I approved the PR...
Why is ocpCredentialsMode a chart annotation and not a value in one of the secret configuration or additional values files?

The reason acsDemoVersion is an annotation is that

1. it describes the infra server behaviour (different demo version => different infra server behavior)
2. frequently changes & needs history
3. can be updated by automation & reviewed

[gavin-stackrox]

It seemed like a good place for constants to me. If that is an anti-pattern it can move to Constants.yaml or somesuch.

[tommartensen]

we could start a values.yaml that is checked into the repo and also put acsDemoVersion there.