What's the expected behavior for rescued tasks in a *finalizer*?

[markuszoeller]

Type of question

How to implement a specific feature

Question

What did you do?

I've implemented a finalizer for my CRD and some tasks in there are in a rescue block and if rescued, the custom resource does NOT get deleted, but the delete is tried forever while there's an error at the end of the finalizer playbook run:

{"level":"error","ts":"2024-08-21T06:06:27Z","msg":"Reconciler error","controller":"mycrd-controller","object":{"name":"foo","namespace":"bar"},"namespace":"bar","name":"foo","reconcileID":"974d2d8e-dc42-4420-ad69-7b0e2f69fc07","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:227"}

What did you expect to see?

The tasks in the finalizer that got rescued do not block the deletion of the custom resource

What did you see instead? Under which circumstances?

The tasks in the finalizer that got rescued DO block the deletion of the custom resource and it gets reconciled over and over.

Environment

Operator type:

language ansible

Kubernetes cluster type:

OpenShift

$ operator-sdk version

bash-5.1$ ansible-operator version
ansible-operator version: "v1.35.0-dirty", commit: "42b5d80c75f1ddda8f2dbe1629b9454d366a8d6a", kubernetes version: "v1.29.0", go version: "go1.22.5", GOOS: "linux", GOARCH: "amd64"

$ go version (if language is Go)

$ kubectl version

Client Version: v1.29.5
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.11+add48d0

Additional context

Considering rescue block seems to have been introduced with https://github.com/operator-framework/operator-sdk/pull/3650/files but I'm wondering if that also applies to finalizers.

[openshift-ci]

@markuszoeller: The label(s) language/ansible cannot be applied, because the repository doesn't have them.

In response to this:

Type of question

How to implement a specific feature

Question

What did you do?

I've implemented a finalizer for my CRD and some tasks in there are in a rescue block and if rescued, the custom resource does NOT get deleted, but the delete is tried forever while there's an error at the end of the finalizer playbook run:

{"level":"error","ts":"2024-08-21T06:06:27Z","msg":"Reconciler error","controller":"mycrd-controller","object":{"name":"foo","namespace":"bar"},"namespace":"bar","name":"foo","reconcileID":"974d2d8e-dc42-4420-ad69-7b0e2f69fc07","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
  /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
  /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
  /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:227"}

What did you expect to see?

The tasks in the finalizer that got rescued do not block the deletion of the custom resource

What did you see instead? Under which circumstances?

The tasks in the finalizer that got rescued DO block the deletion of the custom resource and it gets reconciled over and over.

Environment

Operator type:

/language ansible

Kubernetes cluster type:

OpenShift

$ operator-sdk version

bash-5.1$ ansible-operator version
ansible-operator version: "v1.35.0-dirty", commit: "42b5d80c75f1ddda8f2dbe1629b9454d366a8d6a", kubernetes version: "v1.29.0", go version: "go1.22.5", GOOS: "linux", GOARCH: "amd64"

$ go version (if language is Go)

$ kubectl version

Client Version: v1.29.5
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.11+add48d0

Additional context

Considering rescue block seems to have been introduced with https://github.com/operator-framework/operator-sdk/pull/3650/files but I'm wondering if that also applies to finalizers.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.