Run minimal watcher tempest api test

[raukadah]

Thanks to watcher api service and tls integration, we are now able to run watcher_tempest_plugin.tests.api.admin.test_api_discovery.TestApiDiscovery[1] test in EDPM CI.

It will act as a base for keep existing watcher integration healthy.

Links:
[1]. #47 (comment)

Related-Jira: https://issues.redhat.com/browse/OSPRH-11948

Depends-On: openstack-k8s-operators/ci-framework#2675

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[softwarefactory-project-zuul]

This change depends on a change that failed to merge.

Changes #46, openstack-k8s-operators/ci-framework#2658 are needed.

[softwarefactory-project-zuul]

This change depends on a change that failed to merge.

Change #46 is needed.

[raukadah]

recheck

[softwarefactory-project-zuul]

This change depends on a change that failed to merge.

Changes #46, openstack-k8s-operators/ci-framework#2658 are needed.

[raukadah]

recheck

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/494b306e4db94de3aa9c0576defa30b9

✔️ noop SUCCESS in 0s
✔️ openstack-meta-content-provider SUCCESS in 1h 20m 19s
❌ watcher-operator-validation FAILURE in 1h 09m 05s
✔️ watcher-operator-kuttl SUCCESS in 35m 30s

[raukadah]

recheck

[softwarefactory-project-zuul]

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/watcher-operator for 45,365ebe2c44fe26f2c70a284e81da2625ab13f4e9

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/06d4e3bda72e4a2e99109158ca7978e0

✔️ noop SUCCESS in 0s
✔️ openstack-meta-content-provider SUCCESS in 1h 19m 47s
❌ watcher-operator-validation FAILURE in 1h 08m 47s
✔️ watcher-operator-kuttl SUCCESS in 35m 36s

[cescgina]

this will need at least #40 to work. With #46, the api pod does not have the CA and requests with SSL to keystone fail:

2025-01-22 07:42:10.204 18 CRITICAL keystonemiddleware.auth_token [None req-e418a60f-33cd-49c6-aabf-9f074282567c - - - - - -] Unable to validate token: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)'))): keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)')))

see https://logserver.rdoproject.org/47/47/c764c39c4252c7041fc03f3be734b0931666cd7c/github-check/watcher-operator-validation/4c4862c/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/pods/watcher-api-647b98f55-p7rfh/logs/watcher-api-log.log. With #40 the api commands work for me locally because it adds the CA as a mount to the pod

[raukadah]

this will need at least #40 to work. With #46, the api pod does not have the CA and requests with SSL to keystone fail:

2025-01-22 07:42:10.204 18 CRITICAL keystonemiddleware.auth_token [None req-e418a60f-33cd-49c6-aabf-9f074282567c - - - - - -] Unable to validate token: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)'))): keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)')))

see https://logserver.rdoproject.org/47/47/c764c39c4252c7041fc03f3be734b0931666cd7c/github-check/watcher-operator-validation/4c4862c/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/pods/watcher-api-647b98f55-p7rfh/logs/watcher-api-log.log. With #40 the api commands work for me locally because it adds the CA as a mount to the pod

Thank you @cescgina for the suggestion. let me try that.

[softwarefactory-project-zuul]

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/watcher-operator for 47,73113aaeb27119e58fc4dbcec522afa300171c92

[cescgina]

this will need at least #40 to work. With #46, the api pod does not have the CA and requests with SSL to keystone fail:

2025-01-22 07:42:10.204 18 CRITICAL keystonemiddleware.auth_token [None req-e418a60f-33cd-49c6-aabf-9f074282567c - - - - - -] Unable to validate token: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)'))): keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://keystone-internal.openstack.svc:5000: HTTPSConnectionPool(host='keystone-internal.openstack.svc', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1147)')))

see https://logserver.rdoproject.org/47/47/c764c39c4252c7041fc03f3be734b0931666cd7c/github-check/watcher-operator-validation/4c4862c/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/pods/watcher-api-647b98f55-p7rfh/logs/watcher-api-log.log. With #40 the api commands work for me locally because it adds the CA as a mount to the pod

Thank you @cescgina for the suggestion. let me try that.

actually, the PRs are now changed, so #45 is the want you want to try with

[softwarefactory-project-zuul]

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/watcher-operator for 47,98e1536f312b9383efca8a3ba10492e4c4a31a94

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/6c97ff6ed42b44d2bcf47421d586063a

✔️ noop SUCCESS in 0s
✔️ openstack-meta-content-provider SUCCESS in 1h 23m 13s
❌ watcher-operator-validation FAILURE in 1h 10m 31s
✔️ watcher-operator-kuttl SUCCESS in 36m 21s

[raukadah]

Initial tempest test run result: https://logserver.rdoproject.org/47/47/c75f864df1bc8060a9130c458a28c0df58e11a1b/github-check/watcher-operator-validation/8d8d731/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest/stestr_results.html

watcher_tempest_plugin.tests.api.admin.test_api_discovery.TestApiDiscovery

Rest of them are failing due to following reason.

ft12.3: watcher_tempest_plugin.tests.api.admin.test_goal.TestShowListGoal.test_show_goal_with_links[smoke]testtools.testresult.real._StringException: pythonlogging:'': {{{
2025-01-22 14:08:30,632 27 INFO     [tempest.lib.common.rest_client] Request (TestShowListGoal:test_show_goal_with_links): 404 GET http://watcher-public-openstack.apps-crc.testing/v1/goals/dummy 0.185s
2025-01-22 14:08:30,632 27 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Wed, 22 Jan 2025 14:08:30 GMT', 'server': 'Apache', 'content-length': '121', 'vary': 'OpenStack-API-Version', 'openstack-api-minimum-version': '1.0', 'openstack-api-maximum-version': '1.4', 'openstack-api-version': 'infra-optim 1.0', 'content-type': 'application/json', 'set-cookie': '0a98f3d612a3c8fda809da40f2e41782=84b472899a5958b33873fe14901e1bd5; path=/; HttpOnly', 'connection': 'close', 'status': '404', 'content-location': 'http://watcher-public-openstack.apps-crc.testing/v1/goals/dummy'}
        Body: b'{"error_message": "{\\"faultcode\\": \\"Client\\", \\"faultstring\\": \\"Goal dummy could not be found\\", \\"debuginfo\\": null}"}'
}}}

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/watcher_tempest_plugin/tests/api/admin/test_goal.py", line 49, in test_show_goal_with_links
    _, goal = self.client.show_goal(self.DUMMY_GOAL)
  File "/usr/lib/python3.9/site-packages/watcher_tempest_plugin/services/base.py", line 37, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/watcher_tempest_plugin/services/infra_optim/v1/json/client.py", line 247, in show_goal
    return self._show_request('/goals', goal)
  File "/usr/lib/python3.9/site-packages/watcher_tempest_plugin/services/base.py", line 132, in _show_request
    resp, body = self.get(uri)
  File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 342, in get
    return self.request('GET', url, extra_headers, headers,
  File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 762, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 872, in _error_checker
    raise exceptions.NotFound(resp_body, resp=resp)
tempest.lib.exceptions.NotFound: Object not found
Details: {'error_message': '{"faultcode": "Client", "faultstring": "Goal dummy could not be found", "debuginfo": null}'}

Goals comes as a part of decision engine. then rest of api test will work

[cescgina]

thanks @raukadah this lgtm and IMO it would be great to merge asap, but we need to merge #45 before (just leaving a note for other reviewers) maybe we should add a hold to prevent merge before?

[softwarefactory-project-zuul]

This change depends on a change that failed to merge.

Change openstack-k8s-operators/ci-framework#2675 is needed.

[raukadah]

recheck

[amoralej]

/lgtm

[marios]

/lgtm

[amoralej]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amoralej

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [amoralej]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment