RHSM integration for prefetch task

pipelines/docker-build-multi-platform-oci-ta/README.md

@@ -135,10 +135,10 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### prefetch-dependencies-oci-ta:0.1 task parameters
 |name|description|default value|already set by|
 |---|---|---|---|
+|ACTIVATION_KEY| Name of secret which contains subscription activation key| activation-key| |
 |SOURCE_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the application source code.| None| '$(tasks.clone-repository.results.SOURCE_ARTIFACT)'|
 |caTrustConfigMapKey| The name of the key in the ConfigMap that contains the CA bundle data.| ca-bundle.crt| |
 |caTrustConfigMapName| The name of the ConfigMap to read CA bundle data from.| trusted-ca| |
-|config-file-content| Pass configuration to cachi2. Note this needs to be passed as a YAML-formatted config dump, not as a file path! | | |
 |dev-package-managers| Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk. | false| |
 |input| Configures project packages that will have their dependencies prefetched.| None| '$(params.prefetch-input)'|
 |log-level| Set cachi2 log level (debug, info, warning, error)| info| |

pipelines/docker-build-oci-ta/README.md

@@ -132,10 +132,10 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### prefetch-dependencies-oci-ta:0.1 task parameters
 |name|description|default value|already set by|
 |---|---|---|---|
+|ACTIVATION_KEY| Name of secret which contains subscription activation key| activation-key| |
 |SOURCE_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the application source code.| None| '$(tasks.clone-repository.results.SOURCE_ARTIFACT)'|
 |caTrustConfigMapKey| The name of the key in the ConfigMap that contains the CA bundle data.| ca-bundle.crt| |
 |caTrustConfigMapName| The name of the ConfigMap to read CA bundle data from.| trusted-ca| |
-|config-file-content| Pass configuration to cachi2. Note this needs to be passed as a YAML-formatted config dump, not as a file path! | | |
 |dev-package-managers| Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk. | false| |
 |input| Configures project packages that will have their dependencies prefetched.| None| '$(params.prefetch-input)'|
 |log-level| Set cachi2 log level (debug, info, warning, error)| info| |

pipelines/docker-build/README.md

@@ -131,9 +131,9 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### prefetch-dependencies:0.1 task parameters
 |name|description|default value|already set by|
 |---|---|---|---|
+|ACTIVATION_KEY| Name of secret which contains subscription activation key| activation-key| |
 |caTrustConfigMapKey| The name of the key in the ConfigMap that contains the CA bundle data.| ca-bundle.crt| |
 |caTrustConfigMapName| The name of the ConfigMap to read CA bundle data from.| trusted-ca| |
-|config-file-content| Pass configuration to cachi2. Note this needs to be passed as a YAML-formatted config dump, not as a file path! | | |
 |dev-package-managers| Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk. | false| |
 |input| Configures project packages that will have their dependencies prefetched.| None| '$(params.prefetch-input)'|
 |log-level| Set cachi2 log level (debug, info, warning, error)| info| |

pipelines/tekton-bundle-builder/README.md

@@ -66,9 +66,9 @@
 ### prefetch-dependencies:0.1 task parameters
 |name|description|default value|already set by|
 |---|---|---|---|
+|ACTIVATION_KEY| Name of secret which contains subscription activation key| activation-key| |
 |caTrustConfigMapKey| The name of the key in the ConfigMap that contains the CA bundle data.| ca-bundle.crt| |
 |caTrustConfigMapName| The name of the ConfigMap to read CA bundle data from.| trusted-ca| |
-|config-file-content| Pass configuration to cachi2. Note this needs to be passed as a YAML-formatted config dump, not as a file path! | | |
 |dev-package-managers| Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk. | false| |
 |input| Configures project packages that will have their dependencies prefetched.| None| '$(params.prefetch-input)'|
 |log-level| Set cachi2 log level (debug, info, warning, error)| info| |

task/prefetch-dependencies-oci-ta/0.1/README.md

@@ -26,10 +26,10 @@ params:
 ## Parameters
 |name|description|default value|required|
 |---|---|---|---|
+|ACTIVATION_KEY|Name of secret which contains subscription activation key|activation-key|false|
 |SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|
 |caTrustConfigMapKey|The name of the key in the ConfigMap that contains the CA bundle data.|ca-bundle.crt|false|
 |caTrustConfigMapName|The name of the ConfigMap to read CA bundle data from.|trusted-ca|false|
-|config-file-content|Pass configuration to cachi2. Note this needs to be passed as a YAML-formatted config dump, not as a file path! |""|false|
 |dev-package-managers|Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk. |false|false|
 |input|Configures project packages that will have their dependencies prefetched.||true|
 |log-level|Set cachi2 log level (debug, info, warning, error)|info|false|

task/prefetch-dependencies-oci-ta/0.1/prefetch-dependencies-oci-ta.yaml

@@ -33,6 +33,10 @@ spec:
 
     [available configuration parameters]: https://github.com/containerbuildsystem/cachi2?tab=readme-ov-file#available-configuration-parameters
   params:
+    - name: ACTIVATION_KEY
+      description: Name of secret which contains subscription activation key
+      type: string
+      default: activation-key
     - name: SOURCE_ARTIFACT
       description: The Trusted Artifact URI pointing to the artifact with
         the application source code.
@@ -46,11 +50,6 @@ spec:
       description: The name of the ConfigMap to read CA bundle data from.
       type: string
       default: trusted-ca
-    - name: config-file-content
-      description: |
-        Pass configuration to cachi2.
-        Note this needs to be passed as a YAML-formatted config dump, not as a file path!
-      default: ""
     - name: dev-package-managers
       description: |
         Enable in-development package managers. WARNING: the behavior may change at any time without notice. Use at your own risk.
@@ -79,7 +78,13 @@ spec:
         the application source code.
       type: string
   volumes:
-    - name: config
+    - name: activation-key
+      secret:
+        optional: true
+        secretName: $(params.ACTIVATION_KEY)
+    - name: etc-pki-entitlement
+      emptyDir: {}
+    - name: shared
       emptyDir: {}
     - name: trusted-ca
       configMap:
@@ -104,12 +109,7 @@ spec:
         performing http(s) requests.
       optional: true
   stepTemplate:
-    env:
-      - name: CONFIG_FILE_CONTENT
-        value: $(params.config-file-content)
     volumeMounts:
-      - mountPath: /mnt/config
-        name: config
       - mountPath: /var/workdir
         name: workdir
   steps:
@@ -130,7 +130,7 @@ spec:
           echo -n "" >$(results.CACHI2_ARTIFACT.path)
         fi
     - name: use-trusted-artifact
-      image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:e0e457b6af10e44ff6b90208a9e69adc863a865e1c062c4cb84bf3846037d74d
+      image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:81c4864dae6bb11595f657be887e205262e70086a05ed16ada827fd6391926ac
       args:
         - use
         - $(params.SOURCE_ARTIFACT)=/var/workdir/source
@@ -143,15 +143,158 @@ spec:
           # https://github.com/containerbuildsystem/cachi2/issues/577
           yq 'del(.goproxy_url)' <<<"${CONFIG_FILE_CONTENT}" >/mnt/config/config.yaml
         fi
-    - name: prefetch-dependencies
+    - name: check-prefetch-input
       image: quay.io/redhat-appstudio/cachi2:0.13.0@sha256:eb34cfe3fea20997eebd8164dc93eedb2fd7a60dc1fb4afcc1b1ff43df9d6667
+      env:
+        - name: INPUT
+          value: $(params.input)
+      script: |
+        if [ -z "${INPUT}" ]; then
+          # Confirm input was provided though it's likely the whole task would be skipped if it wasn't
+          echo "No prefetch will be performed because no input was provided for cachi2 fetch-deps"
+          exit 0
+        fi
+    - name: register-red-hat
+      image: quay.io/redhat-appstudio/cachi2@sha256:eb34cfe3fea20997eebd8164dc93eedb2fd7a60dc1fb4afcc1b1ff43df9d6667
+      results:
+        - name: registered
+          type: string
+      volumeMounts:
+        - mountPath: /shared
+          name: shared
+        - mountPath: /activation-key
+          name: activation-key
+      env:
+        - name: INPUT
+          value: $(params.input)
+        - name: ACTIVATION_KEY
+          value: $(params.ACTIVATION_KEY)
+      script: |
+        #!/bin/bash
+        echo "false" >/shared/registered
+        ACTIVATION_KEY_PATH="/activation-key"
+
+        if [ -e /activation-key/org ]; then
+          cp -r --preserve=mode "$ACTIVATION_KEY_PATH" /tmp/activation-key
+          mkdir /shared/rhsm
+
+          echo "Registering with Red Hat subscription manager."
+          subscription-manager register --org "$(cat /tmp/activation-key/org)" --activationkey "$(cat /tmp/activation-key/activationkey)"
+
+          # copy generated certificates to /shared/rhsm
+          cp /etc/pki/entitlement/*.pem /shared/rhsm
+
+          file="$(find /shared/rhsm -regextype egrep -regex '.*[0-9]+\.pem' -printf %f)"
+          echo "file: $file"
+          basename "$file" .pem >/shared/RHSM_ID
+          echo "./RHSM_ID:"
+          cat /shared/RHSM_ID
+
+          # trust the CA used for Red Hat CDN
+          cp /etc/rhsm-host/ca/redhat-uep.pem /shared/rhsm/redhat-uep.pem
+        fi
+    - name: preprocess-input
+      image: quay.io/redhat-appstudio/cachi2@sha256:eb34cfe3fea20997eebd8164dc93eedb2fd7a60dc1fb4afcc1b1ff43df9d6667
+      args:
+        - $(params.input)
+      volumeMounts:
+        - mountPath: /shared
+          name: shared
+      env:
+        - name: INPUT
+          value: $(params.input)
+        - name: ACTIVATION_KEY
+          value: $(params.ACTIVATION_KEY)
+      script: |
+        #!/bin/python3
+        import json
+        import os
+        import sys
+
+        def string_to_json(input: str):
+            if input in ["gomod", "pip", "npm", "yarn", "bundler", "rpm"]:
+                input = '{"type": "%s"}' % input
+                print("json: %s" % input)
+            return input
+
+
+        def json_to_list(input: str):
+            input = json.loads(input)
+            if type(input) is dict:
+                input = [input]
+            return json.dumps(input)
+
+
+        def inject_certs(input: str, rhsm_id: str):
+            input = json.loads(input)
+            if type(input is list):
+                cert = ("/shared/rhsm/%s.pem" % rhsm_id)
+                key = ("/shared/rhsm/%s-key.pem" % rhsm_id)
+                ca_bundle = os.getenv("CA_BUNDLE", None)
+
+                for pkg_man in input:
+                    if pkg_man["type"] == "rpm":
+
+                        # preserve verify setting
+                        verify = \
+                            pkg_man.get("options", {}).get("ssl", {}).get("verify", 1)
+
+                        # preserve other options
+                        options = pkg_man.get('options', {})
+
+                        dnf = options.get('options', None)
+
+                        ssl_options = {
+                            "client_key": key,
+                            "client_cert": cert,
+                            "ca_bundle": ca_bundle,
+                            "ssl_verify": verify}
+
+                        options['ssl'] = ssl_options
+                        pkg_man["options"] = options
+                return (json.dumps(input))
+
+            else:
+                # throw an error
+                print("boooo!")
+
+
+        def convert_input(input, rhsm_id):
+            input = string_to_json(input)
+            input = json_to_list(input)
+            input = inject_certs(input, rhsm_id)
+            return input
+
+
+        if __name__ == '__main__':
+            rhsm_id = ""
+            input = ""
+
+            try:
+                f = open("/shared/RHSM_ID", "r")
+                rhsm_id = f.read().strip("\n")
+                print("RHSM ID is: %s" % rhsm_id)
+            except:
+                print("No RHSM ID found.")
+                input = sys.argv[1]
+
+            if input == "":
+                input = convert_input(sys.argv[1], rhsm_id)
+
+            print("Preprocessing result: %s" % input)
+            with open('/shared/rhsm/preprocessed_input', 'w') as f:
+                f.write(input)
+    - name: prefetch-dependencies
+      image: quay.io/redhat-appstudio/cachi2@sha256:eb34cfe3fea20997eebd8164dc93eedb2fd7a60dc1fb4afcc1b1ff43df9d6667
       volumeMounts:
         - mountPath: /mnt/trusted-ca
           name: trusted-ca
           readOnly: true
+        - mountPath: /activation-key
+          name: activation-key
+        - mountPath: /shared
+          name: shared
       env:
-        - name: INPUT
-          value: $(params.input)
         - name: DEV_PACKAGE_MANAGERS
           value: $(params.dev-package-managers)
         - name: LOG_LEVEL
@@ -165,16 +308,29 @@ spec:
         - name: WORKSPACE_NETRC_PATH
           value: $(workspaces.netrc.path)
       script: |
-        if [ -z "${INPUT}" ]; then
-          # Confirm input was provided though it's likely the whole task would be skipped if it wasn't
-          echo "No prefetch will be performed because no input was provided for cachi2 fetch-deps"
-          exit 0
-        fi
+        #!/bin/bash
+        # Function for cleanup on script exit
+        # cleanup_on_exit() {
+        #   echo "Performing cleanup tasks before script exit..."
 
-        if [ -f /mnt/config/config.yaml ]; then
-          config_flag=--config-file=/mnt/config/config.yaml
-        else
-          config_flag=""
+        #   # run any needed cleanup
+        #   rv=$?
+        #   subscription-manager unregister
+        #   exit "$rv"
+        # }
+
+        # this always returns "/tekton/scripts/script-6-tj9qp: line 1: cleanup_on_exit: command not found"
+        # trap 'cleanup_on_exit' EXIT
+
+        INPUT=$(cat /shared/rhsm/preprocessed_input)
+        export INPUT
+
+        # trust Red Hat CA cert used for Red Hat CDN
+        ls /shared/rhsm/
+        if [ -f /shared/rhsm/redhat-uep.pem ]; then
+          echo "Adding Red Hat CA certificate to trusted roots."
+          cp /shared/rhsm/redhat-uep.pem /etc/pki/ca-trust/source/anchors/
+          update-ca-trust
         fi
 
         if [ "$DEV_PACKAGE_MANAGERS" = "true" ]; then
@@ -212,7 +368,7 @@ spec:
           update-ca-trust
         fi
 
-        cachi2 --log-level="$LOG_LEVEL" $config_flag fetch-deps \
+        cachi2 --log-level="$LOG_LEVEL" fetch-deps \
           $dev_pacman_flag \
           --source=/var/workdir/source \
           --output=/var/workdir/cachi2/output \
@@ -226,7 +382,7 @@ spec:
         cachi2 --log-level="$LOG_LEVEL" inject-files /var/workdir/cachi2/output \
           --for-output-dir=/cachi2/output
     - name: create-trusted-artifact
-      image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:e0e457b6af10e44ff6b90208a9e69adc863a865e1c062c4cb84bf3846037d74d
+      image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:81c4864dae6bb11595f657be887e205262e70086a05ed16ada827fd6391926ac
       args:
         - create
         - --store