add custom trusted-CA handling to getters x509.SystemCertPool AppendCertsFromPEM or via env spec SSL_CERT_FILE

[shalberd]

Kubeflow kfdef / kfctl uses your libraries to download tarballs from HTTPS-URLs.

https://github.com/kubeflow/kfctl/search?q=%22gogetter.getany%22

In an enterprise environment, very often, https-certificates are issued by an internal PKI whose CAs and intermediate CAs are by definition not publicly-trusted. The concept of private PKIs is explained here: https://www.digicert.com/support/resources/faq/identity-and-access-trust/what-is-private-pki-vs-public-pki

There have been requests regarding non-public locations of files in Kubeflow, but my guess is the science-oriented community there does not want to take the time to iron out the dependencies or ask their dependenc

For downloading with python http.get, the concept and environment is explained here in detail for per-request PEM file verify of SSL chain.

opendatahub-io-contrib/jupyterhub-odh#137

More globally and at a go-Level, can you add support for adding custom CAs from a PEM encoded file to the x509.SystemCertPool, please? This would make your tooling much more enterprise-ready and also help make Kubeflow installs more enterprise-ready, especially with regards to on-prem. I think you already have http proxy support and authentication support. Only thing missing now is adding non-public CAs from enterprise private PKIs.

I suppose the location of the file in question is this one:

https://github.com/hashicorp/go-cleanhttp/blob/master/cleanhttp.go

The approach "Append the self-signed cert or CAs to an in-app copy of the host system trust store" is defined here:

https://forfuncsake.github.io/post/2017/08/trust-extra-ca-cert-in-go-app/

This post is quite old, though, from 2017.

My idea in another context, Open Data Hub with Openshift file-based location for trusted CAs, could work here, too.

That is, if we could define a filesystem location for the ca-bundle file, then do AppendCertsFromPEM.

I am not familiar enough with golang to say if an env-based solution is already there and used implicitely by your functions, like here

golang/go@e83bcd9

What do you think?

e.g. env SSL_CERT_FILE read from a configmap ca-bundle.crt section, if that env were present, would your http go-getter routines use that for chain of trust SSL validation?

[shalberd]

SSL_CERT_FILE env works.