diff --git a/docs/container_storage_provider/hpe_alletra_9000/index.md b/docs/container_storage_provider/hpe_alletra_storage_mp/index.md similarity index 93% rename from docs/container_storage_provider/hpe_alletra_9000/index.md rename to docs/container_storage_provider/hpe_alletra_storage_mp/index.md index ed5dabce..fe23f962 100644 --- a/docs/container_storage_provider/hpe_alletra_9000/index.md +++ b/docs/container_storage_provider/hpe_alletra_storage_mp/index.md @@ -1,11 +1,14 @@ # Introduction -The HPE Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) for Kubernetes is part of the [HPE CSI Driver for Kubernetes](../../csi_driver/index.md). The CSP abstract the data management capabilities of the array for use by Kubernetes. +The HPE Alletra Storage MP, Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) for Kubernetes is part of the [HPE CSI Driver for Kubernetes](../../csi_driver/index.md). The CSP abstract the data management capabilities of the array for use by Kubernetes. + +!!! note + The HPE CSI Driver for Kubernetes is only compatible with HPE Alletra Storage MP running with block services, such as HPE GreenLake for Block Storage. [TOC] !!! note - For help getting started with deploying the HPE CSI Driver using HPE Alletra 9000, Primera or 3PAR storage, check out the [tutorial over at HPE Developer](https://developer.hpe.com/blog/9o7zJkqlX5cErkrzgopL/tutorial-how-to-get-started-with-the-hpe-csi-driver-and-hpe-primera-and-). + For help getting started with deploying the HPE CSI Driver using HPE Alletra Storage MP, Alletra 9000, Primera or 3PAR storage, check out the [tutorial over at HPE Developer](https://developer.hpe.com/blog/9o7zJkqlX5cErkrzgopL/tutorial-how-to-get-started-with-the-hpe-csi-driver-and-hpe-primera-and-). ## Platform Requirements @@ -13,17 +16,18 @@ Check the corresponding CSI driver version in the [compatibility and support](.. Refer to the HPE Single Point of Connectivity Knowledge (SPOCK) for specific platform details (requires an HPE Passport account) of the CSP. The documentation reflected here always corresponds to the latest supported version and may contain references to future features and capabilities. +* [HPE Alletra Storage MP](https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=hw_greenlake_block.html) * [HPE Alletra 9000](https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=hw_alletra.html) * [HPE Primera](https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=hw_primera.html) * [HPE 3PAR](https://h20272.www2.hpe.com/SPOCK/Pages/spock2Html.aspx?htmlFile=hw_3par.html) ### Network Port Requirements -The HPE Alletra 9000, Primera and 3PAR Container Storage Provider requires the following TCP ports to be open inbound to the array from the Kubernetes cluster worker nodes running the HPE CSI Driver for Kubernetes. +The HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR Container Storage Provider requires the following TCP ports to be open inbound to the array from the Kubernetes cluster worker nodes running the HPE CSI Driver for Kubernetes. | Port | Protocol | Description | | ---- | -------- | ----------- | -| 443 | HTTPS | WSAPI (HPE Alletra 9000/Primera) | +| 443 | HTTPS | WSAPI (HPE Alletra Storage MP, Alletra 9000/Primera) | | 8080 | HTTPS | WSAPI (HPE 3PAR) | | 22 | SSH | Array communication | @@ -299,4 +303,4 @@ Use this parameter to specify a subset of Fibre Channel (FC) ports on the array ### Support -Please refer to the HPE Alletra 9000 and Primera and 3PAR Storage CSP [support statement](../../legal/support/index.md#hpe_primera_and_hpe_3par_container_storage_provider_support). +Please refer to the HPE Alletra Storage MP, Alletra 9000 and Primera and 3PAR Storage CSP [support statement](../../legal/support/index.md#hpe_primera_and_hpe_3par_container_storage_provider_support). diff --git a/docs/container_storage_provider/index.md b/docs/container_storage_provider/index.md index ce6eea15..7eeb1be5 100644 --- a/docs/container_storage_provider/index.md +++ b/docs/container_storage_provider/index.md @@ -1,4 +1,4 @@ # Container Storage Providers * [HPE Alletra 5000/6000 and Nimble Storage](hpe_alletra_6000/index.md) -* [HPE Alletra 9000, Primera and 3PAR](hpe_alletra_9000/index.md) +* [HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR](hpe_alletra_storage_mp/index.md) diff --git a/docs/csi_driver/archive.md b/docs/csi_driver/archive.md index 032eaec9..8697877f 100644 --- a/docs/csi_driver/archive.md +++ b/docs/csi_driver/archive.md @@ -4,6 +4,75 @@ HPE supports up to three minor releases. These release are kept here for histori [TOC] +#### HPE CSI Driver for Kubernetes 2.2.0 + +Release highlights: + +* Support for Kubernetes 1.24 and Red Hat OpenShift 4.10 +* Added Tolerations, Affinity, Labels and Node Selectors to Helm chart +* Improved automatic recovery for the NFS Server Provisioner +* Added multipath handling for Alletra 9000, Primera and 3PAR +* Volume expansion of encrypted volumes + +Upgrade considerations: + +* Existing [encrypted volumes needs to be migrated](operations.md#migrate_encrypted_volumes) to allow expansion +* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v220). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Kubernetes1.21-1.241
Helm Chartv2.2.0 on ArtifactHub
Operators + v2.2.1 on OperatorHub
+ v2.2.1 via OpenShift console +
Worker OS + RHEL2 7.x & 8.x, RHCOS 4.8 & 4.10
+ Ubuntu 16.04, 18.04 & 20.04
+ SLES 15 SP2 +
Platforms + Alletra OS 6000 6.0.0.x - 6.1.0.x
+ Alletra OS 9000 9.3.x - 9.5.x
+ Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.0.x
+ Primera OS 4.3.x - 4.5.x
+ 3PAR OS 3.3.x +
Data protocolFibre Channel, iSCSI
Release notesv2.2.0 on GitHub
Blogs + Updates and Improvements to HPE CSI Driver for Kubernetes (release blog) +
+ + + 1 = For HPE Ezmeral Runtime Enterprise, SUSE Rancher, Mirantis Kubernetes Engine and others; Kubernetes clusters must be deployed within the currently supported range of "Worker OS" platforms listed in the above table. See [partner ecosystems](../partners) for other variations.
+ 2 = The HPE CSI Driver will recognize CentOS, AlmaLinux and Rocky Linux as RHEL derives and they are supported by HPE. + + #### HPE CSI Driver for Kubernetes 2.1.1 Release highlights: diff --git a/docs/csi_driver/deployment.md b/docs/csi_driver/deployment.md index 98be59ff..e2ef8327 100644 --- a/docs/csi_driver/deployment.md +++ b/docs/csi_driver/deployment.md @@ -42,14 +42,14 @@ Establish a working directory on a bastion Linux host that has HTTP access to th !!! note Only the HPE CSI Driver 1.4.0 and later is supported using this methodology. -Create a working directory and set environment variables referenced throughout the procedure. In this example, we'll use HPE CSI Driver v2.4.0 on Kubernetes 1.28. Available versions are found in the [co-deployments GitHub repo](https://github.com/hpe-storage/co-deployments/tree/master/yaml/csi-driver). +Create a working directory and set environment variables referenced throughout the procedure. In this example, we'll use HPE CSI Driver v2.4.1 on Kubernetes 1.29. Available versions are found in the [co-deployments GitHub repo](https://github.com/hpe-storage/co-deployments/tree/master/yaml/csi-driver). ```text mkdir hpe-csi-driver cd hpe-csi-driver export MY_REGISTRY=registry.enterprise.example.com -export MY_CSI_DRIVER=2.4.0 -export MY_K8S=1.28 +export MY_CSI_DRIVER=2.4.1 +export MY_K8S=1.29 ``` Next, create a list with the CSI driver images. Copy and paste the entire text blob in one chunk. @@ -59,7 +59,7 @@ curl -s https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v${MY_CSI_DRIVER}/nimble-csp.yaml \ https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v${MY_CSI_DRIVER}/3par-primera-csp.yaml \ | grep image: | awk '{print $2}' | sort | uniq > images -echo quay.io/hpestorage/nfs-provisioner:v3.0.2 >> images +echo quay.io/hpestorage/nfs-provisioner:v3.0.3 >> images ``` !!! important @@ -160,6 +160,20 @@ All parameters are mandatory and described below. Example: +```yaml fct_label="HPE Alletra Storage MP" +apiVersion: v1 +kind: Secret +metadata: + name: hpe-backend + namespace: hpe-storage +stringData: + serviceName: alletrastoragemp-csp-svc + servicePort: "8080" + backend: 10.10.0.20 + username: 3paradm + password: 3pardata +``` + ```yaml fct_label="HPE Alletra 5000/6000" apiVersion: v1 kind: Secret @@ -252,6 +266,20 @@ This `Secret` is used by the CSI sidecars in the `StorageClass` to authenticate To create a new `Secret`, specify the name, `Namespace`, backend username, backend password and the backend IP address to be used by the CSP and save it as `custom-secret.yaml` (a detailed description of the parameters are [available above](#secret_parameters)). +```yaml fct_label="HPE Alletra Storage MP" +apiVersion: v1 +kind: Secret +metadata: + name: custom-secret + namespace: hpe-storage +stringData: + serviceName: alletrastoragemp-csp-svc + servicePort: "8080" + backend: 10.10.0.20 + username: 3paradm + password: 3pardata +``` + ```yaml fct_label="HPE Alletra 5000/6000" apiVersion: v1 kind: Secret @@ -406,19 +434,19 @@ kubectl create ns hpe-storage Worker node IO settings and common `CRDs`: ```text -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-linux-config.yaml -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-volumegroup-snapshotgroup-crds.yaml +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-linux-config.yaml +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-volumegroup-snapshotgroup-crds.yaml ``` Container Storage Provider: ```text fct_label="HPE Alletra 5000/6000 and Nimble Storage" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/nimble-csp.yaml +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/nimble-csp.yaml ``` -```text fct_label="HPE Alletra 9000, Primera and 3PAR" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/3par-primera-csp.yaml -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/3par-primera-crd.yaml +```text fct_label="HPE Alletra Storage MP, HPE Alletra 9000, Primera and 3PAR" +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/3par-primera-csp.yaml +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/3par-primera-crd.yaml ``` !!! important @@ -427,19 +455,19 @@ kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/ma Install the CSI driver: ```text fct_label="Kubernetes 1.28" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.28.yaml +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.29.yaml ``` -```text fct_label="Kubernetes 1.27" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.27.yaml +```text fct_label="Kubernetes 1.28" +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.28.yaml ``` -```text fct_label="Kubernetes 1.26" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.26.yaml +```text fct_label="Kubernetes 1.27" +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.27.yaml ``` -```text fct_label="Kubernetes 1.25" -kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.25.yaml +```text fct_label="Kubernetes 1.26" +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.26.yaml ``` !!! seealso @@ -454,38 +482,38 @@ The following steps outline how to uninstall the CSI driver that has been deploy Uninstall Worker node settings: ```text -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-linux-config.yaml +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-linux-config.yaml ``` Uninstall relevant Container Storage Provider: ```text fct_label="HPE Alletra 5000/6000 and Nimble Storage" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/nimble-csp.yaml +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/nimble-csp.yaml ``` -```text fct_label="HPE Alletra 9000, Primera and 3PAR" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/3par-primera-csp.yaml +```text fct_label="HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR" +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/3par-primera-csp.yaml ``` -!!! error "HPE Alletra 9000, Primera and 3PAR users" - If you are reinstalling the HPE CSI Driver, **DO NOT** remove the `crd/hpevolumeinfos.storage.hpe.com` resource. This `CustomResourceDefinition` contains important volume metadata used by the HPE Alletra 9000, Primera and 3PAR CSP. HPE CSI Driver **v2.0.0 and below** share the same YAML file for `crds` and CSP and would require a manual removal of the individual `Service` and `Deployment` in the "hpe-storage" `Namespace`. +!!! error "HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR users" + If you are reinstalling the HPE CSI Driver, **DO NOT** remove the `crd/hpevolumeinfos.storage.hpe.com` resource. This `CustomResourceDefinition` contains important volume metadata used by the HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR CSP. HPE CSI Driver **v2.0.0 and below** share the same YAML file for `crds` and CSP and would require a manual removal of the individual `Service` and `Deployment` in the "hpe-storage" `Namespace`. Uninstall the CSI driver: +```text fct_label="Kubernetes 1.29" +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.29.yaml +``` + ```text fct_label="Kubernetes 1.28" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.28.yaml +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.28.yaml ``` ```text fct_label="Kubernetes 1.27" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.27.yaml +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.27.yaml ``` ```text fct_label="Kubernetes 1.26" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.26.yaml -``` - -```text fct_label="Kubernetes 1.25" -kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.25.yaml +kubectl delete -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.1/hpe-csi-k8s-1.26.yaml ``` If no longer needed, delete the "hpe-storage" `Namespace`. diff --git a/docs/csi_driver/diagnostics.md b/docs/csi_driver/diagnostics.md index 26698d44..44fd7d32 100644 --- a/docs/csi_driver/diagnostics.md +++ b/docs/csi_driver/diagnostics.md @@ -15,7 +15,7 @@ hpe-storage hpe-csi-node-pwq2d 2/2 Running 0 hpe-storage nimble-csp-546c9c4dd4-5lsdt 1/1 Running 0 15s ``` -```text fct_label="HPE Alletra 9000, Primera and 3PAR" +```text fct_label="HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR" kubectl get pods --all-namespaces -l 'app in (primera3par-csp, hpe-csi-node, hpe-csi-controller)' NAMESPACE NAME READY STATUS RESTARTS AGE hpe-storage hpe-csi-controller-7d9cd6b855-fqppd 9/9 Running 0 14s @@ -88,12 +88,39 @@ The NFS Server Provisioner consists of a number of Kubernetes resources per PVC. | Object | Name                           | Purpose | | --------------------- | -------------------- | ------------- | | ConfigMap | hpe-nfs-config | This `ConfigMap` holds the configuration file for the NFS server. Local tweaks may be wanted. Please see the [config file reference](https://github.com/nfs-ganesha/nfs-ganesha/tree/master/src/config_samples) for more details. | -| Deployment | hpe-nfs-<UUID> | The `Deployment` that is running the NFS `Pod`. | -| Service | hpe-nfs-<UUID> | `Pod` `Service` the NFS clients perform mounts against. | -| PVC | hpe-nfs-<UUID> | The RWO claim serving the NFS workload. | +| Deployment | hpe-nfs-UID | The `Deployment` that is running the NFS `Pod`. | +| Service | hpe-nfs-UID | The `Service` the NFS clients perform mounts against. | +| PVC | hpe-nfs-UID | The RWO claim serving the NFS workload. | !!! tip - The `` stems from the user request RWX claim UUID for easy tracking. + The UID stems from the user request RWX `PVC` for easy tracking. Use `kubectl get pvc/my-pvc -o jsonpath='{.metadata.uid}{"\n"}'` to retrieve it. + +### Tracing NFS resources + +When troubleshooting NFS deployments it's common that only the source RWX `PVC` and `Namespace` is known. The next few steps explains how resources can be easily traced. + +Retrieve the "hpe-nfs-UID" from the NFS `Pod` by specifying `PVC` and `Namespace` of the RWX `PVC`: +```text +kubectl get pods -l provisioned-by=my-pvc,provisioned-from=my-namespace -A -o jsonpath='{.items[].metadata.labels.app}{"\n"}' +``` + +Next, enumerate the resources from the "hpe-nfs-UID": +```text +kubectl get pvc,svc,deploy -A -o name --field-selector metadata.name=hpe-nfs-UID +``` + +Example output: + +```text +persistentvolumeclaim/hpe-nfs-UID +service/hpe-nfs-UID +deployment.apps/hpe-nfs-UID +``` + +If only the `PV` name is known, looking from the backend storage perspective, the `PV` name (and `.spec.claimRef.uid`) contains the UID, for example: "pvc-UID". + +!!! seealso "Clarification" + The `hpe-nfs-UID` is abbreviated, it will contain a real UID added on, for example "hpe-nfs-98ce7c80-13f9-45d0-9609-089227bf97f1". ## Volume and Snapshot Groups @@ -143,7 +170,7 @@ CSP logs can be accessed from their respective services. kubectl logs -f deploy/nimble-csp -n hpe-storage ``` -```text fct_label="HPE Alletra 9000, Primera and 3PAR" +```text fct_label="HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR" kubectl logs -f deploy/primera3par-csp -n hpe-storage ``` diff --git a/docs/csi_driver/examples/operations/patch-nfs-server-2.4.1.yaml b/docs/csi_driver/examples/operations/patch-nfs-server-2.4.1.yaml new file mode 100644 index 00000000..41e0fab2 --- /dev/null +++ b/docs/csi_driver/examples/operations/patch-nfs-server-2.4.1.yaml @@ -0,0 +1,18 @@ +spec: + template: + spec: + containers: + - name: hpe-nfs + image: quay.io/hpestorage/nfs-provisioner:v3.0.3 + tolerations: + - effect: NoSchedule + key: csi.hpe.com/hpe-nfs + operator: Exists + - effect: NoExecute + key: node.kubernetes.io/not-ready + operator: Exists + tolerationSeconds: 30 + - effect: NoExecute + key: node.kubernetes.io/unreachable + operator: Exists + tolerationSeconds: 30 diff --git a/docs/csi_driver/img/csi_driver_architecture-2.4.1.png b/docs/csi_driver/img/csi_driver_architecture-2.4.1.png new file mode 100644 index 00000000..2b5dfed5 Binary files /dev/null and b/docs/csi_driver/img/csi_driver_architecture-2.4.1.png differ diff --git a/docs/csi_driver/index.md b/docs/csi_driver/index.md index e3b1d640..c9fb3ed3 100644 --- a/docs/csi_driver/index.md +++ b/docs/csi_driver/index.md @@ -4,7 +4,7 @@ A Container Storage Interface ([CSI](https://github.com/container-storage-interf The CSI driver architecture allows a complete separation of concerns between upstream Kubernetes core, SIG Storage (CSI owners), CSI driver author (HPE) and the backend CSP developer. -![HPE CSI Driver Architecture](img/csi_driver_architecture-2.3.0.png) +![HPE CSI Driver Architecture](img/csi_driver_architecture-2.4.1.png) !!! tip The HPE CSI Driver for Kubernetes is vendor agnostic. Any entity may leverage the driver and provide their own Container Storage Provider. @@ -36,12 +36,13 @@ Below is the official table for CSI features we track and deem readily available | Volume Encryption1 | N/A | 1.18 | 2.0.0 | | Topology | Stable | 1.17 | Future | | Storage Capacity Tracking | Stable | 1.24 | Future | -| Volume Health | Alpha | 1.21 | Future | +| Volume Expansion From Source | Stable | 1.27 | Future | +| ReadWriteOncePod | Stable | 1.29 | Future | | Volume Populator | Beta | 1.24 | Future | -| ReadWriteOncePod | Alpha | 1.22 | Future | +| Volume Health | Alpha | 1.21 | Future | | Cross Namespace Snapshots | Alpha | 1.26 | Future | | Upstream Volume Group Snapshot | Alpha | 1.27 | Future | -| Volume Expansion From Source | Stable | 1.27 | Future | +| Volume Attribute Classes | Alpha | 1.29 | Future | 1 = HPE CSI Driver for Kubernetes specific CSI sidecar. CSP support may vary.
@@ -62,49 +63,51 @@ These are the combinations HPE has tested and can provide official support servi !!! note For Kubernetes 1.12 and earlier please see [legacy FlexVolume drivers](../flexvolume_driver/index.md), do note that the FlexVolume drivers are being deprecated. -#### HPE CSI Driver for Kubernetes 2.4.0 + +#### HPE CSI Driver for Kubernetes 2.4.1 Release highlights: -* Kubernetes 1.27 and 1.28 support -* KubeVirt and OpenShift Virtualization support for Nimble/Alletra 5000/6000 -* Enhanced scheduling for the NFS Server Provisioner -* Multiarch images (Linux ARM64/AMD64) for the CSI driver components and Alletra 9000 CSP -* Major updates to SIG Storage images +* HPE Alletra Storage MP support +* Kubernetes 1.29 support +* Full KubeVirt, OpenShift Virtualization and SUSE Harvester support for HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR +* Full ARM64 support for HPE Alletra 5000/6000 and Nimble Storage +* Support for foreign `StorageClasses` with the NFS Server Provisioner +* SUSE Linux Enterprise Micro OS (SLE Micro) support Upgrade considerations: -* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v240). +* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v241). - + - + - +
Kubernetes1.25-1.2811.26-1.291
Helm Chartv2.4.0 on ArtifactHubv2.4.1 on ArtifactHub
Operators - v2.4.0 on OperatorHub
- v2.4.0 via OpenShift console + v2.4.1 on OperatorHub
Worker OS - RHEL2 7.x, 8.x, 9.x, RHCOS 4.12-4.14
+ Red Hat Enterprise Linux2 7.x, 8.x, 9.x, Red Hat CoreOS 4.12-4.14
Ubuntu 16.04, 18.04, 20.04, 22.04
- SLES 15 SP3, SP4, SP5 + SUSE Linux Enterprise Server 15 SP3, SP4, SP5 and SLE Micro4 equivalents
Platforms3 - + Alletra Storage MP5 10.2.x - 10.3.x
Alletra OS 9000 9.3.x - 9.5.x
- Alletra OS 5000/6000 6.0.0.x - 6.1.1.x
+ Alletra OS 5000/6000 6.0.0.x - 6.1.2.x
Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.2.x
Primera OS 4.3.x - 4.5.x
3PAR OS 3.3.x @@ -120,12 +123,12 @@ Upgrade considerations:
Release notesv2.4.0 on GitHubv2.4.1 on GitHub
Blogs - Introduction to new workload paradigms with HPE CSI Driver for Kubernetes + Introducing HPE Alletra Storage MP to HPE CSI Driver for Kubernetes
@@ -135,120 +138,127 @@ Upgrade considerations: 1 = For HPE Ezmeral Runtime Enterprise, SUSE Rancher, Mirantis Kubernetes Engine and others; Kubernetes clusters must be deployed within the currently supported range of "Worker OS" platforms listed in the above table. See [partner ecosystems](../partners) for other variations. Lowest tested and known working version is Kubernetes 1.21.
2 = The HPE CSI Driver will recognize CentOS, AlmaLinux and Rocky Linux as RHEL derives and they are supported by HPE.
3 = Learn about each data platform's team [support commitment](../legal/support/index.md#container_storage_providers).
+ 4 = SLE Micro nodes may need to be conformed manually, run `transactional-update -n pkg install multipath-tools open-iscsi nfs-client sg3_utils` and reboot if the CSI node driver doesn't start.
+ 5 = The HPE CSI Driver for Kubernetes only support HPE Alletra Storage MP when used with HPE GreenLake for Block Storage. Please see the [VAST CSI Driver](https://support.vastdata.com/s/topic/0TOV40000000TtFOAU/vast-csi-driver) for HPE GreenLake for File Storage.
 -#### HPE CSI Driver for Kubernetes 2.3.0 +#### HPE CSI Driver for Kubernetes 2.4.0 Release highlights: -* Introducing HPE Alletra 5000 -* Security updates -* Support for Kubernetes 1.25-1.26 and Red Hat OpenShift 4.11-4.12 -* Support for SLES 15 SP4, RHEL 9.1 and Ubuntu 22.04 +* Kubernetes 1.27 and 1.28 support +* KubeVirt and OpenShift Virtualization support for Nimble/Alletra 5000/6000 +* Enhanced scheduling for the NFS Server Provisioner +* Multiarch images (Linux ARM64/AMD64) for the CSI driver components and Alletra 9000 CSP +* Major updates to SIG Storage images Upgrade considerations: -* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v230). +* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v240). - + - + - + + + + + - +
Kubernetes1.23-1.2611.25-1.281
Helm Chartv2.3.0 on ArtifactHubv2.4.0 on ArtifactHub
Operators - v2.3.0 on OperatorHub
- v2.3.0 via OpenShift console + v2.4.0 on OperatorHub
+ v2.4.0 via OpenShift console
Worker OS - RHEL2 7.x, 8.x, 9.x, RHCOS 4.10-4.12
+ RHEL2 7.x, 8.x, 9.x, RHCOS 4.12-4.14
Ubuntu 16.04, 18.04, 20.04, 22.04
- SLES 15 SP2, SP3, SP4 + SLES 15 SP3, SP4, SP5
Platforms3 - Alletra OS 5000/6000 6.0.0.x - 6.1.1.x
+ Alletra OS 9000 9.3.x - 9.5.x
- Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.1.x
+ Alletra OS 5000/6000 6.0.0.x - 6.1.1.x
+ Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.2.x
Primera OS 4.3.x - 4.5.x
3PAR OS 3.3.x
Data protocolData protocols Fibre Channel, iSCSI
FilesystemsXFS, ext3/ext4, btrfs, NFSv4*
Release notesv2.3.0 on GitHubv2.4.0 on GitHub
Blogs - Support and security updates for HPE CSI Driver for Kubernetes (release blog) + Introduction to new workload paradigms with HPE CSI Driver for Kubernetes
+ * = The HPE CSI Driver for Kubernetes is a block storage driver primarily. It includes an [NFS Server Provisioner](using.md#using_the_nfs_server_provisioner) that allows "ReadWriteMany" `PersistentVolumeClaims`.
1 = For HPE Ezmeral Runtime Enterprise, SUSE Rancher, Mirantis Kubernetes Engine and others; Kubernetes clusters must be deployed within the currently supported range of "Worker OS" platforms listed in the above table. See [partner ecosystems](../partners) for other variations. Lowest tested and known working version is Kubernetes 1.21.
- 2 = The HPE CSI Driver will recognize CentOS, AlmaLinux and Rocky Linux as RHEL derives and they are supported by HPE. - 3 = Learn about each data platform's team [support commitment](../legal/support/index.md#container_storage_providers). + 2 = The HPE CSI Driver will recognize CentOS, AlmaLinux and Rocky Linux as RHEL derives and they are supported by HPE.
+ 3 = Learn about each data platform's team [support commitment](../legal/support/index.md#container_storage_providers).
 -#### HPE CSI Driver for Kubernetes 2.2.0 +#### HPE CSI Driver for Kubernetes 2.3.0 Release highlights: -* Support for Kubernetes 1.24 and Red Hat OpenShift 4.10 -* Added Tolerations, Affinity, Labels and Node Selectors to Helm chart -* Improved automatic recovery for the NFS Server Provisioner -* Added multipath handling for Alletra 9000, Primera and 3PAR -* Volume expansion of encrypted volumes +* Introducing HPE Alletra 5000 +* Security updates +* Support for Kubernetes 1.25-1.26 and Red Hat OpenShift 4.11-4.12 +* Support for SLES 15 SP4, RHEL 9.1 and Ubuntu 22.04 Upgrade considerations: -* Existing [encrypted volumes needs to be migrated](operations.md#migrate_encrypted_volumes) to allow expansion -* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v220). +* Existing claims provisioned with the NFS Server Provisioner [needs to be upgraded](operations.md#upgrade_to_v230). - + - + - + @@ -259,19 +269,20 @@ Upgrade considerations: - +
Kubernetes1.21-1.2411.23-1.261
Helm Chartv2.2.0 on ArtifactHubv2.3.0 on ArtifactHub
Operators - v2.2.1 on OperatorHub
- v2.2.1 via OpenShift console + v2.3.0 on OperatorHub
+ v2.3.0 via OpenShift console
Worker OS - RHEL2 7.x & 8.x, RHCOS 4.8 & 4.10
- Ubuntu 16.04, 18.04 & 20.04
- SLES 15 SP2 + RHEL2 7.x, 8.x, 9.x, RHCOS 4.10-4.12
+ Ubuntu 16.04, 18.04, 20.04, 22.04
+ SLES 15 SP2, SP3, SP4
PlatformsPlatforms3 - Alletra OS 6000 6.0.0.x - 6.1.0.x
+ Alletra OS 5000/6000 6.0.0.x - 6.1.1.x
Alletra OS 9000 9.3.x - 9.5.x
- Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.0.x
+ Nimble OS 5.0.10.x, 5.2.1.x, 6.0.0.x, 6.1.1.x
Primera OS 4.3.x - 4.5.x
3PAR OS 3.3.x
Release notesv2.2.0 on GitHubv2.3.0 on GitHub
Blogs - Updates and Improvements to HPE CSI Driver for Kubernetes (release blog) + Support and security updates for HPE CSI Driver for Kubernetes (release blog)
- 1 = For HPE Ezmeral Runtime Enterprise, SUSE Rancher, Mirantis Kubernetes Engine and others; Kubernetes clusters must be deployed within the currently supported range of "Worker OS" platforms listed in the above table. See [partner ecosystems](../partners) for other variations.
+ 1 = For HPE Ezmeral Runtime Enterprise, SUSE Rancher, Mirantis Kubernetes Engine and others; Kubernetes clusters must be deployed within the currently supported range of "Worker OS" platforms listed in the above table. See [partner ecosystems](../partners) for other variations. Lowest tested and known working version is Kubernetes 1.21.
2 = The HPE CSI Driver will recognize CentOS, AlmaLinux and Rocky Linux as RHEL derives and they are supported by HPE. + 3 = Learn about each data platform's team [support commitment](../legal/support/index.md#container_storage_providers). #### Release Archive @@ -288,8 +299,6 @@ HPE currently supports up to three minor releases of the HPE CSI Driver for Kube * The CSI driver support a fixed number of volumes per node. Inspect the current limitation by running `kubectl get csinodes -o yaml` and inspect `.spec.drivers.allocatable` for "csi.hpe.com". The "count" element contains how many volumes the node can attach from the HPE CSI Driver (default is 100). * The HPE CSI Driver uses host networking for the node driver. Some CNIs have flaky implementations which prevents the CSI driver components to communicate properly. Especially notorious is Flannel on K3s. Use Calico if possible for the widest compatibility. * The [NFS Server Provisioner](using.md#limitations_and_considerations_for_the_nfs_server_provisioner) and each of the [CSPs](../container_storage_provider/index.md) have known limitations listed separately. -* There's currently no ARM image available for the HPE Alletra 5000/6000 and Nimble Storage CSP. Use the Helm chart "csp.nodeSelector" parameters to schedule the CSP `Pods` on an x86 node. -* Live migration with KubeVirt using "volumeMode: Block" is only supported by the HPE Alletra 5000/6000 and Nimble Storage CSP. ## iSCSI CHAP Considerations diff --git a/docs/csi_driver/install_legacy.md b/docs/csi_driver/install_legacy.md index 9e16a4b4..923620c6 100644 --- a/docs/csi_driver/install_legacy.md +++ b/docs/csi_driver/install_legacy.md @@ -2,6 +2,18 @@ Older versions of the HPE CSI Driver for Kubernetes are kept here for reference. Check the CSI driver GitHub repo for the appropriate YAML files to declare on the cluster for the respective version of Kubernetes. +!!! important + The resources for CSPs, CRDs and ConfigMaps are available in each respective CSI driver version directory [here](https://github.com/hpe-storage/co-deployments/tree/master/yaml/csi-driver). Use the below version mappings as reference. + +# Kubernetes 1.25 + +```text +kubectl apply -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/csi-driver/v2.4.0/hpe-csi-k8s-1.25.yaml +``` + +!!! note + Latest supported CSI driver version is 2.4.0 for Kubernetes 1.25. + # Kubernetes 1.24 ```text diff --git a/docs/csi_driver/monitor.md b/docs/csi_driver/monitor.md index 9cdaff30..3ad58c57 100644 --- a/docs/csi_driver/monitor.md +++ b/docs/csi_driver/monitor.md @@ -62,6 +62,11 @@ spec: claimName: my-pvc ``` +!!! danger + It's imperative that failure scenarios that are being mitigated for the application are properly tested before put into production. It's up to the CSP to fence the `PersistentVolume` attached to an isolated node when a new "NodePublish" request comes in. Node isolation is the most dangerous scenario as the workload continues to run on the node when disconnected from the outside world. Simply shutdown the kubelet to test this scenario and ensure the block device become inaccessible to the isolated node. + ## Limitations * Kubernetes provide automatic recovery for your applications, not high availability. Expect applications to take minutes (up to 8 minutes with the default tolerations for `node.kubernetes.io/not-ready` and `node.kubernetes.io/unreachable`) to fully recover during a node failure or network partition using the Pod Monitor for `Pods` with `PersistentVolumeClaims`. +* From Kubernetes 1.24 and onwards the Pod Monitor is ineffective on `StatefulSets` due to [non-graceful node shutdown](https://kubernetes.io/docs/concepts/architecture/nodes/#non-graceful-node-shutdown). +* Using the Pod Monitor on a workload controller besides `Deployment` is unsupported and may have undesired side effects. diff --git a/docs/csi_driver/operations.md b/docs/csi_driver/operations.md index 66d5e8d0..75f32442 100644 --- a/docs/csi_driver/operations.md +++ b/docs/csi_driver/operations.md @@ -284,16 +284,16 @@ Enabling and setting up the CSI snapshotter and related `CRDs` is not necessary In the event the CSI driver contains updates to the NFS Server Provisioner, any running NFS server needs to be updated manually. -### Upgrade to v2.4.0 +### Upgrade to v2.4.1 -Any prior deployed NFS servers may be upgraded to v2.4.0. +Any prior deployed NFS servers may be upgraded to v2.4.1. !!! important With v2.4.0 and onwards the NFS servers are deployed with default resource requests. Those won't be applied on running NFS servers, only new ones. #### Assumptions -- HPE CSI Driver or Operator v2.4.0 installed. +- HPE CSI Driver or Operator v2.4.1 installed. - All running NFS servers are running in the "hpe-nfs" `Namespace`. - Worker nodes with access to the Quay registry and SCOD. - Access to the commands `kubectl`, `yq` and `curl`. @@ -310,7 +310,7 @@ When patching the NFS `Deployments`, the `Pods` will restart and cause a pause i Patch all NFS `Deployments` with the following. ```text -curl -s {{ config.site_url}}csi_driver/examples/operations/patch-nfs-server-2.4.0.yaml | \ +curl -s {{ config.site_url}}csi_driver/examples/operations/patch-nfs-server-2.4.1.yaml | \ kubectl patch -n hpe-nfs \ $(kubectl get deploy -n hpe-nfs -o name) \ --patch-file=/dev/stdin @@ -321,7 +321,7 @@ curl -s {{ config.site_url}}csi_driver/examples/operations/patch-nfs-server-2.4. ### Validation -This command will list all "hpe-nfs" `Deployments` across the entire cluster. Each `Deployment` should be using v3.0.2 of the "nfs-provisioner" image after the uprade is complete. +This command will list all "hpe-nfs" `Deployments` across the entire cluster. Each `Deployment` should be using v3.0.3 of the "nfs-provisioner" image after the uprade is complete. ```text kubectl get deploy -A -o yaml | \ @@ -436,7 +436,7 @@ systemctl stop multipathd Ensure `multipathd` is running and enabled: ```text -systemctl enable --now `multipathd` +systemctl enable --now multipathd ``` ### Important Considerations diff --git a/docs/csi_driver/standalone_nfs.md b/docs/csi_driver/standalone_nfs.md index 459195c7..f1b58df7 100644 --- a/docs/csi_driver/standalone_nfs.md +++ b/docs/csi_driver/standalone_nfs.md @@ -2,6 +2,9 @@ In certain situations is desirable to run the NFS Server Provisioner image without the dual `PersistentVolumeClaim` (PVC) semantic in a more static fashion on top of a `PVC` provisioned by a non-HPE CSI Driver `StorageClass`. +!!! caution "Notice" + Since HPE CSI Driver for Kubernetes v2.4.1, this functionality is built into the CSI driver. See [Using a Foreign StorageClass](using.md#using_a_foreign_storageclass) how to use it. + [TOC] ## Limitations diff --git a/docs/csi_driver/using.md b/docs/csi_driver/using.md index 07150239..a0a5f255 100644 --- a/docs/csi_driver/using.md +++ b/docs/csi_driver/using.md @@ -50,29 +50,29 @@ kubectl get sts,deploy -A If no prior CRDs or controllers exist, install the snapshot CRDs and common snapshot controller (once per Kubernetes cluster, independent of any CSI drivers). -```text fct_label="HPE CSI Driver v2.4.0" -# Kubernetes 1.25-1.28 +```text fct_label="HPE CSI Driver v2.4.1" +# Kubernetes 1.26-1.29 git clone https://github.com/kubernetes-csi/external-snapshotter cd external-snapshotter -git checkout tags/v6.2.2 -b hpe-csi-driver-v2.4.0 +git checkout tags/v6.3.3 -b hpe-csi-driver-v2.4.1 kubectl kustomize client/config/crd | kubectl create -f- kubectl -n kube-system kustomize deploy/kubernetes/snapshot-controller | kubectl create -f- ``` -```text fct_label="HPE CSI Driver v2.3.0" -# Kubernetes 1.23-1.26 +```text fct_label="HPE CSI Driver v2.4.0" +# Kubernetes 1.25-1.28 git clone https://github.com/kubernetes-csi/external-snapshotter cd external-snapshotter -git checkout tags/v5.0.1 -b hpe-csi-driver-v2.3.0 +git checkout tags/v6.2.2 -b hpe-csi-driver-v2.4.0 kubectl kustomize client/config/crd | kubectl create -f- kubectl -n kube-system kustomize deploy/kubernetes/snapshot-controller | kubectl create -f- ``` -```text fct_label="HPE CSI Driver v2.2.0" -# Kubernetes 1.21-1.24 +```text fct_label="HPE CSI Driver v2.3.0" +# Kubernetes 1.23-1.26 git clone https://github.com/kubernetes-csi/external-snapshotter cd external-snapshotter -git checkout tags/v5.0.1 -b hpe-csi-driver-v2.2.0 +git checkout tags/v5.0.1 -b hpe-csi-driver-v2.3.0 kubectl kustomize client/config/crd | kubectl create -f- kubectl -n kube-system kustomize deploy/kubernetes/snapshot-controller | kubectl create -f- ``` @@ -155,6 +155,7 @@ Common HPE CSI Driver `StorageClass` parameters across CSPs. | fsMode | Octal digits | 1 to 4 octal digits that represent the file mode to be applied to the root directory of the filesystem. | | fsCreateOptions | Text | A string to be passed to the mkfs command. These flags are opaque to CSI and are therefore not validated. To protect the node, only the following characters are allowed: ```[a-zA-Z0-9=, \-]```. | | nfsResources | Boolean | When set to "true", requests against the `StorageClass` will create resources for the NFS Server Provisioner (`Deployment`, RWO `PVC` and `Service`). Required parameter for ReadWriteMany and ReadOnlyMany accessModes. Default: "false" | +| nfsForeignStorageClass | Text | Provision NFS servers on `PVCs` from a different `StorageClass`. See [Using a Foreign StorageClass](#using_a_foreign_storageclass) | | nfsNamespace | Text | Resources are by default created in the "hpe-nfs" `Namespace`. If CSI `VolumeSnapshotClass` and `dataSource` functionality is required on the requesting claim, requesting and backing PVC need to exist in the requesting `Namespace`. A value of "csi.storage.k8s.io/pvc/namespace" will provision resources in the requesting `PVC` `Namespace`. | | nfsMountOptions | Text | Customize NFS mount options for the `Pods` to the server `Deployment`. Uses `mount` command defaults from the node. | | nfsProvisionerImage | Text | Customize provisioner image for the server `Deployment`. Default: Official build from "hpestorage/nfs-provisioner" repo | @@ -387,14 +388,8 @@ spec: volumeMode: Block ``` - - -!!! caution - The CSI driver will allow `volumeMode: Block` `PVCs` with `ReadWriteMany` and `ReadOnlyMany` `accessModes` to be provisioned without any exceptions, but it's currently only supported with the HPE Alletra 6000 CSP. It's expected that the application handles read/write IO, volume locking and access in the event of concurrent block access from multiple nodes. Mapping the device in a `Pod` specification is slightly different than using regular filesystems as a `volumeDevices` section is added instead of a `volumeMounts` stanza: @@ -746,6 +741,35 @@ By default, the NFS Server Provisioner deploy resources in the "hpe-nfs" `Namesp !!! tip A comprehensive [tutorial is available](https://developer.hpe.com/blog/xABwJY56qEfNGMEo1lDj/introducing-a-nfs-server-provisioner-and-pod-monitor-for-the-hpe-csi-dri) on HPE Developer on how to get started with the NFS Server Provisioner and the HPE CSI Driver for Kubernetes. There's also a brief tutorial available in the [Video Gallery](../learn/video_gallery/index.md#multi-writer_workloads_using_the_nfs_server_provisioner). +Example `StorageClass` with "nfsResources" enabled. No CSP specific parameters for clarity. + +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: hpe-standard-file +provisioner: csi.hpe.com +parameters: + csi.storage.k8s.io/controller-expand-secret-name: hpe-backend + csi.storage.k8s.io/controller-expand-secret-namespace: hpe-storage + csi.storage.k8s.io/controller-publish-secret-name: hpe-backend + csi.storage.k8s.io/controller-publish-secret-namespace: hpe-storage + csi.storage.k8s.io/node-publish-secret-name: hpe-backend + csi.storage.k8s.io/node-publish-secret-namespace: hpe-storage + csi.storage.k8s.io/node-stage-secret-name: hpe-backend + csi.storage.k8s.io/node-stage-secret-namespace: hpe-storage + csi.storage.k8s.io/provisioner-secret-name: hpe-backend + csi.storage.k8s.io/provisioner-secret-namespace: hpe-storage + description: "NFS backend volume created by the HPE CSI Driver for Kubernetes" + csi.storage.k8s.io/fstype: ext4 + nfsResources: "true" +reclaimPolicy: Delete +allowVolumeExpansion: true +``` + +!!! note + Using XFS may result in stale NFS handles during node failures and outages. Always use ext4 for NFS `PVCs`. While "allowVolumeExpansion" isn't supported on the NFS `PVC`, the backend "RWO" `PVC` does. + Example use of `accessModes`: ```yaml fct_label="ReadWriteOnce" @@ -817,6 +841,32 @@ spec: Requesting an empty read-only volume might not seem practical. The primary use case is to source existing datasets into immutable applications, using either a backend CSP cloning capability or CSI data management feature such as [snapshots or existing PVCs](#using_csi_snapshots). +#### Using a Foreign StorageClass + +Since HPE CSI Driver for Kubernetes version 2.4.1 it's possible to provision NFS servers on top of non-HPE CSI Driver `StorageClasses`. The most prominent use case for this functionality is to coexist with the vSphere CSI Driver (VMware vSphere Container Storage Plug-in) in FC environments and provide "RWX" `PVCs`. + +##### Example StorageClass using a foreign StorageClass + +The HPE CSI Driver only manages the NFS server `Deployment`, `Service` and `PVC`. There must be an existing `StorageClass` capable of provisioning "RWO" filesystem `PVCs`. + +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: hpe-nfs-servers +provisioner: csi.hpe.com +parameters: + nfsResources: "true" + nfsForeignStorageClass: "my-foreign-storageclass-name" +reclaimPolicy: Delete +allowVolumeExpansion: false +``` + +Next, provision "RWO" or "RWX" claims from the "hpe-nfs-servers" `StorageClass`. An NFS server will be provisioned on a "RWO" `PVC` from the `StorageClass` "my-foreign-storageclass-name". + +!!! note + Only `StorageClasses` that uses HPE storage proxied by partner CSI drivers are supported by HPE. + #### Limitations and Considerations for the NFS Server Provisioner These are some common issues and gotchas that are useful to know about when planning to use the NFS Server Provisioner. @@ -835,6 +885,7 @@ These are some common issues and gotchas that are useful to know about when plan - A single NFS server instance is capable of 100GigE wirespeed with large sequential workloads and up to 200,000 IOPS with small IO using bare-metal nodes and multiple clients. - Using ext4 as the backing filesystem has shown better performance with simultaneous writers to the same file. - Additional configuration and considerations may be required when using the NFS Server Provisioner with Red Hat OpenShift. See [NFS Server Provisioner Considerations](../partners/redhat_openshift/index.md#nfs_server_provisioner_considerations) for OpenShift. +- XFS has proven troublesome to use as a backend "RWO" volume filesystem, leaving stale NFS handles for clients. Use ext4 as the "csi.storage.k8s.io/fstype" `StorageClass` parameter for best results. See [diagnosing NFS Server Provisioner issues](diagnostics.md#nfs_server_provisioner_resources) for further details. diff --git a/docs/img/csi-driver-overview.png b/docs/img/csi-driver-overview.png index de3c10fa..664715aa 100644 Binary files a/docs/img/csi-driver-overview.png and b/docs/img/csi-driver-overview.png differ diff --git a/docs/index.md b/docs/index.md index e893c830..02344c78 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,7 +9,7 @@ --> ## HPE Storage Container Orchestrator Documentation -This is an umbrella documentation project for the HPE CSI Driver for Kubernetes and neighboring ecosystems for HPE primary storage including HPE Alletra 5000/6000, Alletra 9000, Nimble Storage, Primera and 3PAR storage systems. The documentation is tailored for IT Ops, developers and partners. +This is an umbrella documentation project for the HPE CSI Driver for Kubernetes and neighboring ecosystems for HPE primary storage including HPE Alletra Storage MP, Alletra 9000, Alletra 5000/6000, Nimble Storage, Primera and 3PAR storage systems. The documentation is tailored for IT Ops, developers and technology partners. ![HPE CSI Driver for Kubernetes](img/csi-driver-overview.png) diff --git a/docs/learn/video_gallery/index.md b/docs/learn/video_gallery/index.md index ae434502..879624bc 100644 --- a/docs/learn/video_gallery/index.md +++ b/docs/learn/video_gallery/index.md @@ -91,13 +91,13 @@ Explore how to take advantage of the HPE CSI Driver's exclusive features `Volume [Watch on YouTube](https://www.youtube.com/watch?v=zUj-bJ_KqHU) -### Multi-Writer Workloads using the NFS Server Provisioner + ### Adapt stateful workloads dynamically with the HPE CSI Driver for Kubernetes diff --git a/docs/legal/support/index.md b/docs/legal/support/index.md index ec3079aa..4c76f71b 100644 --- a/docs/legal/support/index.md +++ b/docs/legal/support/index.md @@ -24,12 +24,12 @@ Example cases where support may not be possible include: * = In the event where other vendors need to be consulted, the HPE Nimble Support team will not disengage from the Support Action. HPE Nimble Support will continue to partner with the customer and other vendors to search for the correct answers to the issue. -### HPE Alletra 9000 and Primera and 3PAR Storage Container Storage Provider support +### HPE Alletra Storage MP, Alletra 9000 and Primera and 3PAR Container Storage Provider support -Limited to the HPE Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) only. Best effort support is available for the CSP for HPE Alletra 9000, Primera and 3PAR Storage with All-inclusive Single or Multi-System software and an active HPE Pointnext support agreement. Since HPE Pointnext support for the CSP is best effort only, any other support levels like Warranty, Foundation Care, Proactive Care, Proactive Care Advanced and Datacenter Care or other support levels do not apply. Best effort response times are based on local standard business days and working hours. If your location is outside the customary service zone, response time may be longer. +Limited to the HPE Alletra Storage MP, Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) only. Best effort support is available for the CSP for HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR Storage with All-inclusive Single or Multi-System software and an active HPE Pointnext support agreement. Since HPE Pointnext support for the CSP is best effort only, any other support levels like Warranty, Foundation Care, Proactive Care, Proactive Care Advanced and Datacenter Care or other support levels do not apply. Best effort response times are based on local standard business days and working hours. If your location is outside the customary service zone, response time may be longer. -| HPE Alletra 9000, Primera and 3PAR Hardware Contract Type | Phone Number | -| --------------------------------------------------------- | ------------ | -| **Warranty and Foundation Care** | 800-633-3600 | -| **Proactive Care (PC)** | 866-211-5211 | -| **Datacenter Care (DC)** | 888-751-2149 | +| HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR Hardware Contract Type | Phone Number | +| ----------------------------------------------------------------------------- | ------------ | +| **Warranty and Foundation Care** | 800-633-3600 | +| **Proactive Care (PC)** | 866-211-5211 | +| **Datacenter Care (DC)** | 888-751-2149 | diff --git a/docs/partners/redhat_openshift/index.md b/docs/partners/redhat_openshift/index.md index 0990d51f..cb7b971e 100644 --- a/docs/partners/redhat_openshift/index.md +++ b/docs/partners/redhat_openshift/index.md @@ -16,10 +16,11 @@ Software delivered through the HPE and Red Hat partnership follows a [rigorous c | Status | Red Hat OpenShift | HPE CSI Operator | Container Storage Providers | | ----------------------- | --------------------------------- | -------------------------- | ------------------------------------------------ | -| Certified | 4.14 EUS2 | 2.4.0 | [All](../../container_storage_provider/index.md) | -| Certified | 4.13 | 2.4.0 | [All](../../container_storage_provider/index.md) | -| Certified | 4.12 EUS2 | 2.3.0, 2.4.0 | [All](../../container_storage_provider/index.md) | -| Certified | 4.11 | 2.3.0 | [All](../../container_storage_provider/index.md) | +| Certified | 4.15 | 2.4.1 | [All](../../container_storage_provider/index.md) | +| Certified | 4.14 EUS2 | 2.4.0, 2.4.1 | [All](../../container_storage_provider/index.md) | +| Certified | 4.13 | 2.4.0, 2.4.1 | [All](../../container_storage_provider/index.md) | +| Certified | 4.12 EUS2 | 2.3.0, 2.4.0, 2.4.1 | [All](../../container_storage_provider/index.md) | +| EOL1 | 4.11 | 2.3.0 | [All](../../container_storage_provider/index.md) | | EOL1 | 4.10 EUS2 | 2.2.1, 2.3.0 | [All](../../container_storage_provider/index.md) | 1 = End of life support per [Red Hat OpenShift Life Cycle Policy](https://access.redhat.com/support/policy/updates/openshift).
@@ -28,13 +29,11 @@ Software delivered through the HPE and Red Hat partnership follows a [rigorous c Check the table above periodically for future releases. -!!! warning "Important" - Due to an [unresolved issue](https://github.com/hpe-storage/csi-driver/issues/323) with "ReadWriteMany" access modes on `PersistentVolumeClaims` utilizing "volumeMode: Block", the HPE CSI Operator only support OpenShift Virtualization with the HPE Alletra 5000/6000 and Nimble Storage CSP. - !!! seealso "Pointers" - Other combinations may work but will not be supported. - Both Red Hat Enterprise Linux and Red Hat CoreOS worker nodes are supported. - Instructions on this page only reflect the current stable version of the HPE CSI Operator and OpenShift. + - OpenShift Virtualization OS images are only supported on `PVCs` using "RWX" with `volumeMode: Block`. ### Security model @@ -112,6 +111,8 @@ Once the SCC has been applied to the project, login to the OpenShift web console ![Click Install](img/webcon-2.png) *Click 'Install'.* +!!! note + Latest supported HPE CSI Operator on OpenShift 4.14 is 2.4.1 ![Click Install](img/webcon-3.png) *Select the Namespace where the SCC was applied, select 'Manual' Update Approval, click 'Install'.* @@ -266,10 +267,13 @@ parameters: # Unsupported Helm Chart Install -In the event Red Hat releases a new release of OpenShift between HPE CSI driver releases or if interest arises to run the HPE CSI Driver on an uncertified version of OpenShift, it's possible to install the CSI driver using the Helm chart instead. +In the event Red Hat releases a new release of OpenShift between HPE CSI Driver releases or if interest arises to run the HPE CSI Driver on an uncertified version of OpenShift, it's possible to install the CSI driver using the Helm chart instead. It's not recommended to install the Helm chart unless it's listed as "Field Tested" in the [support matrix](#certified_combinations) above. +!!! tip + Helm chart install is also only current method to use beta releases of the HPE CSI Driver. + ## Steps to install. - Follow the steps in the [prerequisites](#prerequisites) to apply the `SCC` in the `Namespace` (Project) you wish to install the driver. diff --git a/docs/partners/vmware/index.md b/docs/partners/vmware/index.md index ea76eafb..78754de3 100644 --- a/docs/partners/vmware/index.md +++ b/docs/partners/vmware/index.md @@ -8,7 +8,7 @@ CNS fully supports Storage Policy-Based Management (SPBM) to provision volumes. ### Feature Comparison -Volume parameters available to the vSphere Container Storage Plug-in will be dependent upon options exposed through the vSphere SPBM and may not include all volume features available. Please refer to the [HPE Primera: VMware ESXi Implementation Guide](https://support.hpe.com/hpesc/public/docDisplay?docId=sd00001341en_us&docLocale=en_US&page=index.html) (includes HPE GreenLake for Block Storage, Alletra 9000 and 3PAR) or [VMware vSphere Virtual Volumes on HPE Nimble Storage Implementation Guide](https://www.hpe.com/psnow/doc/a00044881enw) (includes HPE Alletra 5000/6000 and dHCI) for list of available features. +Volume parameters available to the vSphere Container Storage Plug-in will be dependent upon options exposed through the vSphere SPBM and may not include all volume features available. Please refer to the [HPE Primera: VMware ESXi Implementation Guide](https://support.hpe.com/hpesc/public/docDisplay?docId=sd00001341en_us&docLocale=en_US&page=index.html) (includes HPE Alletra Storage MP, Alletra 9000 and 3PAR) or [VMware vSphere Virtual Volumes on HPE Nimble Storage Implementation Guide](https://www.hpe.com/psnow/doc/a00044881enw) (includes HPE Alletra 5000/6000 and dHCI) for list of available features. For a list of available volume parameters in the HPE CSI Driver for Kubernetes, refer to the respective [CSP](../../container_storage_provider/index.md). @@ -38,7 +38,7 @@ For a list of available volume parameters in the HPE CSI Driver for Kubernetes, 1 = Feature comparison based upon HPE CSI Driver for Kubernetes 2.4.0 and the vSphere Container Storage Plug-in 3.1.2
2 = HPE and VMware fully support features listed as GA for their respective CSI drivers.
- 3 = The HPE Remote Copy Peer Persistence feature of the HPE CSI Driver for Kubernetes is only available with HPE Alletra MP/9000 and Primera storage systems.
+ 3 = The HPE Remote Copy Peer Persistence feature of the HPE CSI Driver for Kubernetes is only available with HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR storage systems.
4 = Peer Persistence is an HPE Storage specific platform feature that isn't abstracted up to the vSphere Container Storage Plug-in. Peer Persistence works with the vSphere Container Storage Plug-in when using VMFS datastores. diff --git a/docs/welcome/img/alletramp.png b/docs/welcome/img/alletramp.png new file mode 100644 index 00000000..3c4ec39c Binary files /dev/null and b/docs/welcome/img/alletramp.png differ diff --git a/docs/welcome/index.md b/docs/welcome/index.md index aa9772af..a31c7424 100644 --- a/docs/welcome/index.md +++ b/docs/welcome/index.md @@ -2,9 +2,9 @@ HPE provides a broad portfolio of products that integrate with Kubernetes and neighboring ecosystems. The following table provides an overview of integrations available for each primary storage platform. -| Ecosystem | ![](img/alletra6000.png)

HPE Alletra 5000/6000 and Nimble | ![](img/alletra9000.png)

HPE Alletra 9000, Primera and 3PAR | +| Ecosystem | ![](img/alletra6000.png)

HPE Alletra 5000/6000 and Nimble | ![](img/alletramp.png)

HPE Alletra Storage MP, Alletra 9000, Primera and 3PAR | | :-: | :-: | :-: | -| Kubernetes | [HPE CSI Driver](../csi_driver/index.md) with [Alletra 6000 CSP](../container_storage_provider/hpe_alletra_6000/index.md) |[HPE CSI Driver](../csi_driver/index.md) with [Alletra 9000 CSP](../container_storage_provider/hpe_alletra_9000/index.md) | +| Kubernetes | [HPE CSI Driver](../csi_driver/index.md) with [Alletra 6000 CSP](../container_storage_provider/hpe_alletra_6000/index.md) |[HPE CSI Driver](../csi_driver/index.md) with [Alletra Storage MP CSP](../container_storage_provider/hpe_alletra_storage_mp/index.md) | - Looking to [deploy the CSI driver](../csi_driver/deployment.md)? @@ -12,11 +12,11 @@ HPE provides a broad portfolio of products that integrate with Kubernetes and ne Interested in acquiring a persistent storage solution for your Kubernetes project? -| Criteria | ![](img/alletra6000.png)

HPE Alletra 5000/6000 | ![](img/alletra9000.png)

HPE Alletra 9000 | +| Criteria | ![](img/alletra6000.png)

HPE Alletra 5000/6000 | ![](img/alletramp.png)

HPE Alletra Storage MP | | :-------------: | :-------------------------------------------------------: | :-------------------------------------------------------: | | Availability | 99.9999% | 100% | | Workloads | Business-critical | Mission-critical | -| Learn more | [hpe.com/storage/alletra](http://hpe.com/storage/alletra) | [hpe.com/storage/alletra](http://hpe.com/storage/alletra) | +| Learn more | [hpe.com/storage/alletra](http://hpe.com/storage/alletra) | [hpe.com/storage/greenlake](https://www.hpe.com/us/en/hpe-greenlake-block-storage.html) | # Other HPE storage platforms diff --git a/hpe_theme/main.html b/hpe_theme/main.html index 09cc9876..b20f31b7 100644 --- a/hpe_theme/main.html +++ b/hpe_theme/main.html @@ -2,8 +2,8 @@ {% block site_meta %} {{ super() }} - {% if page %} - {% endif %} + {% if page %} + {% endif %} diff --git a/mkdocs.yml b/mkdocs.yml index 8e5404f4..0833b470 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -9,7 +9,7 @@ theme: gtag: 'G-PC28RTKKTW' repo_name: 'hpe-storage/scod' repo_url: 'https://github.com/hpe-storage/scod' -copyright: 'Copyright 2020-2023 Hewlett Packard Enterprise Development LP' +copyright: 'Copyright 2020-2024 Hewlett Packard Enterprise Development LP' extra_css: - css/hpedev.css use_directory_urls: false @@ -30,7 +30,8 @@ plugins: 'docker_volume_plugins/hpe_cloud_volumes.md': 'docker_volume_plugins/hpe_cloud_volumes/index.md' 'docker_volume_plugins/hpe_nimble_storage.md': 'docker_volume_plugins/hpe_nimble_storage/index.md' 'container_storage_provider/hpe_nimble_storage/index.md': 'container_storage_provider/hpe_alletra_6000/index.md' - 'container_storage_provider/hpe_3par_primera/index.md': 'container_storage_provider/hpe_alletra_9000/index.md' + 'container_storage_provider/hpe_3par_primera/index.md': 'container_storage_provider/hpe_alletra_storage_mp/index.md' + 'container_storage_provider/hpe_alletra_9000/index.md': 'container_storage_provider/hpe_alletra_storage_mp/index.md' 'partners/rancher_labs/index.md': 'partners/suse_rancher/index.md' 'ezmeral_container_platform/install.md': 'ezmeral/install.md' - include-markdown: @@ -48,7 +49,7 @@ nav: - 'Diagnostics': 'csi_driver/diagnostics.md' - CONTAINER STORAGE PROVIDERS: - 'HPE Alletra 5000/6000 and Nimble': 'container_storage_provider/hpe_alletra_6000/index.md' - - 'HPE Alletra 9000 and Primera/3PAR': 'container_storage_provider/hpe_alletra_9000/index.md' + - 'HPE Alletra Storage MP and Alletra 9000/Primera/3PAR': 'container_storage_provider/hpe_alletra_storage_mp/index.md' # - WORKLOAD BLUEPRINTS: # - 'Running MongoDB on HPE Storage': 'workloads/mongodb/index.md' # - 'Hybrid Cloud CI/CD pipelines': 'workloads/mongodb/hybrid_cloud_cicd.md'