v2.3.0 for nats migration and last bionic

[Migration Release] The purpose of this release is to provide a step to migrate to v3.0.0. This works by the following method: * This version starts using the nats mbus certs instead of username and password, which becomes mandatory in bosh v274.0.0. These certs were installed several releases ago, but were not in use. They must be in use before upgrading beyond this version * Once deployed, each deployment under the bosh director will need to get recreated(`genesis <env> deploy --recreate`) in order to propagate the nats changes. * You can then upgrade to v3.0.0, and then recreate the deployments under the director once again to finalize the upgrade. This is also the last verison to support bionic stemcells. It also supports jammy, using the feature `jammy` to turn it on. Unfortunately, there are no precompiled releases other than bosh-v273.1.0 for bionic, so those will be built from source if using the default bionic configuration.
genesis-community · Dec 19, 2022 · 87f835e · 87f835e
1 parent ecf625f
commit 87f835e
Show file tree

Hide file tree

Showing 96 changed files with 1,026 additions and 927 deletions.
diff --git a/bosh-deployment/.github/pull_request_template.md b/bosh-deployment/.github/pull_request_template.md
@@ -0,0 +1 @@
+Note: Please create PR's against the develop branch
diff --git a/bosh-deployment/.gitignore b/bosh-deployment/.gitignore
@@ -0,0 +1,4 @@
+*.json
+creds.yml
+tmp
+.idea
diff --git a/bosh-deployment/README.md b/bosh-deployment/README.md
@@ -16,12 +16,16 @@ The following certificates are affected by this change and will need to be regen
 If you're using Credhub or another external variable store, then you will need to use `update_mode: converge` as documented here: <https://bosh.io/docs/manifest-v2/#variables>.<br>
 If you are not using Credhub or another external variable store, then you will need to follow the usual procedure for regenerating your certificates.
 
+## Jammy stemcells
+
+We deploy using Jammy stemcells; however, if you would prefer to use the Bionic stemcells, append the ops files `[IAAS]/use-bionic.yml` and `misc/source-releases/bosh.yml` after the ops file `[IAAS]/cpi.yml`.
+
 ## How is bosh-deployment updated?
 An automatic process updates Bosh, and other releases within bosh-deployment
 
 1. A new release of [bosh](https://github.com/cloudfoundry/bosh) is created.
 1. A CI pipeline updates bosh-deployment on `develop` with a compiled bosh release.
-1. Smoke tests are performed to ensure `create-env` works with this potential collection of resources and the new release. 
+1. Smoke tests are performed to ensure `create-env` works with this potential collection of resources and the new release.
 1. A commit to `master` is made.
 
 Other releases such as [UAA](https://github.com/cloudfoundry/uaa-release), [CredHub](https://github.com/pivotal-cf/credhub-release), and various CPIs are also updated automatically.
@@ -60,6 +64,7 @@ Other releases such as [UAA](https://github.com/cloudfoundry/uaa-release), [Cred
 - `bosh.yml`: Base manifest that is meant to be used with different CPI configurations
 - `[alicloud|aws|azure|docker|gcp|openstack|softlayer|vcloud|vsphere|virtualbox]/cpi.yml`: CPI configuration
 - `[alicloud|aws|azure|docker|gcp|openstack|softlayer|vcloud|vsphere|virtualbox]/cloud-config.yml`: Simple cloud configs
+- `[alicloud|aws|azure|docker|gcp|openstack|vcloud|virtualbox|vsphere|warden]/use-bionic.yml`: use Bionic stemcell instead of Jammy stemcell
 - `jumpbox-user.yml`: Adds user `jumpbox` for SSH-ing into the Director (see [Jumpbox User](docs/jumpbox-user.md))
 - `uaa.yml`: Deploys UAA and enables UAA user management in the Director
 - `credhub.yml`: Deploys CredHub and enables CredHub integration in the Director

diff --git a/bosh-deployment/alicloud/cpi.yml b/bosh-deployment/alicloud/cpi.yml
@@ -3,15 +3,15 @@
   type: replace
   value:
     name: bosh-alicloud-cpi
-    sha1: 9f3c5c75bb08b92c047fd128470d392cb5d9e113
-    url: https://github.com/cloudfoundry-incubator/bosh-alicloud-cpi-release/releases/download/v44.0.0/bosh-alicloud-cpi-release-44.0.0.tgz
-    version: 44.0.0
+    sha1: f01fcd9d5a775755086513692d92630e0a77e8ea
+    url: https://github.com/cloudfoundry-incubator/bosh-alicloud-cpi-release/releases/download/v45.0.0/bosh-alicloud-cpi-release-45.0.0.tgz
+    version: 45.0.0
 - name: stemcell
   path: /resource_pools/name=vms/stemcell?
   type: replace
   value:
-    sha1: b4b4291c9c1b5146d4d3cdf7aa4a5779e6d15c25
-    url: https://bosh-alicloud-light-stemcells-cn.oss-cn-hangzhou.aliyuncs.com/light-bosh-stemcell-1.61-alicloud-kvm-ubuntu-bionic-go_agent.tgz
+    sha1: 1b7c8b0673b14d96cdb232de17a8fbba969e0846
+    url: https://bosh-alicloud-light-stemcells-cn.oss-cn-hangzhou.aliyuncs.com/light-bosh-stemcell-1.2-alicloud-kvm-ubuntu-jammy-go_agent.tgz
 - path: /resource_pools/name=vms/cloud_properties?
   type: replace
   value:
@@ -64,13 +64,6 @@
     access_key_secret: ((access_key_secret))
     availability_zone: ((zone))
     region: ((region))
-- path: /cloud_provider/ssh_tunnel?
-  type: replace
-  value:
-    host: ((internal_ip))
-    port: 22
-    private_key: ((private_key))
-    user: vcap
 - path: /instance_groups/name=bosh/properties/agent/env/bosh/ntp?
   type: replace
   value:

diff --git a/bosh-deployment/alicloud/use-bionic.yml b/bosh-deployment/alicloud/use-bionic.yml
@@ -0,0 +1,6 @@
+- name: stemcell
+  path: /resource_pools/name=vms/stemcell?
+  type: replace
+  value:
+    sha1: 87d9ca3b10332e395eca921dd39df63b804a7b8d
+    url: https://bosh-alicloud-light-stemcells-cn.oss-cn-hangzhou.aliyuncs.com/light-bosh-stemcell-1.92-alicloud-kvm-ubuntu-bionic-go_agent.tgz
diff --git a/bosh-deployment/aws/cpi.yml b/bosh-deployment/aws/cpi.yml
@@ -10,20 +10,20 @@
   path: /resource_pools/name=vms/stemcell?
   type: replace
   value:
-    sha1: 2cff3720e7c23be2c1ae05547ce444bcb681a2ef
-    url: https://storage.googleapis.com/bosh-aws-light-stemcells/1.61/light-bosh-stemcell-1.61-aws-xen-hvm-ubuntu-bionic-go_agent.tgz
+    sha1: d4f5b9bd6b72f3402659007bef11a11adecc063f
+    url: https://storage.googleapis.com/bosh-aws-light-stemcells/1.2/light-bosh-stemcell-1.2-aws-xen-hvm-ubuntu-jammy-go_agent.tgz
 - path: /resource_pools/name=vms/cloud_properties?
   type: replace
   value:
     availability_zone: ((az))
     ephemeral_disk:
       size: 25000
-      type: gp2
-    instance_type: m5.xlarge
+      type: gp3
+    instance_type: t3.medium
 - path: /disk_pools/name=disks/cloud_properties?
   type: replace
   value:
-    type: gp2
+    type: gp3
 - path: /networks/name=default/subnets/0/cloud_properties?
   type: replace
   value:
@@ -49,13 +49,6 @@
     default_security_groups: ((default_security_groups))
     region: ((region))
     secret_access_key: ((secret_access_key))
-- path: /cloud_provider/ssh_tunnel?
-  type: replace
-  value:
-    host: ((internal_ip))
-    port: 22
-    private_key: ((private_key))
-    user: vcap
 - path: /cloud_provider/properties/aws?
   type: replace
   value:

diff --git a/bosh-deployment/aws/use-bionic.yml b/bosh-deployment/aws/use-bionic.yml
@@ -0,0 +1,6 @@
+- name: stemcell
+  path: /resource_pools/name=vms/stemcell?
+  type: replace
+  value:
+    sha1: e86da2e4e73a33ec2494b721b7fa4f58c892c9ef
+    url: https://storage.googleapis.com/bosh-aws-light-stemcells/1.92/light-bosh-stemcell-1.92-aws-xen-hvm-ubuntu-bionic-go_agent.tgz
diff --git a/bosh-deployment/azure/cpi.yml b/bosh-deployment/azure/cpi.yml
@@ -3,19 +3,19 @@
   type: replace
   value:
     name: bosh-azure-cpi
-    sha1: 42b9d32621d7cdc0a7eda9754197d01fcdaf4593
-    url: https://bosh.io/d/github.com/cloudfoundry/bosh-azure-cpi-release?v=38.0.0
-    version: 38.0.0
+    sha1: dce9cbc36b19ede2270ad37d73ad5ad434e86f9f
+    url: https://bosh.io/d/github.com/cloudfoundry/bosh-azure-cpi-release?v=39.0.0
+    version: 39.0.0
 - name: stemcell
   path: /resource_pools/name=vms/stemcell?
   type: replace
   value:
-    sha1: 76d1fccd1e09d6486464387324ec5d96ffd44845
-    url: https://storage.googleapis.com/bosh-core-stemcells/1.61/bosh-stemcell-1.61-azure-hyperv-ubuntu-bionic-go_agent.tgz
+    sha1: c1d3b2d7b9f6ba1b87f589ffdc6f99a4c1651a2a
+    url: https://storage.googleapis.com/bosh-core-stemcells/1.2/bosh-stemcell-1.2-azure-hyperv-ubuntu-jammy-go_agent.tgz
 - path: /resource_pools/name=vms/cloud_properties?
   type: replace
   value:
-    instance_type: Standard_D1_v2
+    instance_type: Standard_B2s
 - path: /networks/name=default/subnets/0/cloud_properties?
   type: replace
   value:
@@ -47,13 +47,6 @@
     storage_account_name: ((storage_account_name))
     subscription_id: ((subscription_id))
     tenant_id: ((tenant_id))
-- path: /cloud_provider/ssh_tunnel?
-  type: replace
-  value:
-    host: ((internal_ip))
-    port: 22
-    private_key: ((ssh.private_key))
-    user: vcap
 - path: /cloud_provider/properties/azure?
   type: replace
   value:

diff --git a/bosh-deployment/azure/use-bionic.yml b/bosh-deployment/azure/use-bionic.yml
@@ -0,0 +1,6 @@
+- name: stemcell
+  path: /resource_pools/name=vms/stemcell?
+  type: replace
+  value:
+    sha1: 287b2e164f76420c9e62ad4ccfa26b88653dca7e
+    url: https://storage.googleapis.com/bosh-core-stemcells/1.92/bosh-stemcell-1.92-azure-hyperv-ubuntu-bionic-go_agent.tgz
diff --git a/bosh-deployment/bbr.yml b/bosh-deployment/bbr.yml
@@ -3,9 +3,9 @@
   type: replace
   value:
     name: backup-and-restore-sdk
-    sha1: e26e4d11df866687f3fe5dd2414057ba29e1c1a8
-    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/backup-and-restore-sdk-1.18.35-ubuntu-bionic-1.61-20220302-141721-812919658-20220302141726.tgz
-    version: 1.18.35
+    sha1: 7ea40c99fe8db0233b8d0c26377825f85dda049f
+    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/backup-and-restore-sdk-1.18.49-ubuntu-jammy-1.2-20220811-173129-61955977-20220811173135.tgz
+    version: 1.18.49
 - path: /instance_groups/name=bosh/jobs/-
   type: replace
   value:

diff --git a/bosh-deployment/bosh-lite-docker.yml b/bosh-deployment/bosh-lite-docker.yml
@@ -3,9 +3,9 @@
   type: replace
   value:
     name: bosh-docker-cpi
-    sha1: 3c3d3f988b57d2844027aad79ec391108765c5a5
-    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-docker-cpi-0.0.5-ubuntu-bionic-1.61-20220215-132327-173623787-20220215132331.tgz
-    version: 0.0.5
+    sha1: e48c1606f9bcc9181304be17ab65ea42389cd5e0
+    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-docker-cpi-0.0.7-ubuntu-jammy-1.2-20220726-140852-892114202-20220726140856.tgz
+    version: 0.0.7
 - path: /releases/-
   type: replace
   value:
@@ -16,10 +16,10 @@
 - path: /releases/name=os-conf?
   type: replace
   value:
-    name: "os-conf"
-    sha1: "386293038ae3d00813eaa475b4acf63f8da226ef"
-    url: "https://bosh.io/d/github.com/cloudfoundry/os-conf-release?v=22.1.2"
-    version: "22.1.2"
+    name: os-conf
+    sha1: 78d79f08ff5001cc2a24f572837c7a9c59a0e796
+    url: https://bosh.io/d/github.com/cloudfoundry/os-conf-release?v=18
+    version: 18
 - path: /instance_groups/name=bosh/jobs/-
   type: replace
   value:

diff --git a/bosh-deployment/bosh-lite.yml b/bosh-deployment/bosh-lite.yml
@@ -3,16 +3,16 @@
   type: replace
   value:
     name: garden-runc
-    sha1: 7b6599e704b8e73c3b108c9c328836d158f269f0
-    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/garden-runc-1.20.0-ubuntu-bionic-1.61-20220218-191740-293631392-20220218191743.tgz
-    version: 1.20.0
+    sha1: eaebf126bf09edaf66b03f4b691f7fcf8383545c
+    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/garden-runc-1.21.0-ubuntu-jammy-1.2-20220809-192217-588420557-20220809192220.tgz
+    version: 1.21.0
 - path: /releases/-
   release: bosh-warden-cpi
   type: replace
   value:
     name: bosh-warden-cpi
-    sha1: db9e8d9cece02d5e154fd3f68d3131942465b2c3
-    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-warden-cpi-43-ubuntu-bionic-1.61-20220215-132225-460615422-20220215132227.tgz
+    sha1: 714a807558021a93b4e87aa23be31811ea319b6b
+    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-warden-cpi-43-ubuntu-jammy-1.2-20220726-140843-600524291-20220726140845.tgz
     version: 43
 - path: /instance_groups/name=bosh/jobs/-
   type: replace
@@ -44,18 +44,18 @@
       expand_stemcell_tarball: false
     agent:
       blobstore: null
-      mbus: nats://nats:((nats_password))@10.254.50.4:4222
+      mbus: nats://10.254.50.4:4222
     host_ip: 10.254.50.4
     warden:
       connect_address: 127.0.0.1:7777
       connect_network: tcp
 - path: /releases/name=os-conf?
   type: replace
   value:
-    name: "os-conf"
-    sha1: "386293038ae3d00813eaa475b4acf63f8da226ef"
-    url: "https://bosh.io/d/github.com/cloudfoundry/os-conf-release?v=22.1.2"
-    version: "22.1.2"
+    name: os-conf
+    sha1: 78d79f08ff5001cc2a24f572837c7a9c59a0e796
+    url: https://bosh.io/d/github.com/cloudfoundry/os-conf-release?v=18
+    version: 18
 - path: /instance_groups/name=bosh/jobs/-
   type: replace
   value:

diff --git a/bosh-deployment/bosh.yml b/bosh-deployment/bosh.yml
@@ -49,7 +49,7 @@ instance_groups:
           - time2.google.com
           - time3.google.com
           - time4.google.com
-      mbus: nats://nats:((nats_password))@((internal_ip)):4222
+      mbus: nats://((internal_ip)):4222
     blobstore:
       address: ((internal_ip))
       agent:
@@ -142,13 +142,13 @@ networks:
   type: manual
 releases:
 - name: bosh
-  sha1: e5de3414956e97cd4cd19ed80369b057c6968d29
-  url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-272.0.0-ubuntu-bionic-1.61-20220228-175001-491185238-20220228175002.tgz
-  version: 272.0.0
+  sha1: 2661406efce2c3377bc989bb44f86b42a3e1f8d4
+  url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bosh-273.1.0-ubuntu-jammy-1.2-20220817-212639-489172614-20220817212640.tgz
+  version: 273.1.0
 - name: bpm
-  sha1: c80d092a44bc1425d3de1f1fb0de82cbc0cc9f10
-  url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bpm-1.1.16-ubuntu-bionic-1.61-20220215-132405-853958356-20220215132409.tgz
-  version: 1.1.16
+  sha1: 2d2a133802263d54bdc40fb9d698b2ad9c575d06
+  url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/bpm-1.1.18-ubuntu-jammy-1.2-20220726-140932-290547827-20220726140934.tgz
+  version: 1.1.18
 resource_pools:
 - env:
     bosh:

diff --git a/bosh-deployment/cloudstack/cpi.yml b/bosh-deployment/cloudstack/cpi.yml
@@ -86,14 +86,6 @@
     options:
       blobstore_path: /var/vcap/micro_bosh/data/cache
 
-- path: /cloud_provider/ssh_tunnel?
-  type: replace
-  value:
-    host: ((internal_ip))
-    port: 22
-    private_key: ((private_key))
-    user: vcap
-
 - path: /cloud_provider/properties/cloudstack?
   type: replace
   value: *cloudstack
diff --git a/bosh-deployment/credhub.yml b/bosh-deployment/credhub.yml
@@ -3,9 +3,9 @@
   type: replace
   value:
     name: credhub
-    sha1: bdf818bf30b330af09b5b94244fe14f1df0e6293
-    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/credhub-2.12.0-ubuntu-bionic-1.61-20220303-231341-096485257-20220303231342.tgz
-    version: 2.12.0
+    sha1: 033d0ec54eadcc46127d6bb7721d08631333c43b
+    url: https://s3.amazonaws.com/bosh-compiled-release-tarballs/credhub-2.12.8-ubuntu-jammy-1.2-20220815-171651-280369602-20220815171652.tgz
+    version: 2.12.8
 - path: /instance_groups/name=bosh/jobs/-
   type: replace
   value:

diff --git a/bosh-deployment/docker/cpi-secondary.yml b/bosh-deployment/docker/cpi-secondary.yml
@@ -24,5 +24,5 @@
         certificate: unused
         private_key: unused
     agent:
-      mbus: nats://nats:((nats_password))@((internal_ip)):4222
+      mbus: nats://((internal_ip)):4222
       blobstore: null
diff --git a/bosh-deployment/docker/cpi.yml b/bosh-deployment/docker/cpi.yml
@@ -2,15 +2,15 @@
   type: replace
   value:
     name: bosh-docker-cpi
-    sha1: 075bc0264d2548173da55a40127757ae962a25b1
-    url: https://bosh.io/d/github.com/cppforlife/bosh-docker-cpi-release?v=0.0.5
-    version: 0.0.5
+    sha1: cd14ac95ae66d1e38a5df4523f3250fe7a51a457
+    url: https://bosh.io/d/github.com/cloudfoundry/bosh-docker-cpi-release?v=0.0.7
+    version: 0.0.7
 - name: stemcell
   path: /resource_pools/name=vms/stemcell?
   type: replace
   value:
-    sha1: 5d219eee7b586f6cbcccfd37921e2b598e3ef9a2
-    url: https://storage.googleapis.com/bosh-core-stemcells/1.61/bosh-stemcell-1.61-warden-boshlite-ubuntu-bionic-go_agent.tgz
+    sha1: 6494558d01d89d93ebdbf3293ff879a5fe9793b5
+    url: https://storage.googleapis.com/bosh-core-stemcells/1.2/bosh-stemcell-1.2-warden-boshlite-ubuntu-jammy-go_agent.tgz
 - path: /networks/name=default/subnets/0/cloud_properties?
   type: replace
   value:
@@ -44,7 +44,7 @@
   value:
     agent:
       blobstore: null
-      mbus: nats://nats:((nats_password))@((internal_ip)):4222
+      mbus: nats://((internal_ip)):4222
     docker:
       host: ((docker_host))
       tls: ((docker_tls))

diff --git a/bosh-deployment/docker/ipv6/cpi.yml b/bosh-deployment/docker/ipv6/cpi.yml
@@ -1,3 +1,3 @@
 - type: replace
   path: /instance_groups/name=bosh/properties/docker_cpi/agent/mbus
-  value: "nats://nats:((nats_password))@[((internal_ip))]:4222"
+  value: "nats://[((internal_ip))]:4222"
diff --git a/bosh-deployment/docker/use-bionic.yml b/bosh-deployment/docker/use-bionic.yml
@@ -0,0 +1,6 @@
+- name: stemcell
+  path: /resource_pools/name=vms/stemcell?
+  type: replace
+  value:
+    sha1: 4d6823188f510215355643ad766300e076ec2e5a
+    url: https://storage.googleapis.com/bosh-core-stemcells/1.92/bosh-stemcell-1.92-warden-boshlite-ubuntu-bionic-go_agent.tgz
diff --git a/bosh-deployment/experimental/postgres-10.yml b/bosh-deployment/experimental/postgres-10.yml
@@ -1,3 +1,6 @@
-# Using this file will upgrade postgres to v10. Once this file is used, downgrading back to older versions is not allowed.
-
-# Postgres 10 is now default.
+---
+- type: replace
+  path: /instance_groups/name=bosh/jobs/name=postgres?
+  value:
+    name: postgres-10
+    release: bosh
diff --git a/bosh-deployment/experimental/postgres-9-4.yml b/bosh-deployment/experimental/postgres-9-4.yml
diff --git a/bosh-deployment/external-ip-with-registry-not-recommended.yml b/bosh-deployment/external-ip-with-registry-not-recommended.yml
@@ -25,10 +25,6 @@
   path: /cloud_provider/mbus
   value: https://mbus:((mbus_bootstrap_password))@((external_ip)):6868
 
-- type: replace
-  path: /cloud_provider/ssh_tunnel/host
-  value: ((external_ip))
-
 - type: replace
   path: /variables/name=mbus_bootstrap_ssl/options/alternative_names/-
   value: ((external_ip))
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Note: Please create PR's against the develop branch