Replacing a master node on user-provisioned bare-metal deployment

[ALL-SPACE-Anas]

Hi,

We have a user-provisioned bare-metal deployment of OKD4.15. It's only 3 master servers. One of the servers is faulty so I'm looking to take it out of the cluster and add another server in place of it. This cluster was initially installed by creating a bootstrap VM that provisioned the master nodes.

In the documentation it mentions,

you must create the new control plane node using the same method that was used to originally create it.

Now, I'm not entirely sure to what extend I need to be using the same method. Since this talks only about creating the new control panel, I'm assuming that steps to remove the existing node are still the same as described in the documentation link. About creating the master node using the same method, the bootstrap node that was used to first create the cluster doesn't exist anymore (I do have the ignition files though). I have several questions about the process involved,

• Do I need to create the bootstrap node again and then the master node by booting with the ignition files?
• If yes, wouldn't the certificates in the ignition files expired by now leading to installation failure (as mentioned here)?
• If I generate new ignition files, wouldn't they be representing a different cluster instance?
• Would the new master node sync with the cluster automatically? There must be some manual steps involved, and if there are, what are those?

Basically, what I'm looking for is some well-defined steps or at least a rough guideline on how to do it for user-provisioned instance on bare-metal instead of the "using the same method". Really appreciate if anyone can get back to me on this.

Thanks,

[melledouwsma]

Do I need to create the bootstrap node again and then the master node by booting with the ignition files?

The bootstrap node is only required to create the cluster the first time, it's not needed when replacing a control-plane node in a cluster

If yes, wouldn't the certificates in the ignition files expired by now leading to installation failure (as mentioned here)?
If I generate new ignition files, wouldn't they be representing a different cluster instance?

Yes, generating new ignition files will create a new cluster. You can extract the current ignition files from the cluster and use those. Please see How to retrieve Master or Worker Ignition Configuration from OpenShift Container Platform 4? for more on that.

Would the new master node sync with the cluster automatically? There must be some manual steps involved, and if there are, what are those?

You can read more on replacing nodes on How to replace unhealthy master nodes in UPI on OpenShift 4.x ?.