Skip to content

Latest commit

 

History

History
288 lines (210 loc) · 11.4 KB

README.md

File metadata and controls

288 lines (210 loc) · 11.4 KB

GCP Deployment for Elixir with Terraform

This guide demonstrates how to deploy DeployEx in Google Cloud Platform (GCP) using Terraform to programmatically set up the environment.

Setup

To begin, ensure the following applications are installed:

1. Create a Project in GCP and populate the terraform variables

Using the Google Cloud Dashboard:

  • Create a project, e. g.,deployex
  • Create a service account and generate keys. The JSON file with the keys will be downloaded to your local machine. In this example, save the file as deployex-gcp-terraform.json.
  • Retrieve access token using the google cloud terminal and the command: gcloud beta auth application-default print-access-token
  • Enable the following resources in GCP: Google Compute Engine (GCE) and Secret Manager.

2. Environment Secrets

Ensure you have access to the following secrets for storage in Secrets Manager:

SECRET NAME EXAMPLE SOURCE
DEPLOYEX_SECRET_KEY_BASE 42otsNl...Fpq3dIJ02 mix phx.gen.secret
DEPLOYEX_ERLANG_COOKIE my-cookie
DEPLOYEX_ADMIN_HASHED_PASSWORD $2b$12$...3Lu6ys538TW Bcrypt.hash_pwd_salt("my-pass")
MYAPPNAME_SECRET_KEY_BASE 42otsNl...Fpq3dIJ02 mix phx.gen.secret
MYAPPNAME_ERLANG_COOKIE my-cookie

3. Variables Configuration

Rename the file main_example.tf_ to main.tf and verify and configure the variables according to your specific environment. Ensure that you also review and update the variables file. These variables will be utilized across all Terraform templates to ensure correct setup.

4. Provisioning the Environment

Creating the Environment

Once the variables are configured, proceed with provisioning the environment. Navigate to the ./environments/prod folder and execute the following commands:

terraform plan # Check if the templates are configured correctly
terraform apply # Apply the configurations to create the environment

Important

At this stage, you might encounter issues if billing is not properly set up or if required resources are not enabled. Additionally, the access token may expire over time, so you might need to renew it.

Wait for the environment to be created. Once the provisioning is complete, you can check the instance at this address.

Updating Secret Manager

Navigate to GCP Secrets Manager, locate and update the following secrets:

  • myappname-prod-secrets:

Create a new version of the secret and add the following JSON structure as plain text (You may need to configure additional secrets if your application requires them):

{"MYAPPNAME_SECRET_KEY_BASE":"xxxxxxxxxx","MYAPPNAME_ERLANG_COOKIE":"xxxxxxxxxx"}
  • deployex-myappname-prod-secrets

Create a new version of the secret and add the following JSON structure as plain text:

{"DEPLOYEX_SECRET_KEY_BASE":"xxxxxxxxxx","DEPLOYEX_ERLANG_COOKIE":"xxxxxxxxxx","DEPLOYEX_ADMIN_HASHED_PASSWORD":"xxxxxxxxxx"}
  • myappname-stage-otp-tls-ca, myappname-stage-otp-tls-key, myappname-stage-otp-tls-crt:

Create the TLS certificates for OTP distribution using the Following script, changing the appropriate names and regions inside it.

make tls-distribution-certs

The command will generate three files: ca.crt, deployex.key and deployex.crt. Create a new version for each secret and upload each file to its respective secret using the browser's file upload button.

Warning

DEPLOYEX_ERLANG_COOKIE and MYAPPNAME_ERLANG_COOKIE MUST match, as they will be used by the OTP distribution.

5. GCI Provisioning (Manual Steps)

For initial installations or updates to deployex, access the Google Compute Instance via SSH using the dashboard. After gaining access to the GCI, you need to grant root permissions:

my-user@myfirstapp-prod-instance:~$ sudo su
root@myfirstapp-prod-instance:/home/my-user# cd ../ubuntu/
root@myfirstapp-prod-instance:/home/ubuntu# ls

Check the configuration for DeployEx and your target app by editing the deployex-config.json file:

vi deployex-config.json

Authenticate using the Google Cloud CLI:

gcloud auth login

Follow the instructions: Copy the link provided, open it in your browser, authenticate, and then paste the verification code back into the terminal. You should see a confirmation message similar to:

You are now logged in as [my-user@gmail.com].
Your current project is [xxxxx].  You can change this setting by running:
...

Since the secrets are already updated, we are going to install them in the appropriate addresses

./install-otp-certificates.sh 

# Installing Certificates env: stage at /usr/local/share/ca-certificates #
Retrieving and saving ......
[OK]

Check that the certificates are correctly installed:

ls /usr/local/share/ca-certificates
ca.crt  myappname.crt myappname.key deployex.crt  deployex.key

Edit the gcp-config.json file to add your credentials:

vi gcp-config.json
{
  "type": "service_account" # Populate it after installation
  ...
}

If you are updating DeployEx, you may need to update the deployex.sh script. This step is not necessary during the initial installation, as the script is already installed by default. To update the script, use the following commands:

version=0.3.0
rm deployex.sh
wget https://github.com/thiagoesteves/deployex/releases/download/${version}/deployex.sh -P /home/ubuntu
chmod a+x deployex.sh

Run the script to install (or update) deployex:

root@ip-10-0-1-116:/home/ubuntu# ./deployex.sh --install deployex-config.json
#           Removing Deployex              #
...
# Clean and create a new directory         #
# Start systemd                            #
# Start new service                        #
Created symlink /etc/systemd/system/multi-user.target.wants/deployex.service → /etc/systemd/system/deployex.service.

If you need to update Deployex, follow these steps to ensure that the configuration file reflects the new version:

vi deployex-config.json
{
 ...
  "version": "0.3.0-rc15",
  "os_target": "ubuntu-20.04",
  ...
}

Once the file is updated, run the update command:

root@ip-10-0-1-116:/home/ubuntu# ./deployex.sh --update deployex-config.json

Important

Depending on the new version of DeployEx, you may need to update both the deployex-config.json file and the deployex.sh script

At this point, DeployEx should be running. You can view the logs using the following commands:

tail -f /var/log/deployex/deployex-stdout.log
tail -f /var/log/deployex/deployex-stderr.log

6. Monitored App deployment

Once DeployEx is running, you MUST deploy the monitored app. This deployment involves creating the release package and the current version JSON file in the designated storage path.

Release Version

The release version file MUST be formatted in JSON and include the following information:

{
  "version": "0.1.0-9cad9cd",
  "hash": "9cad9cd3581c69fdd02ff60765e1c7dd4599d84a",
  "pre_commands": []
}

The JSON file MUST be stored at the following path: /versions/{monitored_app}/{env}/current.json

Release package

After DeployEx fetches the release file, it will download the release package for installation. The package should be located at: /dist/{monitored_app}/{monitored_app}-{version}.tar.gz

[CI/CD] Upload files to GCP from Github

Here are some useful resources with suggestions on how to automate the upload of version and release files to your environment using GitHub Actions:

7. Setting Up HTTPS Certificates with Let's Encrypt

Important

Before proceeding, make sure that the DNS is correctly configured to point to the GCP instance.

For HTTPS, you can use free certificates from Let's encrypt. In this example, we'll use cert bot for ubuntu to obtain and configure the certificates:

sudo su
apt update
apt install snapd
snap install --classic certbot
ln -s /snap/bin/certbot /usr/bin/certbot

Before installing the certificate, make a backup of the current Nginx configuration file located at /etc/nginx/sites-available/default. Certbot may modify this file, so keeping a local copy ensures you can restore it if needed. Once the backup is created, run the following command:

certbot --nginx

This command will install Certbot and automatically configure Nginx to use the obtained certificates. After Nginx is configured, the certificate paths will be set up and will look something like this:

vi /etc/nginx/sites-available/default
 ...
           ssl_certificate /etc/letsencrypt/live/myappname.com/fullchain.pem; # managed by Certbot
           ssl_certificate_key /etc/letsencrypt/live/myappname.com/privkey.pem; # managed by Certbot
           include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
           ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

Update your configuration file to include the Let's Encrypt certificate paths. Find the section where it mentions:

           # Add here the letsencrypt paths

replace this comment with the actual certificate paths:

               proxy_set_header Upgrade $http_upgrade;
               proxy_set_header Connection "upgrade";

               proxy_pass http://deployex;
           }
           ssl_certificate /etc/letsencrypt/live/myappname.com/fullchain.pem; # managed by Certbot
           ssl_certificate_key /etc/letsencrypt/live/myappname.com/privkey.pem; # managed by Certbot
           include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
           ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
       }

Also, ensure that port 443 is enabled for both servers. For example:

      server {
          listen 443 ssl; # managed by Certbot

After modifying the configuration file, save the changes and restart Nginx:

sudo su
vi /etc/nginx/sites-available/default
# modify and save file
systemctl reload nginx

Note

After the changes, It may require a reboot.

The commands above will configure Nginx for the correct routing. Once this is set up, verify that the monitored app’s configuration file /config/runtime.exs points to the correct SCHEME/HOST/PORT, For example:

    url: [host: "myappname.com", port: 443, scheme: "https"],