GitLab Kubernetes with CI

Scripts and infrastructure to setup example GitLab instance to demonstrate the CI improvements outlined in my blog post: How We Reduced Our GitLab CI Pipeline Duration by 70% at Student Beans.

Setup

Dependencies

Required CLI tools on your local machine:

• bash (command-not-found)
• terraform v1.2.9 (tfenv can be useful)

AWS Credentials

Create an AWS IAM user with the relevant permissions for the Terraform setup e.g. AWS S3, or use AdministratorAccess for quicker setup.

Add the access keys for the IAM user to the gitlab-kubernetes-with-ci AWS profile in the credentials list on your machine:

sudo vim ~/.aws/credentials

[gitlab-kubernetes-with-ci]
aws_access_key_id = <iam user access key id>
aws_secret_access_key = <iam user secret key>

Kubernetes Cluster

The GitLab application in this repository is configured to deploy to a Kubernetes using Helm. Please ensure you have a Kubernetes cluster created with at least 2 nodes and the cluster configuration file located on your local machine at ~/.kube/config.

The application will deploy to the default namespace in the default cluster.

The infrastructure is set up to work with an install of k3s, please adjust the configuration to match your cluster if required.

The GitLab helm chart variables can be found in terraform/infrastructure/templates/gitlab-values.yaml.tpl with basic config in terraform/.env.

Clone Repository

Clone the project to your local machine and navigate to the project root directory.

Create the Infrastructure Environment Variables

Create .env in the root directory from the .env.example template.

Ignore the TF_VAR_gitlab_runner_registration_token variable for now, since it won't be available until after GitLab is deployed.

Create the Infrastructure

Create the infrastructure:

./scripts/build-infrastructure

Add Public SSH Key to GitLab User

Add GitLab to Hosts

The GitLab UI is configured to use the k3s traefik ingress located in the code here.

The ingress will point to the domain value set in the .env file TF_VAR_gitlab_domain variable, the default value is gitlab.gitlab.example.

If your cluster is not public, you may need to add your custom domain to your hosts file to be able to connect, e.g.

# /etc/hosts
# ...

<cluster-ip> gitlab.gitlab.example

The UI will then be available at https://gitlab.gitlab.example.

Login with root User Password

Get the auto generated GitLab root user password from secret: kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

Login to the GitLab UI using the admin credentials:

username: root
password: <password>

Add your public SSH key to your GitHub user. See the GitLab SSH guide.

Create GitLab Runner

Add Registration Token

Get a runner registration token from the GitLab Runners admin page: https://gitlab.gitlab.example/admin/runners by clicking the register runner button.

Add the token to the TF_VAR_gitlab_runner_registration_token variable in your .env.

Add Container Registry Credentials to .env

Create an access token to your container registry.

Add your container registry username and token to the .env file in variables TF_VAR_container_registry_user and TF_VAR_container_registry_token respectively, these will be added to the GitLab Runner environment.

Notes:

• The runner config is configured to use DockerHub in the docker auth file, if you use something else, e.g. AWS ECR, please edit the auth config DOCKER_AUTH in terraform/infrastructure/templates/gitlab-runner-values.yaml.tpl.

Deploy Runner

Re-deploy the application using ./scripts/apply-infrastructure.

Add Sample Project

Setup Container Repository

Create the following container repositories for the sample project in DockerHub or other registry.

• <registry-user>/sample-project
• <registry-user>/sample-project-cache

Push to GitLab

Push sample project to your GitLab instance:

./scripts/push-sample-project

Run Pipeline

Trigger a GitLab CI pipeline in the sample repository on the main branch to test the runner configuration.

Teardown

To destroy all infrastructure and remote state, run the teardown script:

./scripts/destroy-infrastructure

Other things to remove:

• Remove or disable any authentication tokens you created for the container registry.
• Remove gitlab.gitlab.example entries from /etc/hosts

Notes

• This deployment of GitLab is for demonstration and is not intended to be production ready.