Skip to content

Commit

Permalink
issue-9 support Duck DB in Evidence Docker Devenv (#10)
Browse files Browse the repository at this point in the history 
This PR
1. Fixes:  #9
    * Original issue DuckDB depends on glibc that is not shipped with alpine linux.  
    * Fix was to provide a ubuntu image
3. Push a arm64 image in addition to amd64 image on CI 
4. Push a `lite` image for non-duckdb use in case someone needs to save space on their HD. 
5. Evidence now has a dockerhub org (than an individual
5. Replaced the docs
  • Loading branch information
@ud3sh
ud3sh authored Oct 3, 2023
1 parent 54b7edf commit 1ff7bf7
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 77 deletions.
33 changes: 20 additions & 13 deletions .github/workflows/build-and-publish.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,30 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Login to Docker Hub
uses: docker/login-action@v1
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Build and push
uses: docker/build-push-action@v2
- name: Build and push main image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/devenv:latest
platforms: linux/amd64,linux/arm64
tags: evidencedev/devenv:latest
- name: Build and push lite image (no DuckDB support)
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile-lite
push: true
platforms: linux/amd64,linux/arm64
tags: evidencedev/devenv-lite:latest
9 changes: 5 additions & 4 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
FROM node:19-alpine3.16
FROM sitespeedio/node:ubuntu-22.04-nodejs-18.16.0
ARG WORKSPACE_DIR=/evidence-workspace

RUN apk add --no-cache bash curl wget nano git && \
apk add xdg-utils && \
RUN apt-get update && apt-get install -y \
curl wget nano git xdg-utils && \
npm install -g degit && \
mkdir -p ${WORKSPACE_DIR} && \
mkdir -p /evidence-bin
mkdir -p /evidence-bin && \
rm -rf /var/lib/apt/lists/*

COPY bootstrap.sh /evidence-bin/bootstrap.sh
WORKDIR ${WORKSPACE_DIR}
Expand Down
13 changes: 13 additions & 0 deletions Dockerfile-lite
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM node:18-alpine3.16
ARG WORKSPACE_DIR=/evidence-workspace

RUN apk add --no-cache bash curl wget nano git && \
apk add xdg-utils && \
npm install -g degit && \
mkdir -p ${WORKSPACE_DIR} && \
mkdir -p /evidence-bin

COPY bootstrap.sh /evidence-bin/bootstrap.sh
WORKDIR ${WORKSPACE_DIR}

ENTRYPOINT [ "bash", "/evidence-bin/bootstrap.sh" ]
120 changes: 60 additions & 60 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,88 +1,88 @@
# Docker Development Environment

This repository builds the Evidence development environment Docker image. An instance container can be used as a development environment for Evidence projects using a mounted directory. Using this container allows end users to develop Evidence sites without the need for installing any toolchains besides `Docker`.
The Evidence Docker Development Environment (devenv) image is available on [Docker Hub](https://hub.docker.com/repositories/evidencedev).

## Pre-requisites
Docker tools are installed using [Docker Desktop](https://www.docker.com/products/docker-desktop/) (recommended) OR using [binaries](https://docs.docker.com/engine/install/binaries/).
The `devenv` image can be used as a development environment for Evidence projects by running it as a container with a mounted directory. Utilizing this container allows developers to work on Evidence sites without the need to install any additional toolchains other than `Docker`. For instance, there is no necessity to install `npm` or `node`.

## Running the development environment using Docker commands (Alternative 1)
This repository contains the Dockerfiles, the publishing actions, and usage documentation pertaining to [Evidence `devenv` images](https://hub.docker.com/repositories/evidencedev).

* Creating a **new Evidence project** from scratch using the Evidence project template
```
cd <path-to-your-evidence-project-root> #i.e the directory where you'd like your Evidence project to be rooted
docker pull evidencedev/devenv:latest
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm evidencedev/devenv:latest --init
## Using the Evidence Docker Development Environment

### Pre-requisites
Ensure the Docker tool chain is installed via [Docker Desktop](https://www.docker.com/products/docker-desktop/) (recommended) OR using [binaries](https://docs.docker.com/engine/install/binaries/).

# - You should see the template site up when you point your browser to localhost:3000.
# - You should see new files, copied from the Evidence project template, in <path-to-your-evidence-project-root>.
# - Any subsequent edits made in <path-to-your-evidence-project-root> should be reflected on the browser.
### Starting the Docker Evidence Development Environment

#### Option 1: Create a **new Evidence project** from scratch using the Evidence project template:
```
cd <path-to-your-evidence-project-root>
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm evidencedev/devenv:latest --init
```
* In this case, `<path-to-your-evidence-project-root>` should be an empty directory.
* You should see the template site up when you point your browser to `localhost:3000`.
* You should see new files, copied from the Evidence project template, in `<path-to-your-evidence-project-root>`.
* Any subsequent edits made in `<path-to-your-evidence-project-root>` should be reflected on the browser.
* If you are using Windows without PowerShell, you will need to replace `$(pwd)` with the full path to your Evidence project root directory or `%cd%`

* Work with an **existing Evidence** project
#### Option 2: Work with an **existing Evidence** project
```
cd <path-to-your-evidence-project-root>
docker pull evidencedev/devenv:latest
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm evidencedev/devenv:latest
# - You should see your site up when you point your browser to localhost:3000.
# - Any edits made in <path-to-your-evidence-project-root> should be reflected on the browser.
```
* In this case, `<path-to-your-evidence-project-root>` should should contain an Evidence project.
* You should see your site up when you point your browser to `localhost:3000`.
* Any edits made in `<path-to-your-evidence-project-root>` should be reflected on the browser.
* If you are using Windows without PowerShell, you will need to replace `$(pwd)` with the full path to your Evidence project root directory or `%cd%`

Note: if you are running Evidence from a new Apple Silicon MacBook (or any machine with an `arm` chipset), you'll have to provide a `--platform linux/amd64` argument to Docker as well.
##### Note for M1/M2 Mac Users using DuckDB
* If you are using an ARM M1/M2 Mac (Late 2021 Macs and later), you will need to use the `--platform linux/amd64` flag to run the Evidence devenv container with either of the options above. For instance:
```
cd <path-to-your-evidence-project-root>
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm --platform linux/amd64 evidencedev/devenv:latest
```
* Startup can be quite slow due to emulation. You may want to consider an alternative approach such as using Codespaces mentioned below.

## Running the development environment using a helper script (Alternative 2)
Download `https://github.com/evidence-dev/docker-devenv/blob/main/start-devenv.sh`.
### Alternative Start Options

* To create a **new Evidence project**, execute the following commands
```
chmod +x <path-to>/start-devenv.sh
cd <path-to-your-new-evidence-project-root>
<path-to>/start-devenv.sh --init
```
#### Using a Script to Start the Docker Development Evironment
If you'd rather not type out the docker commands, you have the option of starting the devenv with a [script](./starting-with-script.md) detailed [here](./running-with-script.md).

* To work with an **existing Evidence** project, execute the following commands.
```
chmod +x <path-to>/start-devenv.sh
cd <path-to-your-existing-evidence-project-root>
<path-to>/start-devenv.sh
```
#### Running a Smaller Image
If you are not using DuckDB, and you have limited storage on your disk, you can use a smaller `devenv` image by replacing the `evidencedev/devenv:latest` image with `evidencedev/devenv-lite:latest` in the above commands.

Run `start-devenv.sh --help` for more details on using this script.

As the project develops, we'll likely add more options to this script. This script simply masks all the options that need to be provided to `docker`.
### Connecting to a Database from the Development Container
* You can setup your DB connection as [documented](https://docs.evidence.dev/core-concepts/data-sources/). However there are few caveats to note when using the Evidence development environment container.
* If your database is hosted on your `host` machine, you'll have to ensure that the Database host is set to `host.docker.internal` either via the settings or your database config file (instead of `localhost`, `0.0.0.0`, etc). For instance:
```
{
"host": "host.docker.internal",
"database": "yourDBname",
"port": 5432,
"user": "yourUsername"
}
```
* If your database is hosted externally (e.g on the cloud), you'll have to ensure your docker container has permissions to access the outside world.
## Connecting to a Dababase from the development container
### Stopping the Running devenv container
* Option 1: `docker ps` to list all running containers, and copy the container ID of the running Evidence development environment container, and then run `docker stop <container-id>`.
* Option 2: Use `Ctrl+C` to stop the running container on terminal.
If your database is hosted on your `host` machine, you'll have to ensure that the Database host is set to `host.docker.internal` either via the settings or your database config file (instead of `localhost`, `0.0.0.0`, etc). For instance:
```
{
"host": "host.docker.internal",
"database": "yourDBname",
"port": 5432,
"user": "yourUsername"
}
```
If your database is hosted externally (e.g on the cloud), you'll have to ensure your docker container has permissions to access the outside world.
## Stopping the running container
Abort the terminal window using `CTRL+C` or use the Docker [command line](https://docs.docker.com/engine/reference/commandline/stop/) or Docker Desktop to stop the running container.
## Alternative to Using the Evidence Devenv
Github Codespaces are another way to setup an Evidence dev environment without installing `npm`, `node` etc. See [Evidence installation docs](https://docs.evidence.dev/getting-started/install-evidence) for more information.
## Building and running the image locally
* This only applies to the development of this Docker image.
## Development notes
This section only applies if you are contributing to this repo.
### Testing the image locally
```
docker build -t <image-name> .
docker build -t <test-image-name> .
cd <path-to-your-evidence-project-root>
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm <image-name> <command-to-run>
docker run -v=$(pwd):/evidence-workspace -p=3000:3000 -it --rm <test-image-name> <command-to-run>
```
## Publishing the latest image to Docker Hub
Currently the image is hosted on Dockerhub. To build and publish a new version, follow these steps
1. Login to Dockerhub => `docker login`
2. Build the image => `docker build -t evidencedev/devenv:latest .`
3. Push the image to Dockerhub => `docker push evidencedev/devenv:latest`

For login credentials, see `EvidenceDev Dockerhub Admin` in 1password. This is setup under `udesh@evidence.dev` (didn't think to create a google group for this at the time e.g devs@evidence.dev - will do so in the future - feel free to use it in the meantime).

We should consider hosting this in AWS with CD/CI setup to automatically publish new versions of the image from main
### Manually publishing the latest image to Docker Hub (Evidence team only)
Currently the image is hosted on Docker Hub and are re-built on pushes to main. To build and publish a new version manually, follow these steps
1. Login to Docker Hub => `docker login`
2. Build and push the image => `docker buildx build --platform linux/amd64,linux/arm64 -t evidencedev/devenv:latest . --push`
20 changes: 20 additions & 0 deletions starting-with-script.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## Running the development environment using a helper script
This documents an alternative way to start the Evidence `devenv` that is a wraps `docker` commands described in the [main README](https://github.com/evidence-dev/docker-devenv/tree/main#readme).

* Download `https://github.com/evidence-dev/docker-devenv/blob/main/start-devenv.sh`.
* To create a **new Evidence project**, execute the following commands
```
chmod +x <path-to>/start-devenv.sh
cd <path-to-your-new-evidence-project-root>
<path-to>/start-devenv.sh --init
```
* To work with an **existing Evidence** project, execute the following commands.
```
chmod +x <path-to>/start-devenv.sh
cd <path-to-your-existing-evidence-project-root>
<path-to>/start-devenv.sh
```

Run `start-devenv.sh --help` for more details on using this script.

As the project develops, we'll likely add more options to this script. This script simply masks all the options that need to be provided to `docker`.

0 comments on commit 1ff7bf7

Please sign in to comment.