Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dat feat awsmp sleek rebase #193

Draft
wants to merge 85 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
e7990ea
feat: new chart for aws marketplace product. The product supports lic…
Sep 6, 2022
6397cdd
fix: qoute productSKU value; add logs for checkout license
Sep 6, 2022
b658876
refactor: upgrade values.yaml with updated image pushed to aws mp pro…
Sep 6, 2022
0640270
fix: support region, product, fingerprint and debug env variables; re…
Sep 7, 2022
fdc41c2
fix: run license cron job on @daily. fix values naming bug
Sep 7, 2022
8ebf020
fix: upgrade image in aws mp helm chart values
Sep 7, 2022
17e6251
fix: remove awsmp-... chart. datree-admission-webhook chart should ho…
Sep 12, 2022
e0a348b
fix: add * to ignore luanch.json file
Sep 12, 2022
5289408
fix: remove ** to ignore luanch.json file
Sep 12, 2022
275e001
fix: accidently deleted gh-pages values file with awsmp values file. …
Sep 21, 2022
eeaa96b
fix: added aws.values
Sep 21, 2022
5e73163
feat: added validation for value.yaml file
Sep 12, 2022
2abeacc
fix: combine charts and added values.yaml for each option
Sep 19, 2022
cc12656
feat: script for packaing
Sep 19, 2022
2ccc1f3
fix: added script package for free-datree
Sep 19, 2022
595a75e
feat: added chart
Sep 20, 2022
42fbd5e
feat: charts file structure
Sep 20, 2022
f896fba
fix: combine charts and added values.yaml for each option
Sep 19, 2022
6489875
feat: script for packaing
Sep 19, 2022
150fba7
fix: added script package for free-datree
Sep 19, 2022
002513b
feat: added chart
Sep 20, 2022
f650e64
fix: move tempaltes to lib template
shmu3l Sep 28, 2022
448011f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
b43ef0f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
771c57a
fix: add dynamic webhook server alt name to cert
shmu3l Oct 2, 2022
2002a8d
fix: clear datree webhook values and update schema
shmu3l Oct 2, 2022
9b446e7
feat: awsmp datree use datree lib
shmu3l Oct 2, 2022
af85f57
fix: awsmp file structure
shmu3l Oct 2, 2022
d1ef809
feat: add release admission webhook script
shmu3l Oct 3, 2022
121763b
fix: release admission webhook
shmu3l Oct 3, 2022
37a828d
release chart 0.1.3
shmu3l Oct 3, 2022
8ddb66f
fix: release admission webhook helm script update
shmu3l Oct 6, 2022
23a42d7
fix: update .gitignore
Oct 6, 2022
30c96e4
fix: change messages
Oct 6, 2022
572c8c5
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
a13fabc
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
57a809a
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
b79e351
fix: rename script
Oct 6, 2022
edfa70f
fix: change the order of helm file creations
Oct 6, 2022
88d5416
fix: remove files
Oct 6, 2022
7e60bf9
fix: release file index yaml
Oct 6, 2022
2d54188
fix: dont stash pop
Oct 6, 2022
56e619f
feat: certificate alternative names use datree.namespace template.
Oct 18, 2022
c02ba66
fix: bump lib chart version
Oct 18, 2022
7132337
fix: added dynamic namespace installation. Rename chart according to …
Oct 24, 2022
883f25c
fix: aligned charts with aws-marketplace ECR repositories.
Nov 3, 2022
d4c0867
feat: pull bitnami-kubectl from private ecr registry
Nov 3, 2022
a09986d
fix: updated bitnami in free offer chart
Nov 3, 2022
378ef6a
feat: added new binary in cmd folder for init container
Nov 7, 2022
a5e0e21
wip
Nov 8, 2022
7d9411c
wip: poc of webhook race condition
Nov 9, 2022
076b597
fix: working product on minikube. NOTE: very slow
Nov 10, 2022
d4cd131
fix: working version
Nov 13, 2022
a1d8ba3
fix: wokring production version on minikube
Nov 13, 2022
4e504da
fix: working production version. Minikube. 3minutes sleep time
Nov 13, 2022
956b584
fix: working version ECR repositories on Faragate.
Nov 13, 2022
3cfe527
feat: support uninstall Datree product
Nov 13, 2022
d2eab5f
fix: working version in AWS Marketplace. Datree product
Nov 13, 2022
58e430c
feat: working version. Datree Free product. version 1.0.1-rc.1
Nov 13, 2022
7ba108f
feat: check for webhook existence, wait for running pods. Refactor
Nov 15, 2022
1f1d134
fix: wait for all deployment replicas to be ready. updated ecr images
Nov 15, 2022
07f0a82
fix: update aws mp Chart
Nov 16, 2022
d51dfde
fix: remove .vscode luanch.json
Nov 16, 2022
cb90d7b
fix: update Dockerfiles
Nov 16, 2022
c3a79b2
fix: remove comments
Nov 16, 2022
09fc506
test: added tests for cert-generator
Nov 16, 2022
f5593a7
test: wip
Nov 20, 2022
4ccf30e
test: fixed table testing for k8sClient
Nov 20, 2022
8161364
test: cert-generator tests
Nov 21, 2022
1e4697b
fix: structure packages in webhook-init to not use one another, only …
Nov 21, 2022
a550115
fix: updated makefile
Nov 21, 2022
52ce8b7
test: ensure empty env variables has defaults
Nov 21, 2022
4b57836
fix: bump images versions, working product on minikube.
Nov 21, 2022
f893b31
fix: make code build succesfully
Nov 21, 2022
b3867fe
fix: fix import position to remove changes
Nov 21, 2022
93c7cd2
fix: remove unused script
Nov 21, 2022
5f8e5ee
fix: remove unnessecary changes in gitignore
Nov 21, 2022
c0a13b7
fix: remove luanch.json
Nov 21, 2022
55bacc5
fix: merge manifests conflicts
Nov 22, 2022
e8a9f87
fix: user logger correctly
Nov 22, 2022
3dbba7d
fix: pass env var for webhook pods selector
Nov 22, 2022
65fe7f0
test: test main flow of cer-generator
Nov 22, 2022
8ecc47c
fix: added deploy in makefile
Nov 27, 2022
7f36170
fix: bump images version in chart
Nov 27, 2022
9938af2
fix Makefile to deploy all binaries for awsmp. Ensure TOKEN isnt requ…
Nov 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions Dockerfile

This file was deleted.

152 changes: 127 additions & 25 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,33 +1,135 @@
start-watch:
gow run -tags $(or $(datree_build_env),staging) -ldflags="-X github.com/datreeio/admission-webhook-datree/pkg/config.WebhookVersion=0.0.1" main.go

start:
go run -tags $(or $(datree_build_env),staging) -ldflags="-X github.com/datreeio/admission-webhook-datree/pkg/config.WebhookVersion=0.0.1" main.go
start-dev:
make datree_build_env=dev start
start-staging:
make datree_build_env=staging start
start-production:
make datree_build_env=main start

build:
go build -tags $(or $(datree_build_env),staging) -ldflags="-X github.com/datreeio/admission-webhook-datree/pkg/config.WebhookVersion=0.0.1" -o webhook-datree
build-dev:
make datree_build_env=dev build
build-staging:
make datree_build_env=staging build
build-production:
make datree_build_env=main build

test:
DATREE_ENFORCE="true" go test ./...
#################
# DEFAULTS #
#################

deploy-in-minikube:
bash ./scripts/deploy-in-minikube.sh
CMD_DIR := ./cmd
INIT_WEBHOOK_DIR := $(CMD_DIR)/init-webhook
CERT_GENERATOR_DIR := $(CMD_DIR)/cert-generator
WEBHOOK_SERVER_DIR := $(CMD_DIR)/webhook-server
WEBHOOK_VERSION := 0.0.1
LD_FLAGS := "-X github.com/datreeio/admission-webhook-datree/pkg/config.WebhookVersion=$(WEBHOOK_VERSION)"
BUILD_ARGS_ENV ?= staging
BUILD_ARGS_DIR ?= $(WEBHOOK_SERVER_DIR)
BUILD_ARGS_OUTPUT ?= webhook-server


#################
# RUN #
#################

_runner:
go run -tags ${BUILD_ARGS_ENV} -ldflags=$(LD_FLAGS) $(BUILD_ARGS_DIR)

run-cert-generator-%:
$(MAKE) _runner \
-e BUILD_ARGS_DIR=$(CERT_GENERATOR_DIR) \
-e BUILD_ARGS_ENV="$*"

run-init-webhook-%:
$(MAKE) _runner \
-e BUILD_ARGS_DIR=$(INIT_WEBHOOK_DIR) \
-e BUILD_ARGS_ENV="$*"

run-webhook-server-%:
$(MAKE) _runner \
-e BUILD_ARGS_DIR=$(WEBHOOK_SERVER_DIR) \
-e BUILD_ARGS_ENV="$*"

.PHONY: run-in-minikube
run-in-minikube:
bash ./scripts/run-in-minikube.sh

#################
# TEST #
#################

test:
DATREE_ENFORCE="true" go test ./...

.PHONY: test-in-minikube
test-in-minikube:
bash ./scripts/test-in-minikube.sh

##################
# BUILD #
##################
_builder:
docker build -t ${BUILD_ARGS_OUTPUT} -f $(BUILD_ARGS_DIR)/Dockerfile . --build-arg BUILD_ENVIRONMENT=${BUILD_ARGS_ENV}

build-cert-generator-%:
$(MAKE) _builder \
-e BUILD_ARGS_DIR=$(CERT_GENERATOR_DIR) \
-e BUILD_ARGS_ENV="$*" \
-e BUILD_ARGS_OUTPUT="cert-generator"

build-init-webhook-%:
$(MAKE) _builder \
-e BUILD_ARGS_DIR=$(INIT_WEBHOOK_DIR) \
-e BUILD_ARGS_ENV="$*" \
-e BUILD_ARGS_OUTPUT="init-webhook"

build-webhook-server-%:
$(MAKE) _builder \
-e BUILD_ARGS_DIR=$(WEBHOOK_SERVER_DIR) \
-e BUILD_ARGS_ENV="$*" \
-e BUILD_ARGS_OUTPUT="webhook-server"

#################
# DEPLOY #
#################

.PHONY: deploy-in-minikube
deploy-in-minikube:
bash ./scripts/deploy-in-minikube.sh


deploy-webhook-server:
$(eval IMAGE_TAG := $(shell yq '.image.tag' ./charts/datree-admission-webhook-awsmp/values.yaml | awk -F. '{$$NF = $$NF + 1;} 1' | sed 's/ /./g'))
$(MAKE) build-webhook-server-awsmp
docker tag webhook-server 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-admission-webhook:$(IMAGE_TAG)
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
docker push 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-admission-webhook:${IMAGE_TAG}
tag=${IMAGE_TAG} yq e --inplace '.image."tag" |= strenv(tag)' ./charts/datree-admission-webhook-awsmp/values.yaml


deploy-init-webhook:
$(eval IMAGE_TAG := $(shell yq '.imageWebhook.tag' ./charts/datree-admission-webhook-awsmp/values.yaml | awk -F. '{$$NF = $$NF + 1;} 1' | sed 's/ /./g'))
$(MAKE) build-init-webhook-awsmp
docker tag init-webhook 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/init-webhook:$(IMAGE_TAG)
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
docker push 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/init-webhook:${IMAGE_TAG}
tag=${IMAGE_TAG} yq e --inplace '.imageWebhook."tag" |= strenv(tag)' ./charts/datree-admission-webhook-awsmp/values.yaml

deploy-cert-generator:
$(eval IMAGE_TAG := $(shell yq '.initContainer.tag' ./charts/datree-admission-webhook-awsmp/values.yaml | awk -F. '{$$NF = $$NF + 1;} 1' | sed 's/ /./g'))
$(MAKE) build-cert-generator-awsmp
docker tag cert-generator 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/cert-generator:$(IMAGE_TAG)
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
docker push 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/cert-generator:${IMAGE_TAG}
tag=${IMAGE_TAG} yq e --inplace '.initContainer."tag" |= strenv(tag)' ./charts/datree-admission-webhook-awsmp/values.yaml

deploy-datree-awsmp:
$(MAKE) deploy-cert-generator
$(MAKE) deploy-init-webhook
$(MAKE) deploy-webhook-server

# verify that the chart is valid
helm lint ./charts/datree-admission-webhook-awsmp

# bump Helm Chart version
$(eval HELM_CHART_VERSION=$(shell yq '.version' ./charts/datree-admission-webhook-awsmp/Chart.yaml | awk -F. '{$$NF = $$NF + 1;} 1' | sed 's/ /./g'))
version=${HELM_CHART_VERSION} yq e --inplace '."version" |= strenv(version)' ./charts/datree-admission-webhook-awsmp/Chart.yaml

# helm push chart to ECR
helm package ./charts/datree-admission-webhook-awsmp
aws ecr get-login-password --region us-east-1 | helm login --username AWS --password-stdin 000000000000.dkr.ecr.us-east-1.amazonaws.com
helm push datree-admission-webhook-awsmp-${HELM_CHART_VERSION}.tgz 000000000000.dkr.ecr.us-east-1.amazonaws.com


#################
# HELM #
#################

helm-install-local-in-minikube:
eval $(minikube docker-env) && \
Expand All @@ -42,4 +144,4 @@ helm-uninstall:

helm-install-staging:
helm install -n datree datree-webhook ./charts/datree-admission-webhook --set datree.token="${DATREE_TOKEN}" --set scan_job.image.repository="datree/scan-job-staging" \
--set scan_job.image.tag="latest" --set image.repository="datree/webhook-staging" --set image.tag="latest"
--set scan_job.image.tag="latest" --set image.repository="datree/webhook-staging" --set image.tag="latest"
6 changes: 6 additions & 0 deletions charts/datree-admission-webhook-awsmp/Chart.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dependencies:
- name: datree-lib
repository: file://../datree-lib
version: 0.1.1
digest: sha256:a65b157f4e3e466bfc9a80df08a319a3f61abdb84e4a29dfbe18127dd11edac6
generated: "2022-10-18T15:06:55.309464+03:00"
31 changes: 31 additions & 0 deletions charts/datree-admission-webhook-awsmp/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
apiVersion: v2
name: datree-admission-webhook-awsmp
description: A Helm chart for Datree admission webhook for Kubernetes clusters
icon: https://github.com/datreeio/admission-webhook-datree/blob/main/internal/images/diagram.png
type: application
keywords:
- datree-admission-webhook
- policy agent
- validating webhook
- admissions controller
home: datree.io
sources:
- https://github.com/datreeio/admission-webhook-datree

kubeVersion: ">=1.16.0-0"

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 1.0.2-rc.1

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "0.1.24-rc.2"

dependencies:
- name: datree-lib
version: 0.1.1
repository: file://../datree-lib
Binary file not shown.
25 changes: 25 additions & 0 deletions charts/datree-admission-webhook-awsmp/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{{/* Create chart name and version as used by the chart label. */}}
{{- define "datree.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}


{{/* Helm and Kubernetes required labels */}}
{{- define "datree.labels" -}}
app.kubernetes.io/name: {{.Chart.Name}}
app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
app.kubernetes.io/version: {{ .Chart.AppVersion }}
app.kubernetes.io/part-of: "datree"
meta.helm.sh/release-name: "{{ .Chart.Name }}"
meta.helm.sh/release-namespace: "{{ .Release.Namespace}}"
helm.sh/chart: {{ template "datree.chart" . }}
{{- if .Values.customLabels -}}
{{ toYaml .Values.customLabels }}
{{- end -}}
{{- end -}}

{{/* The namespace name. */}}
{{- define "datree.namespace" -}}
{{- default .Release.Namespace .Values.namespace -}}
{{- end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
apiVersion: batch/v1
kind: Job
metadata:
name: datree-cleanup-namespaces-hook-pre-delete
labels: {{include "datree.labels" . | nindent 4}}
namespace: {{template "datree.namespace" .}}
annotations:
"helm.sh/hook": pre-delete, pre-upgrade
"helm.sh/hook-delete-policy": hook-succeeded, hook-failed
{{- if .Values.customAnnotations }}
{{- toYaml .Values.customAnnotations }}
{{- end }}
spec:
template:
metadata:
labels: {{include "datree.labels" . | nindent 8}}
{{- if .Values.customAnnotations }}
annotations: {{- toYaml .Values.customAnnotations | nindent 4 }}
{{- end }}
spec:
restartPolicy: OnFailure
serviceAccount: datree-cleanup-namespaces-hook-pre-delete
nodeSelector:
kubernetes.io/os: linux
containers:
- name: kubectl-label
image: "{{ .Values.hooks.image.repository }}@{{ .Values.hooks.image.sha }}"
imagePullPolicy: {{.Values.hooks.image.pullPolicy}}
command:
- sh
- "-c"
- >-
kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io datree-webhook -n {{template "datree.namespace" .}};
kubectl delete deployment datree-webhook-server -n {{template "datree.namespace" .}};
kubectl label ns kube-system {{template "datree.namespace" .}} datree.io/skip-;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{- include "datree-lib.clusterrole" .}}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{- include "datree-lib.clusterrolebinding" .}}
Loading