Skip to content

Commit

Permalink
Merge pull request #8 from iwilltry42/feature/inject-init-containers
Browse files Browse the repository at this point in the history
Feature/inject init containers
  • Loading branch information
byxorna authored Mar 11, 2019
2 parents 322bd45 + 947330e commit e12e5e6
Show file tree
Hide file tree
Showing 10 changed files with 316 additions and 273 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Uses MutatingAdmissionWebhook in Kubernetes to inject sidecars into new deployme

At Tumblr, we run some containers that have complicated sidecar setups. A kubernetes pod may run 5+ other containers, with some associated volumes and environment variables. It became clear quickly that keeping these sidecars in line would become an operational hassle; making sure every service uses the correct version of each dependency, updating global environment variable sets as configurations in our DCs change, etc.

To help solve this, we wrote the `k8s-sidecar-injector`. It is a small service that runs in each Kubernetes cluster, and listens to the Kubernetes API via webhooks. For each pod creation, the injector gets a (mutating admission) webhook, asking whether or not to allow the pod launch, and if allowed, what changes we would like to make to it. For pods that have special annotations on them (`injector.tumblr.com/request=some-sidecar-name`), we rewrite the pod configuration to include the containers, volumes, and environment variables defined in the sidecar `some-sidecar-name`'s configuration.
To help solve this, we wrote the `k8s-sidecar-injector`. It is a small service that runs in each Kubernetes cluster, and listens to the Kubernetes API via webhooks. For each pod creation, the injector gets a (mutating admission) webhook, asking whether or not to allow the pod launch, and if allowed, what changes we would like to make to it. For pods that have special annotations on them (`injector.tumblr.com/request=some-sidecar-name`), we rewrite the pod configuration to include the containers, volumes, volume mounts, host aliases, init-containers and environment variables defined in the sidecar `some-sidecar-name`'s configuration.

This enabled us to keep sane, centralized configuration for oft-used, but infrequently cared about configuration for our sidecars.

Expand Down
10 changes: 9 additions & 1 deletion docs/sidecar-configuration-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ env:
volumeMounts:
- name: some-config
mountPath: /etc/some-config

# initContainers will be added, no replacement of existing initContainers with the same names will be done
# this works exactly the same way like adding normal containers does: if you have a conflicting name,
# the server will return an error
initContainers:
- name: some-initcontainer
image: init:1.12.2
imagePullPolicy: IfNotPresent
```
## Configuring new sidecars
Expand All @@ -67,6 +75,6 @@ In order for the injector to know about a sidecar configuration, you need to eit
1. Create a new InjectionConfiguration `yaml`
1. Specify your `name:`. This is what you will request with `injector.tumblr.com/request=$name`
2. Fill in the `containers`, `volumes`, `volumeMounts`, `hostAliases` and `env` fields with your configuration you want injected
2. Fill in the `containers`, `volumes`, `volumeMounts`, `hostAliases`, `initContainers` and `env` fields with your configuration you want injected
2. Either bake your yaml into your Docker image you run (in `--config-directory=conf/`), or configure it as a ConfigMap in your k8s cluster. See [/docs/configmaps.md](/docs/configmaps.md) for information on how to configure a ConfigMap.
3. Deploy a pod with annotation `injector.tumblr.com/request=$name`!
15 changes: 8 additions & 7 deletions internal/pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,13 @@ var (

// InjectionConfig is a specific instance of a injected config, for a given annotation
type InjectionConfig struct {
Name string `json:"name"`
Containers []corev1.Container `json:"containers"`
Volumes []corev1.Volume `json:"volumes"`
Environment []corev1.EnvVar `json:"env"`
VolumeMounts []corev1.VolumeMount `json:"volumeMounts"`
HostAliases []corev1.HostAlias `json:"hostAliases"`
Name string `json:"name"`
Containers []corev1.Container `json:"containers"`
Volumes []corev1.Volume `json:"volumes"`
Environment []corev1.EnvVar `json:"env"`
VolumeMounts []corev1.VolumeMount `json:"volumeMounts"`
HostAliases []corev1.HostAlias `json:"hostAliases"`
InitContainers []corev1.Container `json:"initContainers"`
}

// Config is a struct indicating how a given injection should be configured
Expand All @@ -47,7 +48,7 @@ type Config struct {

// String returns a string representation of the config
func (c *InjectionConfig) String() string {
return fmt.Sprintf("%s: %d containers, %d volumes, %d environment vars, %d volume mounts, %d host aliases", c.Name, len(c.Containers), len(c.Volumes), len(c.Environment), len(c.VolumeMounts), len(c.HostAliases))
return fmt.Sprintf("%s: %d containers, %d init containers, %d volumes, %d environment vars, %d volume mounts, %d host aliases", c.Name, len(c.Containers), len(c.InitContainers), len(c.Volumes), len(c.Environment), len(c.VolumeMounts), len(c.HostAliases))
}

// ReplaceInjectionConfigs will take a list of new InjectionConfigs, and replace the current configuration with them.
Expand Down
Loading

0 comments on commit e12e5e6

Please sign in to comment.