Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

batch_exec: inject docker container registry as a prefix #945

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

batch_exec: inject docker container registry as a prefix #945

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0834ccd
batch_exec: inject docker container registry as a prefix
danieldides Mar 1, 2023
fbb47f9
Remove empty line
danieldides Mar 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 23 additions & 6 deletions cmd/src/batch_exec.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"io"
"os"
"path/filepath"
"strings"
"time"

"github.com/sourcegraph/src-cli/internal/batches/docker"
Expand Down Expand Up @@ -174,6 +175,28 @@ func executeBatchSpecInWorkspaces(ctx context.Context, flags *executorModeFlags)
return errors.New("invalid execution, no steps to process")
}

// Pass the os.Environ to run steps to allow access to the secrets set
// in the executor environment.
// The executor runtime takes care of not forwarding any sensitive secrets
// from the host, so this is safe.
globalEnv := os.Environ()

// If the docker prefix registry is set, inject it into each step's container
// to fully qualify the container name
environ := map[string]string{}
for _, e := range globalEnv {
pair := strings.SplitN(e, "=", 2)
environ[pair[0]] = pair[1]
}

dockerPrefix, ok := environ["DOCKER_PREFIX_REGISTRY_URL"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we instead make this a flag that the server passes? Or do we want to configure this per-executor?
That way we wouldn't need to parse the env here.

if ok {
for i := 0; i < len(task.Steps); i++ {
step := &task.Steps[i]
step.Container = fmt.Sprintf("%s/%s", dockerPrefix, step.Container)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the step might change the cache key, which which would case a mismatch between backend and src-cli 🤔 I'd need to validate that, but if that's the case we should only change it when we actually use the images, not on the step element itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to validate that yourself, you'd want to run a spec with 2 steps, execute, change the 2nd step command or whatever, and see if a re-execution would hit a cache result for step1 and then if src-cli correctly sees it and skips step 1 in execution.

}
}

imageCache := docker.NewImageCache()

ui.PreparingContainerImages()
Expand All @@ -194,12 +217,6 @@ func executeBatchSpecInWorkspaces(ctx context.Context, flags *executorModeFlags)
taskExecUI.Start([]*executor.Task{task})
taskExecUI.TaskStarted(task)

// Pass the os.Environ to run steps to allow access to the secrets set
// in the executor environment.
// The executor runtime takes care of not forwarding any sensitive secrets
// from the host, so this is safe.
globalEnv := os.Environ()

opts := &executor.RunStepsOpts{
Logger: &log.NoopTaskLogger{},
WC: workspace.NewExecutorWorkspaceCreator(tempDir, repoDir),
Expand Down