Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for toolbox containers #3107

Open
afbjorklund opened this issue Jan 11, 2025 · 6 comments
Open

Support for toolbox containers #3107

afbjorklund opened this issue Jan 11, 2025 · 6 comments
Labels
enhancement New feature or request

Comments

@afbjorklund
Copy link
Member

afbjorklund commented Jan 11, 2025

Description

I was looking into support for "toolbox containers", both the original root variants and the more recent rootless one:

https://github.com/coreos/toolbox/tree/0.0.9 (systemd)

https://github.com/coreos/toolbox (switched to podman)

https://github.com/flatcar/toolbox (went back to docker)

https://containertoolbx.org/ (rewritten, from Bash to Go)

Unfortunately the more recent toolbox requires podman, which in turn almost requires fedora (or get an old podman).

There is support for Ubuntu distribution (and a podman-toolbox package), and there is support for Podman in Ubuntu.

But it is not possible to use another container engine such as systemd or nerdctl, since this is not a goal of the project...

anders@lima-default:~$ toolbox
Command 'toolbox' not found, but can be installed with:
apt install podman-toolbox
Please ask your administrator.
anders@lima-default:~$ sudo apt install podman-toolbox
Installing:                     
  podman-toolbox

Installing dependencies:
aardvark-dns libabsl20230802 libjbig0 libxcb-render0
adwaita-icon-theme libasound2-data libjpeg-turbo8 libxcb-shm0
alsa-topology-conf libasound2t64 libjpeg8 libxcomposite1
alsa-ucm-conf libatk-bridge2.0-0t64 liblcms2-2 libxcursor1
at-spi2-common libatk1.0-0t64 liblerc4 libxdamage1
at-spi2-core libatspi2.0-0t64 libmalcontent-0-0 libxfixes3
bubblewrap libavahi-client3 libostree-1-1 libxi6
buildah libavahi-common-data libpango-1.0-0 libxinerama1
catatonit libavahi-common3 libpangocairo-1.0-0 libxrandr2
conmon libavahi-glib1 libpangoft2-1.0-0 libxrender1
containernetworking-plugins libcairo-gobject2 libpipewire-0.3-0t64 libxtst6
crun libcairo2 libpipewire-0.3-common libyajl2
dconf-gsettings-backend libcolord2 libpixman-1-0 netavark
dconf-service libcups2t64 librsvg2-2 p11-kit
desktop-file-utils libdatrie1 librsvg2-common p11-kit-modules
flatpak libdconf1 libsharpyuv0 passt
fontconfig libdeflate0 libslirp0 podman
fontconfig-config libepoxy0 libspa-0.2-modules slirp4netns
fonts-dejavu-core libfontconfig1 libsubid4 ubuntu-mono
fonts-dejavu-mono libgdk-pixbuf-2.0-0 libthai-data uidmap
fuse-overlayfs libgdk-pixbuf2.0-bin libthai0 x11-common
golang-github-containers-common libgdk-pixbuf2.0-common libtiff6 xdg-dbus-proxy
golang-github-containers-image libgraphite2-3 libwayland-client0 xdg-desktop-portal
gsettings-desktop-schemas libgtk-3-0t64 libwayland-cursor0 xdg-desktop-portal-gtk
gtk-update-icon-cache libgtk-3-bin libwayland-egl1
hicolor-icon-theme libgtk-3-common libwebp7
humanity-icon-theme libharfbuzz0b libwebrtc-audio-processing-1-3

Suggested packages:
containers-storage malcontent-gui colord liblcms2-utils docker-compose xdg-desktop-portal-gnome
libwasmedge0 alsa-utils cups-common pipewire accountsservice
avahi-daemon libasound2-plugins gvfs librsvg2-bin evince

Summary:
Upgrading: 0, Installing: 106, Removing: 0, Not Upgrading: 20
Download size: 60.9 MB
Space needed: 262 MB / 100 GB available

Continue? [Y/n] n
Abort.

anders@lima-default:~$ toolbox enter
No toolbox containers found. Create now? [y/N] y
Image required to create toolbox container.
Download quay.io/toolbx-images/ubuntu-toolbox:24.10 (500MB)? [y/N]: y
Error: failed to pull image quay.io/toolbx-images/ubuntu-toolbox:24.10
If it was a private image, log in with: podman login quay.io
Use 'toolbox --verbose ...' for further details.
anders@lima-default:~$ toolbox enter --distro fedora
Error: option '--release' is needed
Distribution fedora doesn't match the host.
Run 'toolbox --help' for usage.
anders@lima-default:~$ toolbox --version
toolbox version 0.0.99.3
anders@lima-default:~$ podman --version
podman version 5.0.3

Anyway, the idea is that you use single VM (possibly read-only or spartan) - and then run all your work in containers.

The feature of the "toolbox" is that it preps the container for you, by selecting a common image and/or creating a user.

So that would make it more similar to WSL2, and the Hyper-V VM used: https://learn.microsoft.com/en-us/windows/wsl/

Where WSL actually does not add the user for you, just suggesting that you might want to look into adding one yourself.

Like so, example from Alma: https://wiki.almalinux.org/documentation/wsl.html (creating a system container)

Compared with: https://wiki.almalinux.org/cloud/Generic-cloud-on-local.html (creating an AlmaLinux VM/"lima")

So there could some integration with the "default" template and nerdctl, to access such a container running in Lima.

To start with, I made some small updates to the old toolbox (replaced rkt with crane) and it seems to run just fine...


I think that colima started on some similar features called "layers", but that term is somewhat overloaded (overlayfs)

They are called "distros" in WSL, but that term is even worse. And there seems to be plenty of "toolbox" around as well.

But the goal would be to have some variant of lima, that would drop you in a container instead of on the machine.

It is basically just running nerdctl run -it, the trick is setting up the parameters and the mounts - and the image.

Toolbox: https://github.com/afbjorklund/systemd-toolbox

limactl start --containerd none
apt install systemd-container
install crane, install toolbox

$ lima toolbox
░ Spawning container anders-fedora-latest on /var/lib/toolbox/anders-fedora-latest.
░ Press Ctrl-] three times within 1s to kill container.
[root@lima-default ~]# 

limactl start --containerd system

$ lima sudo nerdctl run -it -v /:/run/host fedora:latest
[root@494ddcce9f1c /]#

Archive size:
56M fedora_latest.tar (nerdctl save format)
55M fedora_latest.tar.gz
43M fedora_latest.tar.zst


With rootless and Lima it would look more similar to (but also needs some more setup):

limactl start

$ nerdctl.lima run -it -u `id -u`:`id -g` -v $HOME:$HOME -w $HOME fedora:latest
docker.io/library/fedora:latest:                                                  resolved       |++++++++++++++++++++++++++++++++++++++| 
index-sha256:3ec60eb34fa1a095c0c34dd37cead9fd38afb62612d43892fcf1d3425c32bc1e:    done           |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:9cfb3a7ad0a36a1e943409def613ec495571a5683c45addb5d608c2c29bb8248: done           |++++++++++++++++++++++++++++++++++++++| 
config-sha256:aa6787b90fe61e801687142277458584287469c9596c91766a43fa9f1e524c22:   done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:0c5a86865c5d3e78a4ab19ac7c516ffe93e41e0fd67f052a72f52d07cd2c59f9:    done           |++++++++++++++++++++++++++++++++++++++| 
elapsed: 18.0s                                                                    total:  55.9 M (3.1 MiB/s)                                       
bash-5.2$ whoami
whoami: cannot find name for user ID 1000

So the user, group and home directory needs adding. To the image, or the container at init.

@afbjorklund
Copy link
Member Author

The apptainer experience is very similar to this, main difference is that it uses a squashfs image and not a directory.

But you remain as the same user, and you have access to your files and so on. Otherwise it calls it --containall.

@afbjorklund
Copy link
Member Author

The "user" image looks something like this: (adding an entry to the system, with the user and group)

https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user#_creating-a-nonroot-user

@afbjorklund
Copy link
Member Author

afbjorklund commented Jan 12, 2025

Creating a user image:

FROM ubuntu:latest
RUN userdel --remove ubuntu

RUN groupadd -g 1000 anders && useradd -u 1000 -g 1000 -d /home/anders.linux -s /bin/bash -m -N anders && rm -f /etc/.pwd.lock
WORKDIR /home/anders.linux
USER anders
CMD ["/bin/bash"]

Running user container:

$ nerdctl.lima run -it -u `id -u`:`id -g` -v $HOME:$HOME -w $HOME -e TERM anders-ubuntu-latest
anders@39ca671a1094:/home/anders$

Note: probably better to create the container in the background, and then exec into it with the shell?

It should also feature an --init process, to care of the zombies. And have a --name, like any pet would.

@afbjorklund
Copy link
Member Author

A "toolbox" would typically run as a privileged container, and use the host namespaces for network and pid.

It would also mount the entire host filesystem (somewhere like /media/root or /run/host), and add sudo.

@afbjorklund
Copy link
Member Author

afbjorklund commented Jan 16, 2025

Someone mentioned that there is also "distrobox", which is another system that is similar to toolbx...

Distrobox uses podman, docker or lilipod to create containers using the Linux distribution of your choice

Note: distrobox uses the GNU General Public License version 3, unlike toolbox which uses Apache License version 2

It is written in shell (/bin/sh), unlike toolbx which is using Go. There are lots of examples and different distributions.


EDIT: It will need some love to work with nerdctl and dash, though. It wants to use docker and /bin/sh as bash...

anders@lima-default:~$ distrobox create
Missing dependency: we need a container manager.
Please install one of podman,  docker or lilipod.
You can follow the documentation on:
	man distrobox-compatibility
or:
	https://github.com/89luca89/distrobox/blob/main/docs/compatibility.md

sudo ln -s nerdctl /usr/local/bin/docker
/usr/local/bin/distrobox-enter: 3: eval: Syntax error: ";" unexpected

The problems are with the integrated Go templates, that work with docker and podman - but not (yet) with nerdctl

@nirs nirs added the enhancement New feature or request label Jan 18, 2025
@afbjorklund
Copy link
Member Author

afbjorklund commented Jan 18, 2025

For the lima handbook

toolbx

limactl start template://podman
export LIMA_INSTANCE=podman
lima sudo dnf install -y toolbox

$ lima toolbox enter
No Toolbx containers found. Create now? [y/N] y
Image required to create Toolbx container.
Download registry.fedoraproject.org/fedora-toolbox:41 ( ... MB)? [y/N]: y
Error: directory /home/anders not found in container fedora-toolbox-41
Using /home/anders.linux instead.

Welcome to the Toolbx; a container where you can install and run
all your tools.

 - Use DNF in the usual manner to install command line tools.
 - To create a new tools container, run 'toolbox create'.

For more information, see the documentation.

⬢ [anders@toolbx ~]$ 

distrobox

limactl start --name=docker --containerd=none #needs "docker.io" deb
export LIMA_INSTANCE=docker
lima sudo apt update && lima sudo apt install -y docker.io distrobox
lima sudo usermod -aG docker $USER && ssh -F $(limactl ls --format='{{.SSHConfigFile}}' docker) lima-docker -O exit

$ lima distrobox enter
Error response from daemon: No such container: my-distrobox
Create it now, out of image registry.fedoraproject.org/fedora-toolbox:39? [Y/n]: 
Creating the container my-distrobox
39: Pulling from fedora-toolbox
40eb91ad8a1c: Pull complete 
Digest: sha256:11f3634d4a8f2d4a69c4ad8442133f69979be49fa6269eccc6ab0863c39d59d0
Status: Downloaded newer image for registry.fedoraproject.org/fedora-toolbox:39
registry.fedoraproject.org/fedora-toolbox:39
Creating 'my-distrobox' using image registry.fedoraproject.org/fedora-toolbox:39	 [ OK ]
Distrobox 'my-distrobox' successfully created.
To enter, run:

distrobox enter my-distrobox

uccessfully copied 2.56kB to /tmp/my-distrobox.os-release
Starting container...                   	 [ OK ]
Installing basic packages...            	 [ OK ]
Setting up devpts mounts...             	 [ OK ]
Setting up read-only mounts...          	 [ OK ]
Setting up read-write mounts...         	 [ OK ]
Setting up host's sockets integration...	 [ OK ]
Integrating host's themes, icons, fonts...	 [ OK ]
Setting up package manager exceptions...	 [ OK ]
Setting up rpm exceptions...            	 [ OK ]
Setting up distrobox profile...         	 [ OK ]
Setting up sudo...                      	 [ OK ]
Setting up user groups...               	 [ OK ]
Setting up kerberos integration...      	 [ OK ]
Setting up user's group list...         	 [ OK ]
Adding user...                          	 [ OK ]
Setting up user home...                 	 [ OK ]
Ensuring user's access...               	 [ OK ]

Container Setup Complete!
⚠️  First time user password setup ⚠️ 
Changing password for user anders.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
anders@lima-docker:/run/host/home/anders$ . /etc/profile.d/distrobox_profile.sh
📦[anders@my-distrobox anders]$

No lima changes need, for those to work.

EDIT: Except for the prompt being broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants