Help getting Oracle Linux guest running

[jackdouglas]

I use firecracker for many of the database backends on db<>fiddle, which has been working fine for years. I can run Debian and even Alpine guests but I've always had trouble with RHEL clones like Oracle Linux.

They boot ok but don't let me log in - with this dockerfile I get stuck without a tty:

FROM mysql:8.4-oraclelinux9
RUN microdnf install -y systemd
RUN echo '[Service]' > /etc/systemd/system/fiddle.service \
  && echo 'ExecStart=/fiddle.sh' >> /etc/systemd/system/fiddle.service \
  && echo '[Install]' >> /etc/systemd/system/fiddle.service \
  && echo 'WantedBy=default.target' >> /etc/systemd/system/fiddle.service \
  && systemctl enable fiddle
RUN echo '[Service]' > /etc/systemd/system/mysqld.service \
  && echo 'ExecStart=/usr/sbin/mysqld --user=root --skip-networking --daemonize' >> /etc/systemd/system/mysqld.service \
  && echo '[Install]' >> /etc/systemd/system/mysqld.service \
  && echo 'RequiredBy=fiddle.service' >> /etc/systemd/system/mysqld.service \
  && systemctl enable mysqld
RUN echo ttyS0 > /etc/securetty
RUN microdnf clean all
RUN echo "root:Docker!" | chpasswd
ENTRYPOINT ["bash"]

and today I finally got a login prompt for the first time, but logging in a 'no login shell' error with this dockerfile:

FROM mysql:8.4-oraclelinux9
RUN microdnf install -y systemd
RUN echo '[Service]' > /etc/systemd/system/fiddle.service \
  && echo 'ExecStart=/fiddle.sh' >> /etc/systemd/system/fiddle.service \
  && echo '[Install]' >> /etc/systemd/system/fiddle.service \
  && echo 'WantedBy=default.target' >> /etc/systemd/system/fiddle.service \
  && systemctl enable fiddle
RUN echo '[Service]' > /etc/systemd/system/mysqld.service \
  && echo 'ExecStart=/usr/sbin/mysqld --user=root --skip-networking --daemonize' >> /etc/systemd/system/mysqld.service \
  && echo '[Install]' >> /etc/systemd/system/mysqld.service \
  && echo 'RequiredBy=fiddle.service' >> /etc/systemd/system/mysqld.service \
  && systemctl enable mysqld
RUN echo ttyS0 > /etc/securetty \
  && echo '[Service]' > /etc/systemd/system/mygetty.service \
  && echo 'ExecStart=/usr/sbin/agetty -L 9600 ttyS0 vt102' >> /etc/systemd/system/mygetty.service \
  && echo '[Install]' >> /etc/systemd/system/mygetty.service \
  && echo 'RequiredBy=fiddle.service' >> /etc/systemd/system/mygetty.service \
  && systemctl enable mygetty
RUN microdnf clean all
RUN echo "root:Docker!" | chpasswd
ENTRYPOINT ["bash"]

in both cases I'm building the rootfs like this:

DOCKER_BUILDKIT=1 docker build -t dummy - < DOCKERFILE
dd if=/dev/zero bs=1M count=1000 > rootfs.ext4
mkfs.ext4 rootfs.ext4
mount -o loop rootfs.ext4 mnt
docker run --rm -ti -v mnt:/my-rootfs dummy
for d in bin etc home lib lib64 opt root sbin usr dev run var; do tar c "/$d" | tar x -C /my-rootfs; done
for dir in proc sys; do mkdir /my-rootfs/${dir}; done
exit

Is there something obvious I'm doing wrong? I'm really keen to add later versions of MySQL to dbfiddle but the official docker images are all Oracle Linux based since 8.0.

[pb8o]

[Converting to a discussion since it's a guest issue more than a Firecracker issue]

Hi @jackdouglas I would imagine this is some issue with the getty program or systemd. Our guest rootfs does something similar but we also seem to be only testing Debian-based and Alpine. I will look into doing this for some RHEL-based rootfs and report back if it sheds some light.

[pb8o]

So I did this for AL2023. I created a rootfs from amazonlinux:2023 similar to your approach and then it was not working by default. I think it's something with systemd/agetty where serial-getty@ttyS0 times out because dev-ttyS0.device times out. I could not figure that one out, but I followed your approach and created a systemd getty service by hand with some different options we use for Ubuntu:

cat > /etc/systemd/system/mygetty.service <<EOF2
[Service]
ExecStart=/usr/sbin/agetty --keep-baud 115200,38400,9600 ttyS0 dumb
Type=idle
Restart=always
[Install]
WantedBy=getty.target
EOF2
systemctl mask serial-getty@ttyS0.service
systemctl enable mygetty
rm -fv /etc/systemd/system/getty.target.wants/getty@tty1.service

... and this worked fine for me (making sure you configure the console in Firecracker).

...
[  OK  ] Finished systemd-update-ut… - Record Runlevel Change in UTMP.

Amazon Linux 2023.6.20241212
Kernel 6.1.102 on an x86_64 (ttyS0)

localhost login:

[jackdouglas]

I will look into doing this for some RHEL-based rootfs and report back if it sheds some light.

Thanks @pb8o. I'm selighted to say I've got Oracle Linux 9 working, at least enough to run db<>fiddle backends. My dockerfile now looks something like this:

FROM mysql:8.4-oraclelinux9
RUN microdnf install -y systemd
RUN microdnf install -y oracle-epel-release-el9
RUN microdnf install -y haveged
RUN systemctl disable getty@tty1.service \
  && echo ttyS0 > /etc/securetty \
  && echo '[Service]' > /etc/systemd/system/mygetty.service \
  && echo 'ExecStart=/usr/sbin/agetty -L 9600 ttyS0 vt102' >> /etc/systemd/system/mygetty.service \
  && echo '[Install]' >> /etc/systemd/system/mygetty.service \
  && echo 'WantedBy=default.target' >> /etc/systemd/system/mygetty.service \
  && systemctl enable mygetty
RUN microdnf clean all
RUN echo "root:Docker!" | chpasswd
RUN sed -i 's/enforcing/disabled/g' /etc/selinux/config
ENTRYPOINT ["bash"]

The only outstanding issues is that I am getting lots of log messages on the console which make debugging hard:

[   ***] A start job is running for /dev/ttyS0 (1min 9s / no limit)

If I leave them long enough the startup job times out with:

[ TIME ] Timed out waiting for device /dev/ttyS0.
[DEPEND] Dependency failed for Serial Getty on ttyS0.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.
         Starting Record Runlevel Change in UTMP...
[  OK  ] Finished Record Runlevel Change in UTMP.

[pb8o]

Great!