Can not boot after xpadneo installed

[zshzebra]

Version of xpadneo

0.9.5

Controller Model

• Xbox One S controller
• Xbox Elite 2 controller
• Xbox Series X|S controller
• Other:

Connection mode

• Bluetooth connection
• USB cable (not yet supported)
• Xbox Dongle connection (not yet supported)

Installed Software

• Anti-Micro (may affect button mappings)
• OpenRGB (may mess up mappings and rumble stability)
• Steam Input (enabled by default via Steam Desktop client)
• Steam Link (usually via Raspberry Pi or other micro computers)
• devices with QMK firmware (may affect udev rules, similar to OpenRGB)
• netstick (shares input devices via network similar to Steam Link)
• xboxdrv (user-space gamepad driver)
• xone (kernel-space gamepad driver using the Xbox dongle or USB)
• xow (alternative driver using the Xbox dongle)

Protocol Information

Please help us identify at which layer the problem can be found if you want
to report mapping errors or if the controller fails to be detected:

• Steam Proton games are having issues
• Steam Linux-native games are having issues
  • I don't use Steam or did not try
• games running through Lutris, wine and/or Bottles are having issues
  • I don't use Lutris, Bottles, wine or did not try
• Linux-native games are having issues
  • I don't use native games or did not try
• Other software is having issues (describe software and issues below)
• Running evtest is showing issues (describe the issues below)
  • Keep in mind that BTN_NORTH and BTN_WEST are intentionally swapped
• Running jstest is showing issues (describe the issues below)
  • I don't have this tool or don't know how to use it
• Running gamepad-tool is showing issues (post console output below)
  • I don't have this tool

Please describe how it is failing below in the next sections.

Severity / Impact

• I've read the docs and the bug reporting instructions
• I've applied the latest firmware update to the controller
• I've tried disabling or running without above mentioned software
• It does not work at all
• It used to work in a previous version
• It mostly works but sometimes it doesn't
• I found a work-around
• I probably didn't figure it all out but it's too early to give up
• I don't know how to ...
• It's too complicated
• Fantastic work but ...
• I can code and I want to help

Describe the Bug

After installing, could connect, but after reboot I boot into busybox

Steps to Reproduce

• install xpadneo
• reboot

Expected Behavior

A busybox terminal

Screenshots / GIFs / Videos

System Information

Pop! Os 23.04

# uname -a
* cannot access uname -a

# xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:*/report_descriptor | tee >(cksum)

Controller and Bluetooth Information

Additional Context

[kakra]

How did you install xpadneo? What's the screen output when the system fails to boot?

[zshzebra]

Just the grey screen you get when you load Pop! OS, where you expect the use selection menu to appear, but after 5 seconds more it goes into ubuntu busybox. There is no text displayed on the screen, just a terminal of busybox. This was after I installed by Debian instructions. I stopped following guide after install.sh, so I might have missed something.

[kakra]

This sounds like it's dropping you into an emergency shell, and it may happen because it cannot load a driver or device during boot. Maybe it tries to load xpadneo during initramfs phase but DKMS didn't incorporate it into the ram disk image. This is probably a distribution bug then, xpadneo isn't needed at that phase of boot.

Could you run dmesg in busybox to look for any obvious errors? Does the system continue to boot if you press Ctrl+D? I do not know Pop! OS but it should be possible to boot in verbose mode so you actually see text logs instead of a graphical boot screen. That may help, too.

Also, if systemd is already active at that stage, the output of journalctl -b may be helpful.

Do you use dual boot? A quick googling shows that there might be problems in dual boot setups when rebuilding initramfs.

Try if running sudo update-initramfs -c -k all after installing xpadneo but before rebooting fixes the problem. If it does, then DKMS hooks are broken on your distribution, and you should report there.

[zshzebra]

I tried your fixes, and none have worked. I will use live boot to try and get access, if not I will have to reinstall.

I also dual boot with windows 11, I have no recovery partition either.

Legit 1 month before having to probably reinstall again.

[zshzebra]

I have reinstalled my os.

[kakra]

I wonder if this would happen again if you installed xpadneo now, or if there was some other problem lurking around already.

[zshzebra]

I wonder if this would happen again if you installed xpadneo now, or if there was some other problem lurking around already.

I'll create a VM and see

[kakra]

I could imagine that rebuilding initramfs with DKMS somehow failed because of missing dependencies. If that's the problem, we should find which deps are needed and document these.

[CodyMarkix]

@kakra Author of #436 here, I'm not sure if this will be usable considering some of the stuff scroll so fast but here are recordings of dmesg in Busybox and lsinitramfs : https://imgur.com/a/wNAv6dP

[kakra]

Okay, I'm not familiar with secure boot but it looks like Pop! OS doesn't properly sign kernel and modules when installing via DKMS. Usually, DKMS is expected to have distribution hooks to properly sign installed modules. Could you try disabling secure boot if it is enabled in the BIOS?

Also, when generating the initramfs, you can run initramfs ... | tee log.txt to generate a log from screen output. This also works for dmesg|tee dmesg.txt but I'm guessing it would be difficult to store the file somewhere from busybox because probably no permanent storage is mounted.

Your video says it dropped to busybox because it timed out waiting for the rootfs. But did it really wait? Usually, it may wait several minutes for the rootfs. But this looks like it wasn't able to load the SATA chipset modules and thus immediately finished waiting for any more devices - which can be explained by module signing missing for ALL modules and drivers.

I wonder a few things here:

Why does Pop! OS put drivers for late devices into the initramfs at all? All you need there is chipset, GPU and file systems. But yeah, probably input devices, too, and xpadneo belongs to those. But actually you'd only need standard keyboard drivers - and all those should be built into the kernel so they become loaded no matter what.

Why does DKMS not properly sign modules, or the other way around: Why does initramfs accept/put unsigned modules into initramfs?

I think Pop! OS misses some important hook somewhere. We do not provide instructions to manually run the initramfs generator. Is there some document which recommends this? I'd like to setup a VM to reproduce this here because this is one of the worst experiences you could get by installing a third party driver: It must never result in an unbootable system. Either it should bail out in the first place during installation, or should still boot but refuse to load just this single driver.

[CodyMarkix]

I think module signing is out of the question because Secure Boot is disabled on my machine and Pop!_OS practically requires you to disable Secure Boot as the USB environment straight up won't boot.

Also, when generating the initramfs, you can run initramfs ... | tee log.txt to generate a log from screen output.

Do you mean when regenerating the initramfs image? I actually tried that on a whim, thinking that it might fix the issue but no dice (maybe it was built incorrectly?). It did skip directly to the rootfs timeout but that may have been either because of the fact that update-initramfs tried to run efibootmgr from a chroot environment and failed (why? I have no clue, probably because chroot assumes the mounted disk doesn't belong to the PC running chroot) or because of the fact that I temporarily removed quiet splash from kernel params when testing the regenerated initramfs image

I also thought about checking the filesystem on the off chance that a unlucky coincidence happened and the fs had problems not because of xpadneo. But running fsck from a live environment showed the rootfs was healthy.

But this looks like it wasn't able to load the SATA chipset modules and thus immediately finished waiting for any more devices - which can be explained by module signing missing for ALL modules and drivers.

I have an NVME drive, not really important but may alter some stuff down the line.

Why does DKMS not properly sign modules, or the other way around: Why does initramfs accept/put unsigned modules into initramfs?

Most likely due to the fact that ya boy has Secure Boot disabled.

About the missing hook, I did find it strange that /etc/initramfs-tools/hooks was completely empty when I was troubleshooting, but then I found /usr/share/initramfs-tools/hooks and it seemed like all the necessary hooks were in place. (Keep in mind I don't know shit about what specific hooks are necessary, I just saw a lot of hooks that seemed to relate to input and display so I assumed that all essentials were present) I would also test this out in a VM myself but I'm kinda tight on space on my Windows volume 😅

[CodyMarkix]

If anything, this just shows that you should have a seperate /home partition or at the very least have TimeShift installed or it will bite you in the butt. (Though timeshift has proven to be useless for me because it just insists that you have grub and not some other boot loader already bundled in with your init system)

[EnderActually]

I have the same issue, A fix would be very helpful.
When im in busy box i cant even input anything.

[kakra]

Are you guys able to create a small VM image where this happens?

[EnderActually]

Sure, I will work on that later

[noxware]

Pop Os 22.04 here, experiencing the same issue after running install.sh and restarting. Could start the system selecting the oldkern option in systemd-boot menu for now, but I still don't know how to fix the system properly .

[noxware]

With some aid of chatgpt I could fix the normal boot entry. I did the following.

# To find the xpadneo module.
dkms status 

# Then, removed the module, replace `hid-xpadneo/v0.9-172-gfd0cd48` with what you had in `dkms status`
sudo dkms remove hid-xpadneo/v0.9-172-gfd0cd48 --all

# Removed the files
sudo rm -rf /usr/src/hid-xpadneo-v0.9-172-gfd0cd48

# Did this thing :P
sudo update-initramfs -u -k all

After these steps, I rebooted but the normal entry was still broken so I booted into oldkern again and did:

sudo apt update && sudo apt upgrade -y

And the next restart I could log in into my system normally.

[kakra]

Is this still an issue? We may have one related issue #431. But I still think an OS should handle problems gracefully during first boot after an update, so I tend to say the distributions should generally fix this.

[Afafard]

Yep. Not reporter, but can confirm. Still an issue. I just encountered similar to the above thread sequence. Pop OS 22.04

[kakra]

Please test #517

[k-dx]

This happened to me after the fix, i.e. I cloned the commit 8d20a23, after b418e9f. PopOS 22.04 LTS.
Boot was stuck on "cryptdata: waiting for encrypted source device UUID=...", even though the device (my SSD) with this UUID was present.
Uninstalling xpadneo when chrooted then repairing the bootloader as in https://support.system76.com/articles/bootloader/ did NOT fix it.
PopOS Refresh Install option from recovery partition worked.

[kakra]

Okay, so I could test with a PopOS 22.04 LTS image in a VM then.