Support ubuntu jammy, noble, and oracular with their appropriate qt

[adamlamar]

Adds package profiles for various versions of ubuntu - jammy, noble, oracular, and uses their appropriate versions of Qt.

When specifying packages, you can now use the LSB codename in the PACKAGE_NAME_MAPPERS dictionary, giving more control about whether (say) a qt package should be installed as version 5 or 6. For example ubuntu_jammy_native_dyn and ubuntu_noble_native_dyn are now checked to find the package list. Similarly debian_bookworm_native_dyn could be used, but I didn't change any existing package lists.

[kelson42]

@adamlamar Thank you for the PR. @veloman-yunkan will have a look to this PR this WE.

[veloman-yunkan]

I don't like the code duplication introduced here and in the other change in this (_get_packages()) function.

You can avoid it with the help of an auxiliary function _get_mapper_names_for_config(config):

def _get_mapper_names_for_config(config):
    host = neutralEnv("distname")
    codename = neutralEnv("codename")
    return ( f"{host}_{config}", f"{host}_{codename}_{config}" )

Then your code here will turn into:

        for config in ConfigInfo.all_running_configs.values():
            for mapper_name in _get_mapper_names_for_config(config):
                package_name_mapper = PACKAGE_NAME_MAPPERS.get(mapper_name, {})
                packages_list += package_name_mapper.get("COMMON", [])

[adamlamar]

Understood about the duplication. There aren't any unit tests so I was avoiding any refactors, but I can adjust as you requested. May need a few days before I can get to it.

[adamlamar]

This cleaned up nicely. Let me know what you think.

[kelson42]

@adamlamar I don't really understand the whole logic behind this PR... but it seems to assume that jammy will always build in Qt5 which is wrong: our CD should statically compile Kiwix Desktop on jammy (because we don't want to compile against a recent glibc) and this should be with Qt6.

It seems we have here some kind of double constraints: having at least one CI with Qt5 (to keep compatibility checking) and Jammy is the most appropriate... but our CD should use Jammy to build release linux binaries with Qt6.

To me it seems we need to have Qt5 AND Qt6 installed on the Jammy base container... and then choose the version of Qt based on the need (CI or CD).

[adamlamar]

@kelson42 This PR only provides more flexibility in the package names used for a specific distribution name and codename. This is important because 1) debian and ubuntu don't always have the same packages (I believe qt5-default is part of debian buster but not ubuntu jammy+ or debian bullseye+) or have them in various transitional states, and 2) the package names differ between qt5 and qt6.

Without this, kiwix-build is broken on all ubuntus (even jammy IIRC) because it doesn't supply the right package list. I think jammy does work if you install the right packages ahead of invoking kiwix-build, but not on a fresh install. I could be misremembering, I jumped around to a lot of different VMs when I was testing.

This PR fixes kiwix-build on the recent ubuntus. Although kiwix-desktop itself tests on jammy, noble, and oracular, kiwix-build was not working properly. And while kiwix-build tests many architectures, AFAICT there are currently no CI/CD jobs that test kiwix-build specifically on the ubuntus except for a few combinations.

We could definitely install qt5 and qt6 packages for jammy and then choose which one to use at build time. I'll work on that in a couple of days.

[adamlamar]

For the record, kiwix-build kiwix-desktop does fail on a fresh jammy install:

sudo apt-get install libqt5gui5 qtbase5-dev qt5-default qtwebengine5-dev aria2
[sudo] password for ubuntu:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package qt5-default is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'qt5-default' has no installation candidate
Traceback (most recent call last):
  File "/home/ubuntu/kiwix-build/bin/kiwix-build", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/kiwix-build/lib/python3.10/site-packages/kiwixbuild/__init__.py", line 166, in main
    builder.run()
  File "/home/ubuntu/kiwix-build/lib/python3.10/site-packages/kiwixbuild/builder.py", line 195, in run
    self.install_packages()
  File "/home/ubuntu/kiwix-build/lib/python3.10/site-packages/kiwixbuild/builder.py", line 185, in install_packages
    subprocess.check_call(command, shell=True)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'sudo apt-get install libqt5gui5 qtbase5-dev qt5-default qtwebengine5-dev aria2' returned non-zero exit status 100.

[kelson42]

@kelson42 This PR only provides more flexibility in the package names used for a specific distribution name and codename. This is important because 1) debian and ubuntu don't always have the same packages (I believe qt5-default is part of debian buster but not ubuntu jammy+ or debian bullseye+) or have them in various transitional states, and 2) the package names differ between qt5 and qt6.

Without this, kiwix-build is broken on all ubuntus (even jammy IIRC) because it doesn't supply the right package list. I think jammy does work if you install the right packages ahead of invoking kiwix-build, but not on a fresh install. I could be misremembering, I jumped around to a lot of different VMs when I was testing.

Thank you for the clarification, sounds obvious to me now :)

This PR fixes kiwix-build on the recent ubuntus. Although kiwix-desktop itself tests on jammy, noble, and oracular, kiwix-build was not working properly. And while kiwix-build tests many architectures, AFAICT there are currently no CI/CD jobs that test kiwix-build specifically on the ubuntus except for a few combinations.

We could definitely install qt5 and qt6 packages for jammy and then choose which one to use at build time. I'll work on that in a couple of days.

Thank you, even if this is something I could do. Should pretty trivial. Might do straight a PR. Edit: See kiwix/container-images#273

[kelson42]

@adamlamar Out of curiosity, have you run kiwix-build on Fedora (34?)? I ask because of #504, maybe we have fixed this issue as well already?

[adamlamar]

@kelson42 I updated packages.py to use qt6 for jammy. I think this is close to what you want, maybe it would be easier for you to adjust after merge if you want something different?

I haven't tested on Fedora. Since Fedora 34 is EOL maybe we should target Fedora 40 or 41?

[veloman-yunkan]

The second commit combines two unrelated changes. The ideal structure for this PR is as follows:

1. Refactor the old code by introducing the _get_mapper_names_for_config() (that returns a single-element (f"{host}_{config}",) tuple)
2. Add support for ubuntu jammy, noble, and oracular with their appropriate qt (enhance the _get_mapper_names_for_config() method here)
3. Use qt6 by default on jammy