Docker builds with uid/gid 0 (root) #968

05nelsonm · 2023-09-10T02:35:46Z

Describe the bug
When building native binaries via make native-all, output directories in /target have root user permissions (uid 0, gid 0) resulting in build error Permission denied

To Reproduce

From linux environment, run make native-all
Check directory permissions after build fails

Expected behavior
building via dockcross should utilize the user id and group id from the shell of the user executing it such that their output matches that of the user.

e.g.

// Makefile

U_ID:="$(shell id -u)"
G_ID:="$(shell id -g)"

// For all other architectures as shown in attached image
linux64: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -u "$(U_ID):$(G_ID)" .......

Logs

mkdir: cannot create directory 'target/classes/org/sqlite/native/Linux-Musl/aarch64': Permission denied
make: *** [Makefile:125: src/main/resources/org/sqlite/native/Linux-Musl/aarch64/libsqlitejdbc.so] Error 1
make: *** [Makefile:162: linux-musl-arm64] Error 2

Environment (please complete the following information):

OS:

$ cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

CPU architecture: x86_64
sqlite-jdbc version: 3.43.0.0

Additional context

The text was updated successfully, but these errors were encountered:

gotson · 2023-09-10T04:23:08Z

If you already have a fix feel free to send a PR.

05nelsonm · 2023-09-10T10:30:16Z

freebsd32 and freebsd64 use docker image empterdose/freebsd-cross-build:9.3 and are unable to specify an unprivileged user via option -u $(U_ID):$(G_ID).

From the README it says bumping to 11.4 will allow the ability to specify a user. Thoughts on bumping it to 11.4, or would that be an issue?

These 2 builds are the only ones that are unhappy.

EDIT: empterdose docker images aren't really working b/c of the need to install bash, openjdk8 and perl, which can't be done when specifying a user (locked). Also getting errors env: can't execute bash: No such file or directory

Unsure what to do other than to build and use a different image.

gotson · 2023-09-10T13:04:43Z

@michael-o knows about freebsd much more than I do, and may advise on supporting only v11+.

It seems the empterdose image has been updated and supports aarch64 now, which would allow to drop my own fork.

gotson · 2023-09-10T13:06:52Z

empterdose docker images aren't really working b/c of the need to install bash, openjdk8 and perl, which can't be done when specifying a user (locked). Also getting errors env: can't execute bash: No such file or directory

Those should be installable with sudo, even when running the container with an unprivileged user, no?

michael-o · 2023-09-10T13:07:09Z

@gotson Let me try to understand the issue and how I can help here.

michael-o · 2023-09-10T13:08:14Z

First of all note:

I'd first update and then retry and see results. 9.3 is very old now (2016-12-31).

gotson · 2023-09-10T13:09:00Z

@05nelsonm are you able to clarify why the original issue of having directories / files owned by root is a problem?

It's been like this for some time, and never has been a problem for anyone. Is that because your build system differs in some ways?

We even build all native libs on github actions without any issues.

05nelsonm · 2023-09-10T13:22:23Z

@05nelsonm are you able to clarify why the original issue of having directories / files owned by root is a problem?

It's been like this for some time, and never has been a problem for anyone. Is that because your build system differs in some ways?

Yes I am building on laptop from non-root user (user id 1000) and kept running into build errors related to permissions resulting in the inability to copy files >> Permission denied.

Have everything building now with the proper user id:group id, except the freebsd* binaries, and the build is successful.

We even build all native libs on github actions without any issues.

Yes the container for GH actions is fine b/c it's user id:group id is 0:0 (i.e. root)

In general, running docker as root is ill-advised.

First of all note:

Update to 12.4 with clang MrDOS/freebsd-cross-build#1

Upgrade FreeBSD cross image fusesource/jansi#257

I'd first update and then retry and see results. 9.3 is very old now (2016-12-31).

Looking into it now

michael-o · 2023-09-10T13:23:08Z

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

05nelsonm · 2023-09-10T13:24:24Z

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

Yes I've tried updating to new version and adding --user $(U_ID):$(G_ID), but freebsd complains because it is unable to install dependencies (non-root user). Will try some other images.

michael-o · 2023-09-10T13:24:54Z

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

Yes I've tried updating to new version and adding --user $(U_ID):$(G_ID), but freebsd complains. Will try some other images.

What is the exact output? I bet that the source user does not exist in the container, how can it?

05nelsonm · 2023-09-10T13:27:05Z

What is the exact output? I bet that the source user does not exist in the container, how can it?

The --user ... command is a docker thing, it will automatically run using that uid:gid. The issue is with alpine linux which the freebsd containers are based on, and the inability to execute apk add <lib>. So, it exits with error as it is unable to find bash or perl from that user.

michael-o · 2023-09-10T13:31:11Z

What is the exact output? I bet that the source user does not exist in the container, how can it?

The --user ... command is a docker thing, it will automatically run using that uid:gid. The issue is with alpine linux which the freebsd containers are based on, and the inability to execute apk add <lib>. So, it exits with error as it is unable to find bash or perl from that user.

So it is not a suitable replacement for the old one at the moment or you need a layer on top with OpenJDK 8?

05nelsonm · 2023-09-10T13:40:40Z

So it is not a suitable replacement for the old one at the moment or you need a layer on top with OpenJDK 8?

Here's the output from upgrading to 11.4

with uid:gid

docker run --rm -u 1000:1000 -v $PWD:/workdir empterdose/freebsd-cross-build:11.4 sh -c 'apk add bash; apk add openjdk8; apk add perl; make clean-native native OS_NAME=FreeBSD OS_ARCH=x86 CROSS_PREFIX=i386-freebsd11-'
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
Makefile.common:4: Set JAVA_HOME environment variable
Will build using target: FreeBSD-x86 (detected os: FreeBSD arch: x86)
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
rm -rf target/sqlite-3.43.0-FreeBSD-x86
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3ext.h': Permission denied
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c': Permission denied
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c.tmp': Permission denied
make: *** [Makefile:218: clean-native] Error 1
make: *** [Makefile:150: freebsd32] Error 2

with out uid:gid

make: i386-freebsd11-gcc: No such file or directory
make: *** [Makefile:72: target/sqlite-3.43.0-FreeBSD-x86/sqlite3.o] Error 127
i386-freebsd11-gcc -o target/sqlite-3.43.0-FreeBSD-x86/sqlite3.o -c -Itarget/sqlite-3.43.0-FreeBSD-x86 -Itarget/sqlite-amalgamation-3430000 -I/include -Ilib/inc_linux -Os -fPIC -fvisibility=hidden   \
    -DSQLITE_ENABLE_LOAD_EXTENSION=1 \
    -DSQLITE_HAVE_ISNAN \
    -DHAVE_USLEEP=1 \
    -DSQLITE_ENABLE_COLUMN_METADATA \
    -DSQLITE_CORE \
    -DSQLITE_ENABLE_FTS3 \
    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
    -DSQLITE_ENABLE_FTS5 \
    -DSQLITE_ENABLE_RTREE \
    -DSQLITE_ENABLE_STAT4 \
    -DSQLITE_ENABLE_DBSTAT_VTAB \
    -DSQLITE_ENABLE_MATH_FUNCTIONS \
    -DSQLITE_THREADSAFE=1 \
    -DSQLITE_DEFAULT_MEMSTATUS=0 \
    -DSQLITE_DEFAULT_FILE_PERMISSIONS=0666 \
    -DSQLITE_MAX_VARIABLE_NUMBER=250000 \
    -DSQLITE_MAX_MMAP_SIZE=1099511627776 \
    -DSQLITE_MAX_LENGTH=2147483647 \
    -DSQLITE_MAX_COLUMN=32767 \
    -DSQLITE_MAX_SQL_LENGTH=1073741824 \
    -DSQLITE_MAX_FUNCTION_ARG=127 \
    -DSQLITE_MAX_ATTACHED=125 \
    -DSQLITE_MAX_PAGE_COUNT=4294967294 \
     \
    target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c
make: *** [Makefile:150: freebsd32] Error 2

EDIT: Yes I think a layer on top with bash, openjdk8, perl installed would do the trick.

michael-o · 2023-09-10T13:49:49Z

Let's start without uid first:
https://github.com/xerial/sqlite-jdbc/blob/3eaabec2475ca0577f025b3606a992a99f1cd318/Makefile.common#L154C1-L156
You need to override the first two to the actual command names, then JAVA_HOME isn't set, so one needs the OpenJDK inside of the container.

gotson · 2023-09-10T13:56:52Z

I don't quite understand how root on your laptop could not have the permissions to write to your own user folder.

05nelsonm · 2023-09-10T14:02:06Z

I don't quite understand how root on your laptop could not have the permissions to write to your own user folder.

It's the base files for linux x86_64. Docker builds them and outputs with root user to /target, then from my shell when preparing the next build target, am unable to copy b/c no permissions.

05nelsonm · 2023-09-10T14:23:11Z

Let's start without uid first: https://github.com/xerial/sqlite-jdbc/blob/3eaabec2475ca0577f025b3606a992a99f1cd318/Makefile.common#L154C1-L156 You need to override the first two to the actual command names, then JAVA_HOME isn't set, so one needs the OpenJDK inside of the container.

Not 100% certain on what those variables in Makefile.common need to be overridden to. Sorry, am a bit out of my depth when it comes to the native side of the house.

michael-o · 2023-09-11T07:54:06Z

Installed a AL8 VM here and trying this myself. Needs a bit of time.

05nelsonm · 2023-09-11T09:25:26Z

Installed a AL8 VM here and trying this myself. Needs a bit of time.

K.

My commit for adding a user to other builds is 05nelsonm@56b45e3

05nelsonm · 2023-09-11T09:47:43Z

Got the OP semi-resolved on freebsd32 and freebsd64 via addgroup and adduser and then using sudo (because su - doesn't work due to home directory change).

It does not work though if the U_ID or G_ID are 0 (root) b/c it will fail to add the user/group.

freebsd32: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -v $$PWD:/workdir empterdose/freebsd-cross-build:9.3 sh -c 'addgroup --gid $(G_ID) UBUILD; adduser --ingroup UBUILD --uid $(U_ID) --disabled-password --no-create-home --gecos "" UBUILD; apk add sudo; apk add bash; apk add openjdk8; apk add perl; sudo -u UBUILD make clean-native native OS_NAME=FreeBSD OS_ARCH=x86 CROSS_PREFIX=i386-freebsd9-'

freebsd64: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -v $$PWD:/workdir empterdose/freebsd-cross-build:9.3 sh -c 'addgroup --gid $(G_ID) UBUILD; adduser --ingroup UBUILD --uid $(U_ID) --disabled-password --no-create-home --gecos "" UBUILD; apk add sudo; apk add bash; apk add openjdk8; apk add perl; sudo -u UBUILD make clean-native native OS_NAME=FreeBSD OS_ARCH=x86_64 CROSS_PREFIX=x86_64-freebsd9-'

gotson · 2023-09-11T09:55:37Z

@05nelsonm what's your local setup ?

I am myself running this locally on a mac, and i have no problem. My mac user is not root.

05nelsonm · 2023-09-11T09:59:38Z

@05nelsonm what's your local setup ?

I am myself running this locally on a mac, and i have no problem. My mac user is not root.

Yeah mac is a little different than linux.

$ cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

$ id -un
administrator
$ id -u
1000
$ id -g
1000
$ cat /etc/group | grep "administrator"
sudo:x:27:administrator
administrator:x:1000:
libvirt:x:131:administrator
docker:x:998:administrator

Cloned the repository into my project's directory and ran make native-all which initially failed b/c output was permissioned for root (0:0).

05nelsonm · 2023-09-11T13:47:51Z

So, with the exception of freebsd* target permissions, everything builds for me now via make native-all package test without error and produces a runnable .jar file.

Makefile modifications >> 05nelsonm@56b45e3

FreeBSD permissions

Would it be advisable to submit a PR and then handle the freebsd complications at a later date in time?

michael-o · 2023-09-11T14:25:52Z

I cannot properly link with the 11.4 image because of:

[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:11.4 which i386-freebsd11-strip

[osipovmi@al8 sqlite-jdbc]$
[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:9.3 which i386-freebsd9-strip
/usr/local/bin/i386-freebsd9-strip

The prefixes strip command is missing.

michael-o · 2023-09-11T14:40:13Z

See also: MrDOS/freebsd-cross-build#1 (comment)

05nelsonm added the triage label Sep 10, 2023

gotson added enhancement:build Enhancement specific to the build process and removed triage labels Sep 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker builds with uid/gid 0 (root) #968

Docker builds with uid/gid 0 (root) #968

05nelsonm commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 •

edited

Loading

gotson commented Sep 10, 2023

gotson commented Sep 10, 2023

michael-o commented Sep 10, 2023

michael-o commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 •

edited

Loading

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 •

edited

Loading

michael-o commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 11, 2023

05nelsonm commented Sep 11, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading

gotson commented Sep 11, 2023

05nelsonm commented Sep 11, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading

michael-o commented Sep 11, 2023

michael-o commented Sep 11, 2023

Docker builds with uid/gid 0 (root) #968

Docker builds with uid/gid 0 (root) #968

Comments

05nelsonm commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 • edited Loading

gotson commented Sep 10, 2023

gotson commented Sep 10, 2023

michael-o commented Sep 10, 2023

michael-o commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 • edited Loading

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 10, 2023

05nelsonm commented Sep 10, 2023 • edited Loading

michael-o commented Sep 10, 2023

gotson commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

05nelsonm commented Sep 10, 2023

michael-o commented Sep 11, 2023

05nelsonm commented Sep 11, 2023 • edited Loading

05nelsonm commented Sep 11, 2023 • edited Loading

gotson commented Sep 11, 2023

05nelsonm commented Sep 11, 2023 • edited Loading

05nelsonm commented Sep 11, 2023 • edited Loading

michael-o commented Sep 11, 2023

michael-o commented Sep 11, 2023

05nelsonm commented Sep 10, 2023 •

edited

Loading

05nelsonm commented Sep 10, 2023 •

edited

Loading

05nelsonm commented Sep 10, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading

05nelsonm commented Sep 11, 2023 •

edited

Loading