-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Healthcheck always shows "starting" for quadlet service #25034
Comments
Hi @Keyruu, can you share an example of such a Quadlet file? |
Hi @ygalblum, of course!
|
Thanks, I see that it refers to a Quadlet network: |
Pretty standard I think:
|
I just tried it on a Fedora 40 machine running podman 5.3.1 as well and from the looks of it, it seems to be working. I did have to be more explicit in the image:
But, from then on: $ sudo podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a696f4a34de0 docker.io/library/redis@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc redis-server 3 minutes ago Up 3 minutes (healthy) 6379/tcp immich-redis $ systemctl status immich-redis.service
● immich-redis.service - Podman container immich-redis
Loaded: loaded (/etc/containers/systemd/immich-redis.container; generated)
Drop-In: /usr/lib/systemd/system/service.d
└─10-timeout-abort.conf
Active: active (running) since Fri 2025-01-17 09:47:21 EST; 5min ago
Main PID: 224960 (conmon)
Tasks: 11 (limit: 4534)
Memory: 3.7M (peak: 19.6M)
CPU: 754ms
CGroup: /system.slice/immich-redis.service
├─libpod-payload-a696f4a34de0cd8da5277a3a745eb60e5973bb0c52711b936b74cf22aee05ecd
│ └─224962 "redis-server *:6379"
└─runtime
├─224943 /usr/libexec/podman/aardvark-dns --config /run/containers/networks/aardvark-dns -p 53 run
└─224960 /usr/bin/conmon --api-version 1 -c a696f4a34de0cd8da5277a3a745eb60e5973bb0c52711b936b74cf22aee05ecd -u a696f4a34de0cd8da5277a3a745eb60e5973bb0c52711b936b74cf22aee05ec>
Jan 17 09:47:21 lima-default immich-redis[224960]: 1:C 17 Jan 2025 14:47:21.281 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Am I missing something? |
I mean, I'm on NixOS but I don't know if that changes anything with the podman setup. |
This is the whole [
{
"Id": "20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa",
"Created": "2025-01-17T16:02:35.105166961+01:00",
"Path": "docker-entrypoint.sh",
"Args": [
"redis-server"
],
"State": {
"OciVersion": "1.2.0",
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 186999,
"ConmonPid": 186997,
"ExitCode": 0,
"Error": "",
"StartedAt": "2025-01-17T16:02:36.408050869+01:00",
"FinishedAt": "0001-01-01T00:00:00Z",
"Health": {
"Status": "starting",
"FailingStreak": 0,
"Log": null
},
"CgroupPath": "/system.slice/immich-redis.service/libpod-payload-20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa",
"CheckpointedAt": "0001-01-01T00:00:00Z",
"RestoredAt": "0001-01-01T00:00:00Z"
},
"Image": "0e8d64a9df81426ee4c67aa65212f721dfc5829d2e577a3bc6f26ae1f740fd85",
"ImageDigest": "sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc",
"ImageName": "docker.io/library/redis@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc",
"Rootfs": "",
"Pod": "",
"ResolvConfPath": "/run/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/resolv.conf",
"HostnamePath": "/run/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/hostname",
"HostsPath": "/run/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/hosts",
"StaticDir": "/var/lib/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata",
"OCIConfigPath": "/var/lib/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/config.json",
"OCIRuntime": "crun",
"ConmonPidFile": "/run/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/conmon.pid",
"PidFile": "/run/containers/storage/overlay-containers/20c2cea6d57ed04a008b8c47e6e48d6899ef689dd058d09ef1e02d93431a38fa/userdata/pidfile",
"Name": "immich-redis",
"RestartCount": 0,
"Driver": "overlay",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"EffectiveCaps": [
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYS_CHROOT"
],
"BoundingCaps": [
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_KILL",
"CAP_NET_BIND_SERVICE",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYS_CHROOT"
],
"ExecIDs": [],
"GraphDriver": {
"Name": "overlay",
"Data": {
"LowerDir": "/var/lib/containers/storage/overlay/72ee2f4a23a3808ece6d3f2fa6cf2f43a59d0b88489895b028317452bd71606d/diff:/var/lib/containers/storage/overlay/590e916d6556325b080438902f6ac115760264eb70485c86a75808a51e921b25/diff:/var/lib/containers/storage/overlay/ffb77dd9e22fafb753c6cd7ae3c9afcb321f19a60e9c04e66c4c51cb7b4d6f91/diff:/var/lib/containers/storage/overlay/482888ad13034190206919b7bb39bbbaeecedf020cd27d8e7e958ecbf4e266e3/diff:/var/lib/containers/storage/overlay/96c60cc75b4b4584d8d4d4c277d785531c805c0d55253539eb718a31fcbb1ea1/diff:/var/lib/containers/storage/overlay/dbeb705e177d78c96df409d774518bdf5b3e70a655aa9904e95e45541a0bfc00/diff:/var/lib/containers/storage/overlay/cf2c2995c89d2c0d67323e293b01b9685a03c249be2b9f517800caab49020d6b/diff:/var/lib/containers/storage/overlay/5af4f8f59b764c64c6def53f52ada809fe38d528441d08d01c206dfb3fc3b691/diff",
"MergedDir": "/var/lib/containers/storage/overlay/1321b690d6a62eaf66d4f1256930305da04028c4d990dd5251a1367c96937063/merged",
"UpperDir": "/var/lib/containers/storage/overlay/1321b690d6a62eaf66d4f1256930305da04028c4d990dd5251a1367c96937063/diff",
"WorkDir": "/var/lib/containers/storage/overlay/1321b690d6a62eaf66d4f1256930305da04028c4d990dd5251a1367c96937063/work"
}
},
"Mounts": [
{
"Type": "volume",
"Name": "00f82960fa059356479e1bd8781cfb29ab63d821c9cbc94ac3f2ce6575fcbc68",
"Source": "/var/lib/containers/storage/volumes/00f82960fa059356479e1bd8781cfb29ab63d821c9cbc94ac3f2ce6575fcbc68/_data",
"Destination": "/data",
"Driver": "local",
"Mode": "",
"Options": [
"nodev",
"exec",
"nosuid",
"rbind"
],
"RW": true,
"Propagation": "rprivate"
}
],
"Dependencies": [],
"NetworkSettings": {
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"Bridge": "",
"SandboxID": "",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"6379/tcp": null
},
"SandboxKey": "/run/netns/netns-4c66e6ed-23f4-65a8-13d4-0ae4a0fb69a6",
"Networks": {
"systemd-immich": {
"EndpointID": "",
"Gateway": "10.89.1.1",
"IPAddress": "10.89.1.140",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "5e:fb:41:f4:9a:bc",
"NetworkID": "systemd-immich",
"DriverOpts": null,
"IPAMConfig": null,
"Links": null,
"Aliases": [
"redis",
"20c2cea6d57e"
]
}
}
},
"Namespace": "",
"IsInfra": false,
"IsService": false,
"KubeExitCodePropagation": "invalid",
"lockNumber": 8,
"Config": {
"Hostname": "20c2cea6d57e",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=podman",
"GOSU_VERSION=1.17",
"REDIS_VERSION=6.2.14",
"REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-6.2.14.tar.gz",
"REDIS_DOWNLOAD_SHA=34e74856cbd66fdb3a684fb349d93961d8c7aa668b06f81fd93ff267d09bc277",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"HOME=/root",
"HOSTNAME=20c2cea6d57e"
],
"Cmd": [
"redis-server"
],
"Image": "docker.io/library/redis@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc",
"Volumes": null,
"WorkingDir": "/data",
"Entrypoint": [
"docker-entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"PODMAN_SYSTEMD_UNIT": "immich-redis.service"
},
"Annotations": {
"io.container.manager": "libpod",
"io.podman.annotations.autoremove": "TRUE",
"io.podman.annotations.cid-file": "/run/immich-redis.cid",
"org.opencontainers.image.stopSignal": "15",
"org.systemd.property.KillSignal": "15",
"org.systemd.property.TimeoutStopUSec": "uint64 10000000"
},
"StopSignal": "SIGTERM",
"Healthcheck": {
"Test": [
"CMD-SHELL",
"redis-cli ping || exit 1"
],
"Interval": 30000000000,
"Timeout": 30000000000,
"Retries": 3
},
"HealthcheckOnFailureAction": "none",
"HealthLogDestination": "local",
"HealthcheckMaxLogCount": 5,
"HealthcheckMaxLogSize": 500,
"CreateCommand": [
"/nix/store/szxc93qlzmy33si8nvwnl33qn4ypf8qd-podman-5.3.1/bin/podman",
"run",
"--name",
"immich-redis",
"--cidfile=/run/immich-redis.cid",
"--replace",
"--rm",
"--cgroups=split",
"--network-alias",
"redis",
"--network",
"systemd-immich",
"--sdnotify=conmon",
"-d",
"--health-cmd",
"redis-cli ping || exit 1",
"docker.io/library/redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc"
],
"Umask": "0022",
"Timeout": 0,
"StopTimeout": 10,
"Passwd": true,
"sdNotifyMode": "conmon",
"sdNotifySocket": "/run/systemd/notify",
"ExposedPorts": {
"6379/tcp": {}
}
},
"HostConfig": {
"Binds": [
"00f82960fa059356479e1bd8781cfb29ab63d821c9cbc94ac3f2ce6575fcbc68:/data:rprivate,rw,nodev,exec,nosuid,rbind"
],
"CgroupManager": "systemd",
"CgroupMode": "private",
"ContainerIDFile": "/run/immich-redis.cid",
"LogConfig": {
"Type": "journald",
"Config": null,
"Path": "",
"Tag": "",
"Size": "0B"
},
"NetworkMode": "bridge",
"PortBindings": {},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": true,
"AutoRemoveImage": false,
"Annotations": {
"io.container.manager": "libpod",
"io.podman.annotations.autoremove": "TRUE",
"io.podman.annotations.cid-file": "/run/immich-redis.cid",
"org.opencontainers.image.stopSignal": "15",
"org.systemd.property.KillSignal": "15",
"org.systemd.property.TimeoutStopUSec": "uint64 10000000"
},
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": [],
"CapDrop": [],
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": [],
"GroupAdd": [],
"IpcMode": "shareable",
"Cgroup": "",
"Cgroups": "default",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "private",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [],
"Tmpfs": {},
"UTSMode": "private",
"UsernsMode": "",
"ShmSize": 65536000,
"Runtime": "oci",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": 0,
"OomKillDisable": false,
"PidsLimit": 2048,
"Ulimits": [
{
"Name": "RLIMIT_NOFILE",
"Soft": 1048576,
"Hard": 1048576
},
{
"Name": "RLIMIT_NPROC",
"Soft": 1048576,
"Hard": 1048576
}
],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"CgroupConf": null
}
}
] |
Issue Description
I'm trying to setup some containers using quadlet and podman but the healthchecks are stuck at "starting" and never execute. It is basically the same thing that was discussed here #19381, but I definitely have systemd on my system. I am doing this on a NixOS machine.
It does work when I manually run
podman healthcheck run id
and it works if I run the same container just from my user withpodman run
. So it definitely is an issue with systemd.Steps to reproduce the issue
Steps to reproduce the issue
Describe the results you received
The container never reaches a
healthy
state.Describe the results you expected
A container with a healthy state.
podman info output
Podman in a container
No
Privileged Or Rootless
None
Upstream Latest Release
No
Additional environment details
NixOS on a Proxmox VM
Additional information
No response
The text was updated successfully, but these errors were encountered: