Multiple replicas in Kubernetes cause Homepage to sometimes not apply settings.yaml

[Zeashh]

Description

Hi,

I've deployed Homepage in Kubernetes with a known working config (without automatic discovery) and I've noticed odd behavior when running more than one replica (in my case 2).

On some browser sessions Homepage will, as far as I can tell, apply bookmarks.yaml, services.yaml, widgets.yaml, custom.js and custom.css, but not settings.yaml (causing the background and layout to revert to defaults, which is how I noticed it to begin with). Both Pods can read the file as well as the background image mounted through a RWX volume on NFS.

Pods are running in K3s v1.28.7+k3s1.

homepage version

v0.9.2

Installation method

Other (please describe above)

Configuration

Kubernetes configuration:

- Deployment and service:

 yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: homepage
  labels:
    app: homepage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: homepage
  template:
    metadata:
      labels:
        app: homepage
    spec:
      containers:
        - name: homepage
          image: "ghcr.io/gethomepage/homepage:latest"
          ports:
            - name: http
              containerPort: 3000
              protocol: TCP
          volumeMounts:
            - mountPath: /app/config/custom.js
              name: homepage-config
              subPath: custom.js
            - mountPath: /app/config/custom.css
              name: homepage-config
              subPath: custom.css
            - mountPath: /app/config/bookmarks.yaml
              name: homepage-config
              subPath: bookmarks.yaml
            - mountPath: /app/config/docker.yaml
              name: homepage-config
              subPath: docker.yaml
            - mountPath: /app/config/kubernetes.yaml
              name: homepage-config
              subPath: kubernetes.yaml
            - mountPath: /app/config/services.yaml
              name: homepage-config
              subPath: services.yaml
            - mountPath: /app/config/settings.yaml
              name: homepage-config
              subPath: settings.yaml
            - mountPath: /app/config/widgets.yaml
              name: homepage-config
              subPath: widgets.yaml
            - mountPath: /app/public/images
              name: homepage-images
      volumes:
        - name: homepage-config
          secret:
            secretName: homepage
        - name: homepage-images
          persistentVolumeClaim:
            claimName: homepage-images
---
apiVersion: v1
kind: Service
metadata:
  name: homepage
  labels:
    app: homepage
  annotations:
    traefik.ingress.kubernetes.io/service.sticky.cookie: "true"
spec:
  type: ClusterIP
  ports:
    - port: 3000
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app: homepage

• Ingress (homepage-redirect is just for redirecting http to https):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: homepage
  annotations:
    kubernetes.io/ingressClassName: traefik
    traefik.ingress.kubernetes.io/router.middlewares: homepage-redirect@kubernetescrd
spec:
  tls:
  rules:
    - host: "homepage.domain"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: homepage
                port:
                  name: http

• PVC for images:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: homepage
  name: homepage-images
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi
  storageClassName: nfs

Container Logs

Setting LOG_LEVEL to debug didn't tell me much, except for this:

debug: No containers were found with homepage labels.

Browser Logs

The browser console doesn't output anything in either case.

Troubleshooting

Again, one replica works fine, 2 don't.

I tried circumventing the Ingress by changing the service type to NodePort, but it didn't fix the issue.

[glime-ai[bot]]

Thanks for posting a discussion about homepage here on GitHub. Homepage now uses a fancy new AI tool thanks to our friends at Glime, so the answer below is the AI bot's attempt to answer your question. If you feel it's sufficient you can mark the answer and close this post, otherwise feel free to leave it open of course!

Thanks for using Homepage! ❤️

---

Based on the information provided, it seems that the issue might be related to the configuration of the settings.yaml file not being applied consistently across replicas. Since you mentioned that the settings.yaml file is not being respected in some browser sessions, this could indicate a problem with the way the configuration is being loaded or synchronized across the different replicas.

Here are a few troubleshooting steps you can take:

1. Check ConfigMap and Volume Mounts: Ensure that the settings.yaml is correctly defined in the ConfigMap and that the volume mounts are correctly set up in the deployment configuration to reflect the changes across all replicas.

2. Sticky Sessions: Since you are using Traefik as an Ingress controller and have sticky sessions enabled, make sure that the sticky session configuration is working correctly. Sticky sessions can sometimes cause issues with load balancing across replicas.

3. File Permissions: Verify that the file permissions for settings.yaml are correctly set so that the application can read the file in all replicas.

4. Synchronization: If the settings.yaml file is being updated dynamically, ensure that the updates are synchronized across all replicas. This might not be the case if you are using a volume with ReadWriteMany access mode, but it's worth checking.

5. Logs: Since you've already set LOG_LEVEL to debug and found a message indicating that no containers were found with homepage labels, investigate further to see if there are any other relevant log entries that could provide more insight.

6. Replica Specific Data: Check if there's any replica-specific data or state that could cause the settings.yaml to not be applied. This could be related to caching or session data that is not shared across replicas.

7. Kubernetes Version: You mentioned that you are using K3s v1.28.7+k3s1. Check if there are any known issues with this version of K3s that could affect the behavior of your deployment.

8. Homepage Version: You are using Homepage v0.9.2. Review the release notes for this version to see if there are any known issues or changes related to configuration loading that could affect your setup.

If none of these steps resolve the issue, consider reaching out to the Homepage community or maintainers for further assistance, providing them with the details of your configuration and the behavior you are observing.

This comment was generated by Glime.

[brendanpolin]

I'm having a similar issue,

The settings file is being correctly put in the container and the pod can read it, but it's not being applied to the site.

I just used the icon as an example, but the columns and other settings aren't being applied as well.

Cluster Ver: v1.29.5+k3s1
number of Homepage Pods: 1
Other note: I use helm

[brendanpolin]

I found my issue at least, the bookmarks yaml was being incorrectly created by the helm chart resulting in a yaml error and causing it to not read the settings file.

My solution: After I commented out the bookmarks section of my helm values homepage started reading the settings file again

[litetex]

I'm actually having quite the same problem but it's not related to Kubernetes.

I found out the following:

• I'm using Firefox as browser
• I'm running homepage in a Docker container
• When I open the dashboard after a restart and after a long time it is displayed incorrectly (with default settings)
• Once I restart the server everything shows up correctly

This behavior seems to be Browser Cache related.
It looks like if there is no cache and you startup homepage the error is triggered due to some race condition...

[github-actions[bot]]

This discussion has been automatically closed due to inactivity. See our contributing guidelines for more details.

[github-actions[bot]]

This discussion has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion for related concerns. See our contributing guidelines for more details.