[BUG] operator/ cluster example not spinning up a cluster

[eddykaya]

What is the bug?

When using the operatur/cluster example here with slight adjustments
No cluster is spun up.

The only pods created are the following:

How can one reproduce the bug?

I am using the following cluster definition which is almost identical to the example (only images are proxied)

cluster:
  general:
    setVMMaxMapCount: false
    image: <image-proxy>/dockerhub/opensearchproject/opensearch
    httpPort: 9200
    serviceName: my-first-cluster
    version: 2.14.0
    drainDataNodes: true
  dashboards:
    image: <image-proxy>/dockerhub/opensearchproject/opensearch-dashboards
    enable: false
    tls:
      enable: true
      generate: true
    version: 2.14.0
    replicas: 1
    resources:
      requests:
        memory: "512Mi"
        cpu: "200m"
      limits:
        memory: "512Mi"
        cpu: "200m"
  nodePools:
    - component: masters
      replicas: 3
      resources:
        requests:
          memory: "4Gi"
          cpu: "1000m"
        limits:
          memory: "4Gi"
          cpu: "1000m"
      roles:
        - "data"
        - "cluster_manager"
      persistence:
        emptyDir: {}
  security:
    tls:
      transport:
        generate: true
        perNode: true
      http:
        generate: true

What is the expected behavior?

A 3 node cluster is expected

What is your host/environment?

kubernetes 1.29

Do you have any screenshots?

The following log is from the node:

[2025-01-09T13:33:27,741][WARN ][o.o.c.c.ClusterFormationFailureHelper] [opensearch-cluster-dev-masters-0] cluster-manager not discovered yet, this node has not previously joined a bootstrap
ped cluster, and this node must discover cluster-manager-eligible nodes [opensearch-cluster-dev-bootstrap-0] to bootstrap a cluster: have discovered [{opensearch-cluster-dev-masters-0}{D5csH
MgzT0qUVz5n1byJ7g}{Cg6HPjlsTQGieLbFdsI1tw}{opensearch-cluster-dev-masters-0}{192.168.131.9:9300}{dm}{shard_indexing_pressure_enabled=true}]; discovery will continue using [192.168.131.11:930
0] from hosts providers and [{opensearch-cluster-dev-masters-0}{D5csHMgzT0qUVz5n1byJ7g}{Cg6HPjlsTQGieLbFdsI1tw}{opensearch-cluster-dev-masters-0}{192.168.131.9:9300}{dm}{shard_indexing_press
ure_enabled=true}] from last-known cluster state; node term 0, last-accepted version 0 in term 0                                                                                              
[2025-01-09T13:33:27,831][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:28,832][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:29,832][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:30,478][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-cluster-dev-masters-0] Not yet initialized (you may need to run securityadmin)                                        
[2025-01-09T13:33:30,832][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:31,832][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:32,832][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:33,833][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:34,833][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:35,833][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:36,833][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:37,742][WARN ][o.o.c.c.ClusterFormationFailureHelper] [opensearch-cluster-dev-masters-0] cluster-manager not discovered yet, this node has not previously joined a bootstrap
ped cluster, and this node must discover cluster-manager-eligible nodes [opensearch-cluster-dev-bootstrap-0] to bootstrap a cluster: have discovered [{opensearch-cluster-dev-masters-0}{D5csH
MgzT0qUVz5n1byJ7g}{Cg6HPjlsTQGieLbFdsI1tw}{opensearch-cluster-dev-masters-0}{192.168.131.9:9300}{dm}{shard_indexing_pressure_enabled=true}]; discovery will continue using [192.168.131.11:930
0] from hosts providers and [{opensearch-cluster-dev-masters-0}{D5csHMgzT0qUVz5n1byJ7g}{Cg6HPjlsTQGieLbFdsI1tw}{opensearch-cluster-dev-masters-0}{192.168.131.9:9300}{dm}{shard_indexing_press
ure_enabled=true}] from last-known cluster state; node term 0, last-accepted version 0 in term 0                                                                                              
[2025-01-09T13:33:37,834][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...                                                     
[2025-01-09T13:33:38,834][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-cluster-dev-masters-0] Wait for cluster to be available ...   

Do you have any additional context?

I am unable to run root pods in my cluster, so I disabled the init container according to the docs.

[eddykaya]

Not a bug, I found out that the cluster we are using has a default deny all policy established in any namespace which prevented any container from communicating with any other container.