After restarting UDR, AMF loses the RanUe info

[gatici]

Describe the bug

After restarting UDR pod, AMF, authentication request failed in AMF by getting RanUe is not exist logs.

To Reproduce

Deploy the environment using tutorial: https://canonical-charmed-5g.readthedocs-hosted.com/en/latest/tutorials/getting_started/#
Run the simulation
Delete UDR pod
Run the simulation again
See that the simulation fails
Check the AMF logs

Expected behavior

The simulation should be succeeded.

Logs

2024-02-28T14:38:59.998Z [amf] 2024-02-28T14:38:59Z [INFO][Config5g][GRPC] connectivity ready
2024-02-28T14:39:00.000Z [amf] 2024-02-28T14:39:00Z [INFO][Config5g][GRPC] stream msg recieved
2024-02-28T14:39:00.000Z [amf] 2024-02-28T14:39:00Z [INFO][Config5g][GRPC] Complete config deleted
2024-02-28T14:39:00.000Z [amf] 2024-02-28T14:39:00Z [INFO][AMF][GRPC] Received updateConfig in the amf app : RestartCounter:125114909 ConfigUpdated:1
2024-02-28T14:39:36.046Z [amf] 2024-02-28T14:39:36Z [INFO][Config5g][GRPC] stream msg recieved
2024-02-28T14:39:36.046Z [amf] 2024-02-28T14:39:36Z [INFO][Config5g][GRPC] First time config Received RestartCounter:125114909 NetworkSlice:{Name:"default" Nssai:{Sst:"1" Sd:"010203"}} ConfigUpdated:1
2024-02-28T14:39:36.046Z [amf] 2024-02-28T14:39:36Z [INFO][AMF][GRPC] Received updateConfig in the amf app : RestartCounter:125114909 NetworkSlice:{Name:"default" Nssai:{Sst:"1" Sd:"010203"}} ConfigUpdated:1
2024-02-28T14:39:36.046Z [amf] 2024-02-28T14:39:36Z [INFO][AMF][GRPC] Network Slice Name default
2024-02-28T14:39:36.050Z [amf] 2024-02-28T14:39:36Z [INFO][Config5g][GRPC] stream msg recieved
2024-02-28T14:39:36.050Z [amf] 2024-02-28T14:39:36Z [INFO][Config5g][GRPC] Complete config deleted
2024-02-28T14:39:36.050Z [amf] 2024-02-28T14:39:36Z [INFO][AMF][GRPC] Received updateConfig in the amf app : RestartCounter:125114909 ConfigUpdated:1
2024-02-28T14:40:11.009Z [amf] 2024-02-28T14:40:11Z [INFO][Config5g][GRPC] stream msg recieved
2024-02-28T14:40:11.009Z [amf] 2024-02-28T14:40:11Z [INFO][Config5g][GRPC] Complete config deleted
2024-02-28T14:40:11.009Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Received updateConfig in the amf app : RestartCounter:125114909 ConfigUpdated:1
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][Config5g][GRPC] stream msg recieved
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][Config5g][GRPC] First time config Received RestartCounter:125114909 NetworkSlice:{Name:"default" Nssai:{Sst:"1" Sd:"010203"} DeviceGroup:{Name:"default-default" IpDomainDetails:{Name:"pool1" DnnName:"internet" UePool:"172.250.1.0/16" DnsPrimary:"8.8.8.8" Mtu:1460 UeDnnQos:{DnnMbrUplink:20000000 DnnMbrDownlink:200000000 TrafficClass:{Name:"platinum" Qci:8 Arp:6 Pdb:300 Pelr:6}}} Imsi:"208930100007487"} Site:{SiteName:"demo" Gnb:{Name:"core77-gnbsim-gnbsim" Tac:1} Plmn:{mcc:"208" mnc:"93"} Upf:{UpfName:"upf-external.core77.svc.cluster.local"}} AppFilters:{PccRuleBase:{FlowInfos:{FlowDesc:"permit out ip from any to assigned" TosTrafficClass:"IPV4" FlowDir:BIDIRECTIONAL} RuleId:"DefaultRule" Qos:{Var5qi:9 Arp:{PL:1 PC:MAY_PREEMPT PV:PREEMPTABLE}} Priority:255}} OperationType:SLICE_UPDATE AddUpdatedImsis:"208930100007487"} ConfigUpdated:1
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Received updateConfig in the amf app : RestartCounter:125114909 NetworkSlice:{Name:"default" Nssai:{Sst:"1" Sd:"010203"} DeviceGroup:{Name:"default-default" IpDomainDetails:{Name:"pool1" DnnName:"internet" UePool:"172.250.1.0/16" DnsPrimary:"8.8.8.8" Mtu:1460 UeDnnQos:{DnnMbrUplink:20000000 DnnMbrDownlink:200000000 TrafficClass:{Name:"platinum" Qci:8 Arp:6 Pdb:300 Pelr:6}}} Imsi:"208930100007487"} Site:{SiteName:"demo" Gnb:{Name:"core77-gnbsim-gnbsim" Tac:1} Plmn:{mcc:"208" mnc:"93"} Upf:{UpfName:"upf-external.core77.svc.cluster.local"}} AppFilters:{PccRuleBase:{FlowInfos:{FlowDesc:"permit out ip from any to assigned" TosTrafficClass:"IPV4" FlowDir:BIDIRECTIONAL} RuleId:"DefaultRule" Qos:{Var5qi:9 Arp:{PL:1 PC:MAY_PREEMPT PV:PREEMPTABLE}} Priority:255}} OperationType:SLICE_UPDATE AddUpdatedImsis:"208930100007487"} ConfigUpdated:1
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Network Slice Name default
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Network Slice has site name: demo
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Plmn mcc 208
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] SupportedPlmnLIst: {{208 93} [{1 010203}]}, SupportGuamiLIst: {0xc000756300 cafe00} received fromRoc
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] SupportedPlmnLIst: [{{208 93} [{1 010203}]}], SupportGuamiLIst: [{0xc000756300 cafe00}] in AMF
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Tai list present in Slice {0xc000365b40 1} []
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] Gnb Updated in existing Plmn, SupportTAILIst received from Roc: [{0xc000365b40 1}]
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][GRPC] SupportTAILIst in AMF [{0xc000365b40 1}]
2024-02-28T14:40:11.016Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][Util] amfconfig Info: Version[1.0.0] Description[AMF initial configuration]
2024-02-28T14:40:11.040Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][Init] Started KeepAlive Timer: 60 sec
2024-02-28T14:40:11.040Z [amf] 2024-02-28T14:40:11Z [INFO][AMF][CFG] Sent Register NF Instance with updated profile
2024-02-28T14:40:27.159Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP] [AMF] SCTP Accept from: 10.1.146.2/192.168.251.5:9487
2024-02-28T14:40:27.159Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP] Create a new NG connection for: 10.1.146.2/192.168.251.5:9487
2024-02-28T14:40:27.160Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] Handle NG Setup request
2024-02-28T14:40:27.160Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] Supported Tai List in AMF Plmn: &{208 93}, Tac: 0x000001 Tac: 1
2024-02-28T14:40:27.160Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] Send NG-Setup response
2024-02-28T14:40:27.162Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] RanUe is not exist
2024-02-28T14:40:27.162Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] Handle Initial UE Message
2024-02-28T14:40:27.162Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] RanUe is not exist
2024-02-28T14:40:27.162Z [amf] 2024-02-28T14:40:27Z [DEBU][DRSM][App] Allocate new chunk
2024-02-28T14:40:27.162Z [amf] 2024-02-28T14:40:27Z [DEBU][DRSM][App] Found chunk Id block 12550
2024-02-28T14:40:27.185Z [amf] 2024/02/28 14:40:27 Adding chunk 12550 success
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][Context] Allocate AmfUeNgapID : 12852199
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][DRSM][App] received Chunk Doc: &{chunkid-12550 chunkid-12550 amf-0 10.1.146.12 3cfa4e60-1b28-460a-956a-4818d21f81ff 0001-01-01 00:00:00 +0000 UTC chunk}
2024-02-28T14:40:27.185Z [amf] 2024/02/28 14:40:27 id received: chunkid-12550 value
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][DRSM][App] Chunk id 12550, podChunks map[12550:0xc0002b0e10 14006:0xc0008b42d0]
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][Comm] Security header type: PlainNas Message
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][Context] Allocate TMSI : 12852198
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487][AMF_UE_NGAP_ID:12852199] Antype from new RanUe : 3GPP_ACCESS
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199] New EventChannel created
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199] updated nashandler
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199] updated nashandler
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NAS][AMF_UE_NGAP_ID:12852199] Handle Nas Message
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][LIB][FSM] Handle event[Gmm Message], transition from [Deregistered] to [Deregistered]
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199] Handle Registration Request
2024-02-28T14:40:27.185Z [amf] 2024-02-28T14:40:27Z [INFO][LIB][FSM] Handle event[Start Authentication], transition from [Deregistered] to [Authentication]
2024-02-28T14:40:27.257Z [amf] 2024-02-28T14:40:27Z [ERRO][AMF][Kafka] Error publishing nf status event: dial tcp: lookup sd-core-kafka-headless on 10.152.183.10:53: no such host
2024-02-28T14:40:27.257Z [amf] 2024-02-28T14:40:27Z [ERRO][AMF][Kafka] Could not publish ue context event, error [dial tcp: lookup sd-core-kafka-headless on 10.152.183.10:53: no such host]
2024-02-28T14:40:27.257Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199][SUCI:suci-0-208-93-0-0-0-0100007487] Authentication procedure
2024-02-28T14:40:27.272Z [amf] 2024-02-28T14:40:27Z [ERRO][AMF][GMM][AMF_UE_NGAP_ID:12852199][SUCI:suci-0-208-93-0-0-0-0100007487] Nausf_UEAU Authenticate Request Failed: &{Type: Title: Status:500 Detail: Instance: Cause:AV_GENERATION_PROBLEM InvalidParams:[]}
2024-02-28T14:40:27.272Z [amf] 2024-02-28T14:40:27Z [INFO][LIB][FSM] Handle event[Authentication Error], transition from [Authentication] to [Deregistered]
2024-02-28T14:40:27.272Z [amf] 2024-02-28T14:40:27Z [ERRO][AMF][GMM][AMF_UE_NGAP_ID:12852199][SUCI:suci-0-208-93-0-0-0-0100007487] Handle Authentication Error
2024-02-28T14:40:27.272Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:12852199][SUCI:suci-0-208-93-0-0-0-0100007487] Send Registration Reject
2024-02-28T14:40:27.272Z [amf] 2024-02-28T14:40:27Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487][AMF_UE_NGAP_ID:12852199] Send Downlink Nas Transport
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [INFO][AMF][NGAP] Handle SCTP Notification[addr: ]
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] SCTP_SHUTDOWN_EVENT notification, close the connection
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [ERRO][AMF][Kafka] Error publishing nf status event: dial tcp: lookup sd-core-kafka-headless on 10.152.183.10:53: no such host
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [INFO][AMF][NGAP][10.1.146.2/192.168.251.5:9487] Remove RAN Context[ID: <PlmnID: {Mcc:208 Mnc:93}, GNbID: 000102>]
2024-02-28T14:40:32.278Z [amf] RanUe has been deleted2024-02-28T14:40:32Z [DEBU][DRSM][App] ID Released: 12852199
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [INFO][AMF][NGAP] Handle SCTP Notification[addr: ]
2024-02-28T14:40:32.278Z [amf] 2024-02-28T14:40:32Z [WARN][AMF][NGAP] RAN context has been removed[addr: ]
2024-02-28T14:44:36.352Z [pebble] GET /v1/files?action=list&path=%2Ffree5gc%2Fconfig&itself=true 76.38µs 200

Environment

• Charm / library version (if relevant):
• Juju version (output from juju --version): 3.1.7
• Cloud Environment:
• Kubernetes version (output from kubectl version --short): v1.27.11
• Terraform version (output from terraform version):

[gatici]

We checked that this is an issue caused by NMS and webconsole, not related with UDR.