[Bug]: k3s container is unable to start on wsl2 #10375
Description
Module
K3S
Testcontainers version
1.20.3
Using the latest Testcontainers version?
Yes
Host OS
Windows 11 WSL2
Host Arch
x64
Docker version
Client:
Version: 28.1.1-rd
API version: 1.47 (downgraded from 1.49)
Go version: go1.23.8
Git commit: 4d7f01e
Built: Fri Apr 18 18:11:58 2025
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 27.3.1
API version: 1.47 (minimum version 1.24)
Go version: go1.23.9
Git commit: 41ca978a0a5400cc24b274137efa9f25517fcc0b
Built: Thu May 8 20:02:17 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.0.0
GitCommit: 207ad711eabd375a01713109a8a197d197ff6542
runc:
Version: 1.2.2
GitCommit: 7cb363254b69e10320360b63fb73e0ffb5da7bf2
docker-init:
Version: 0.19.0
GitCommit:What happened?
Context:
Using K3sContainer with Testcontainers to start an embedded cluster for integration tests. Some pods will be spawned for test purpose via the Operator SDK framework. We are using the org.testcontainers:junit-jupiter and org.testcontainers:k3s dependencies.
Rancher Desktop is providing a docker compatible API via its moby container engine. We have this setup on multiple machines using Windows 11 x64 and Mac.
Expected Result:
The pods should be able to start without any problems on different OS.
Actual Result:
The pods are unable to start due to issues with the cgroup system. At least on Windows. Tests on a Mac machine work without problems.
Existence of a workaround:
I have found a workaround in order to proceed with my daily tasks. Please find a more detailed explanation of it below.
Relevant log output
2025-06-27 10:27:21,578 INFO [org.tes.DockerClientFactory] (main) Testcontainers version: 1.20.6
2025-06-27 10:27:21,657 INFO [org.tes.doc.DockerClientProviderStrategy] (main) Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2025-06-27 10:27:21,803 INFO [org.tes.doc.DockerClientProviderStrategy] (main) Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2025-06-27 10:27:21,824 INFO [org.tes.DockerClientFactory] (pool-2-thread-1) Docker host IP address is localhost
2025-06-27 10:27:21,836 INFO [org.tes.DockerClientFactory] (pool-2-thread-1) Connected to docker:
Server Version: 27.3.1
API Version: 1.47
Operating System: Rancher Desktop WSL Distribution
Total Memory: 40109 MB
2025-06-27 10:27:21,875 INFO [tc.tes.11.0] (pool-2-thread-1) Creating container for image: testcontainers/ryuk:0.11.0
2025-06-27 10:27:21,954 INFO [org.tes.uti.RegistryAuthLocator] (pool-2-thread-1) Credential helper/store (docker-credential-wincred.exe) does not have credentials for https://index.docker.io/v1/
2025-06-27 10:27:22,072 INFO [tc.tes.11.0] (pool-2-thread-1) Container testcontainers/ryuk:0.11.0 is starting: 3e33972713c713daee535d3c006e1f0ee6d343ac2afbe2acd63998bcbdb86377
2025-06-27 10:27:22,415 INFO [tc.tes.11.0] (pool-2-thread-1) Container testcontainers/ryuk:0.11.0 started in PT0.540254449S
2025-06-27 10:27:22,417 INFO [org.tes.uti.RyukResourceReaper] (pool-2-thread-1) Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
2025-06-27 10:27:22,418 INFO [org.tes.DockerClientFactory] (pool-2-thread-1) Checking the system...
2025-06-27 10:27:22,418 INFO [org.tes.DockerClientFactory] (pool-2-thread-1) ✔︎ Docker server version should be at least 1.6.0
2025-06-27 10:27:22,418 INFO [tc.ran.33.1-k3s1] (pool-2-thread-1) Creating container for image: rancher/k3s:v1.33.1-k3s1
2025-06-27 10:27:22,764 WARN [org.tes.con.ContainerDef] (pool-2-thread-1) Unable to mount a file from test host into a running container. This may be a misconfiguration or limitation of your Docker environment. Some features might not work.
2025-06-27 10:27:22,808 INFO [tc.ran.33.1-k3s1] (pool-2-thread-1) Container rancher/k3s:v1.33.1-k3s1 is starting: 4b25df6ebe50bc5f3a374e6cc3816e8c660f9e690a08ff12c01debd22659ab00
K3S: time="2025-06-27T08:27:23Z" level=info msg="Starting k3s v1.33.1+k3s1 (99d91538)"
K3S: time="2025-06-27T08:27:23Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
K3S: time="2025-06-27T08:27:23Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
K3S: time="2025-06-27T08:27:23Z" level=info msg="Database tables and indexes are up to date"
K3S: time="2025-06-27T08:27:23Z" level=info msg="Kine available at unix://kine.sock"
.........
.........
K3S: I0627 08:27:36.853024 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:36.853042 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:36.854615 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Starting network policy controller version v2.2.1, built on 2025-05-23T19:10:39Z, go1.24.2"
K3S: I0627 08:27:37.542464 55 network_policy_controller.go:164] Starting network policy controller
K3S: I0627 08:27:37.560735 55 network_policy_controller.go:176] Starting network policy controller full sync goroutine
K3S: time="2025-06-27T08:27:37Z" level=info msg="Started tunnel to 172.17.0.3:6443"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Stopped tunnel to 127.0.0.1:6443"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Proxy done" err="context canceled" url="wss://127.0.0.1:6443/v1-k3s/connect"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Connecting to proxy" url="wss://172.17.0.3:6443/v1-k3s/connect"
K3S: time="2025-06-27T08:27:37Z" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Handling backend connection request [4b25df6ebe50]"
K3S: time="2025-06-27T08:27:37Z" level=info msg="Remotedialer connected to proxy" url="wss://172.17.0.3:6443/v1-k3s/connect"
K3S: I0627 08:27:38.848375 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:38.848863 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:38.849176 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:38.854361 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49b5e9da6608cf77e641373a299ac9a392c2ad9fb7fe25fbe4b7ecce5cf35656"
K3S: I0627 08:27:38.855340 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fb9ba02d6ff5fc379d9c4b9bcc302d056c485172bc2f3c699c09bce5a527da5"
K3S: I0627 08:27:38.856644 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23d6a099babf6955da1c400863b285d96c9e55fabdce0eb81a6f894c93d30d0b"
K3S: I0627 08:27:40.847543 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:40.847739 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:40.848011 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:41.205541 55 kuberuntime_manager.go:1746] "Updating runtime config through cri with podcidr" CIDR="10.42.0.0/24"
K3S: I0627 08:27:41.206069 55 kubelet_network.go:61] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="10.42.0.0/24"
K3S: E0627 08:27:41.795322 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"49b5e9da6608cf77e641373a299ac9a392c2ad9fb7fe25fbe4b7ecce5cf35656\" is not running" containerID="2146dc8d1fb34c77af0d10a1d999ec4e89ec438da92edbbf493bc13d894792bf"
K3S: E0627 08:27:41.795420 55 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:local-path-provisioner,Image:rancher/local-path-provisioner:v0.0.31,Command:[local-path-provisioner start --config /etc/config/config.json],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:POD_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:false,MountPath:/etc/config/,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-z954w,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd): RunContainerError: sandbox container \"49b5e9da6608cf77e641373a299ac9a392c2ad9fb7fe25fbe4b7ecce5cf35656\" is not running"
K3S: E0627 08:27:41.796493 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with RunContainerError: \"sandbox container \\\"49b5e9da6608cf77e641373a299ac9a392c2ad9fb7fe25fbe4b7ecce5cf35656\\\" is not running\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: E0627 08:27:42.698177 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"4fb9ba02d6ff5fc379d9c4b9bcc302d056c485172bc2f3c699c09bce5a527da5\" is not running" containerID="e5417a211a69f5ec95c7360f02c069e39b3feb36ebe51c5a6283353270c814ec"
K3S: E0627 08:27:42.698282 55 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:metrics-server,Image:rancher/mirrored-metrics-server:v0.7.2,Command:[],Args:[--cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=15s --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:10250,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{73400320 0} {<nil>} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-hr9bv,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:2,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84): RunContainerError: sandbox container \"4fb9ba02d6ff5fc379d9c4b9bcc302d056c485172bc2f3c699c09bce5a527da5\" is not running"
K3S: E0627 08:27:42.699384 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with RunContainerError: \"sandbox container \\\"4fb9ba02d6ff5fc379d9c4b9bcc302d056c485172bc2f3c699c09bce5a527da5\\\" is not running\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:42.846544 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:42.846603 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:42.846808 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:42.868209 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f249b2642f91a93ed9eca12a6e4b0f9b57e0f699d2248988bc9eb2c02346d69"
K3S: I0627 08:27:42.868261 55 scope.go:117] "RemoveContainer" containerID="2146dc8d1fb34c77af0d10a1d999ec4e89ec438da92edbbf493bc13d894792bf"
K3S: E0627 08:27:42.876097 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"23d6a099babf6955da1c400863b285d96c9e55fabdce0eb81a6f894c93d30d0b\" is not running" containerID="4722bb32891fd3736ba4173a397d05aaaf1b0ebead92a6ff0298e55dcc498180"
K3S: E0627 08:27:42.876339 55 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:rancher/mirrored-coredns-coredns:1.12.1,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {<nil>} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{73400320 0} {<nil>} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:custom-config-volume,ReadOnly:true,MountPath:/etc/coredns/custom,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-wmtv6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:2,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[all],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3): RunContainerError: sandbox container \"23d6a099babf6955da1c400863b285d96c9e55fabdce0eb81a6f894c93d30d0b\" is not running"
K3S: E0627 08:27:42.877475 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with RunContainerError: \"sandbox container \\\"23d6a099babf6955da1c400863b285d96c9e55fabdce0eb81a6f894c93d30d0b\\\" is not running\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:43.074768 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:43.874658 55 scope.go:117] "RemoveContainer" containerID="7fe28a8188a9eaa436e49ff17f0675eba6cef527dbe1fcce04a0e3b6a1cb5e1b"
K3S: E0627 08:27:43.874732 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:43.876266 55 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podStartSLOduration=3.064503207 podStartE2EDuration="7.876256355s" podCreationTimestamp="2025-06-27 08:27:36 +0000 UTC" firstStartedPulling="2025-06-27 08:27:37.871928407 +0000 UTC m=+13.919543984" lastFinishedPulling="2025-06-27 08:27:42.683681551 +0000 UTC m=+18.731297132" observedRunningTime="2025-06-27 08:27:43.876010156 +0000 UTC m=+19.923625740" watchObservedRunningTime="2025-06-27 08:27:43.876256355 +0000 UTC m=+19.923871940"
K3S: I0627 08:27:44.276498 55 controller.go:667] quota admission added evaluator for: jobs.batch
K3S: I0627 08:27:44.470449 55 alloc.go:328] "allocated clusterIPs" service="test/nginx-service" clusterIPs={"IPv4":"10.43.71.84"}
K3S: I0627 08:27:44.473612 55 cidrallocator.go:277] updated ClusterIP allocator for Service CIDR 10.43.0.0/16
K3S: I0627 08:27:44.559193 55 alloc.go:328] "allocated clusterIPs" service="test/keycloak-service" clusterIPs={"IPv4":"10.43.69.208"}
.........
.........
K3S: I0627 08:27:46.897461 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3f78c4978c38e0b082ec72eaef6cae4ddd80ef47cc0cc75cd7b45eb277f84be"
K3S: I0627 08:27:46.898023 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfa9a75f85361f5bca946233845b472665d8c21ce87ef99b73bb2ea9f3aa7f7f"
K3S: I0627 08:27:46.898832 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8838ddfec5b2f44071e18d5d26a6865ec4930871debc6d638652e2c5e6a73c3a"
K3S: I0627 08:27:46.899382 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e295afd6569f5f7105c2ea845a28e944f9e3a7b8d460c6571ff3c13872f6e67"
K3S: I0627 08:27:46.901122 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aec42fa38c1f131f7a195d5f52fb4d9faef6b7f3aac0d1c7dfa9658fd3010b76"
K3S: I0627 08:27:46.902663 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4410eb5dc71622d7920973c2e4ced1715162e5e1c47a3483dcaf1abe04a80a0f"
K3S: I0627 08:27:46.904963 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cac716960ca152d3f43512d38fd90f3d6553454ecc41b9c83d802d4cd0baaba0"
K3S: I0627 08:27:46.906109 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c33e5bb371fff9b5360d696d0bdc5350ab424ef7e1d8ae7d8e4964da6b64cc8"
K3S: I0627 08:27:46.907367 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a05e04c6fd0fc111ed0d77871cada5d05dd8bf187c6c65a7d786e14bb277b50"
K3S: E0627 08:27:47.159471 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: E0627 08:27:47.160876 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:47.188254 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:47.911107 55 scope.go:117] "RemoveContainer" containerID="f9c3747fc3bc773aa9efa298d4e30f83c736abedc840b692224673486204e84a"
K3S: E0627 08:27:47.911213 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:47.913610 55 scope.go:117] "RemoveContainer" containerID="7fe28a8188a9eaa436e49ff17f0675eba6cef527dbe1fcce04a0e3b6a1cb5e1b"
K3S: E0627 08:27:47.913732 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:47.915773 55 scope.go:117] "RemoveContainer" containerID="408a940c4cbc78223a6030a619cc15d9cf018879aa1a8b57a40f44982dfffeca"
K3S: E0627 08:27:47.915871 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:48.846902 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:48.847097 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:48.847141 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:48.919326 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e6aa8893a3b7d4bf57b875ba5fd998ddad5cb7c10f3bdf510a829ee65bcc438"
K3S: I0627 08:27:48.921776 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3250ab0073eb2a632218e74301c0edc08a4522359ecd846cc591019fdc839eaf"
K3S: I0627 08:27:48.923992 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9d05acb0dd9a5332b8010ccea38d9c10a87816c546c34efdb7c53f30602c452"
K3S: E0627 08:27:49.163430 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: E0627 08:27:49.166339 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:49.167638 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:49.928120 55 scope.go:117] "RemoveContainer" containerID="408a940c4cbc78223a6030a619cc15d9cf018879aa1a8b57a40f44982dfffeca"
K3S: E0627 08:27:49.928221 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:49.930222 55 scope.go:117] "RemoveContainer" containerID="f9c3747fc3bc773aa9efa298d4e30f83c736abedc840b692224673486204e84a"
K3S: E0627 08:27:49.930302 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:49.932312 55 scope.go:117] "RemoveContainer" containerID="7fe28a8188a9eaa436e49ff17f0675eba6cef527dbe1fcce04a0e3b6a1cb5e1b"
K3S: E0627 08:27:49.932403 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:50.846372 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:50.846556 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:50.846591 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:50.936492 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16838158f20e10e5fb36d4d30323cc37488870f28176666ff59bf8b850b94c1f"
K3S: I0627 08:27:50.938735 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e53946fa05ef4669bdbe9dd0029922d2dc88151d36fbfacbfa07bb2efa2f34ed"
K3S: I0627 08:27:50.941433 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89e5af92c8eee73b3606c9e9841ca331320291d96d3b1118316b2931336f57c4"
K3S: E0627 08:27:51.211902 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:51.213127 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: E0627 08:27:51.215220 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:51.945099 55 scope.go:117] "RemoveContainer" containerID="408a940c4cbc78223a6030a619cc15d9cf018879aa1a8b57a40f44982dfffeca"
K3S: E0627 08:27:51.945207 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:51.948040 55 scope.go:117] "RemoveContainer" containerID="f9c3747fc3bc773aa9efa298d4e30f83c736abedc840b692224673486204e84a"
K3S: E0627 08:27:51.948144 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:51.950823 55 scope.go:117] "RemoveContainer" containerID="7fe28a8188a9eaa436e49ff17f0675eba6cef527dbe1fcce04a0e3b6a1cb5e1b"
K3S: E0627 08:27:51.950918 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:52.846356 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:52.846512 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:52.846642 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:52.954559 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a41d4267eb10c130693291870c12c96c9dab9fb7b5f6219342545521795877"
K3S: I0627 08:27:52.957055 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66c7160928b46f3cb4f1d0e76c2b8174253fa5035504749bfb07a34f04fa301e"
K3S: I0627 08:27:52.959496 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e71a8899fd3268ae57f3221ea2a2ef6eb47782b79113f9f6ec7c937ada084fd"
K3S: E0627 08:27:53.244311 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: E0627 08:27:53.248277 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:53.963255 55 scope.go:117] "RemoveContainer" containerID="f9c3747fc3bc773aa9efa298d4e30f83c736abedc840b692224673486204e84a"
K3S: E0627 08:27:53.963357 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 10s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:53.968827 55 scope.go:117] "RemoveContainer" containerID="408a940c4cbc78223a6030a619cc15d9cf018879aa1a8b57a40f44982dfffeca"
K3S: E0627 08:27:53.968956 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 10s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:53.975871 55 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podStartSLOduration=14.066817742 podStartE2EDuration="17.975857672s" podCreationTimestamp="2025-06-27 08:27:36 +0000 UTC" firstStartedPulling="2025-06-27 08:27:37.870164162 +0000 UTC m=+13.917779744" lastFinishedPulling="2025-06-27 08:27:41.779204093 +0000 UTC m=+17.826819674" observedRunningTime="2025-06-27 08:27:53.97508407 +0000 UTC m=+30.022699655" watchObservedRunningTime="2025-06-27 08:27:53.975857672 +0000 UTC m=+30.023473256"
K3S: I0627 08:27:54.847225 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:54.847371 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:54.847371 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:54.972415 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d90d519ce6aa8aa51e5f5fc4c873c7bc8c0a3a849b585e3bdd4ef5f0a4ab3ffd"
K3S: I0627 08:27:54.975019 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54929ded732d3afdc1f297d9875d62075fecd4facfe2f269e3cdf705cfd517ba"
K3S: I0627 08:27:54.979207 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2ad6287498445f677ac031abfba583cdeae173f2accf941ee2b94fd8ac837b4"
K3S: I0627 08:27:54.979235 55 scope.go:117] "RemoveContainer" containerID="7fe28a8188a9eaa436e49ff17f0675eba6cef527dbe1fcce04a0e3b6a1cb5e1b"
K3S: E0627 08:27:55.259640 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 20s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: E0627 08:27:55.338875 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"f3f78c4978c38e0b082ec72eaef6cae4ddd80ef47cc0cc75cd7b45eb277f84be\" is not running" containerID="feda796405a649165ebe40ca7137af39ed4a3d9ab50c484fe284647aa1aa65bc"
K3S: E0627 08:27:55.338972 55 kuberuntime_manager.go:1358] "Unhandled Error" err="init container &Container{Name:fix-permissions,Image:public.ecr.aws/docker/library/busybox:latest,Command:[sh -c mkdir -p /opt/storage/shared/deployments && chown -R 1000:1000 /opt/storage/shared],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:storage,ReadOnly:false,MountPath:/opt/storage,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-8rlns,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod storage-5486697b4d-t86qd_test(c8dbfece-6c57-43e2-b495-03e779a19f20): RunContainerError: sandbox container \"f3f78c4978c38e0b082ec72eaef6cae4ddd80ef47cc0cc75cd7b45eb277f84be\" is not running"
K3S: E0627 08:27:55.340051 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"fix-permissions\" with RunContainerError: \"sandbox container \\\"f3f78c4978c38e0b082ec72eaef6cae4ddd80ef47cc0cc75cd7b45eb277f84be\\\" is not running\"" pod="test/storage-5486697b4d-t86qd" podUID="c8dbfece-6c57-43e2-b495-03e779a19f20"
K3S: E0627 08:27:55.344607 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"1c33e5bb371fff9b5360d696d0bdc5350ab424ef7e1d8ae7d8e4964da6b64cc8\" is not running" containerID="5cadd7ce36991593a9661781a2903baa0f031ca248a358201c61f1cbd5c2c786"
K3S: E0627 08:27:55.344703 55 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:storage-cleanup,Image:public.ecr.aws/docker/library/busybox,Command:[sh -c rm -rf /opt/storage/nodes/*],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:storage,ReadOnly:false,MountPath:/opt/storage,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-pnvn6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod storage-cleanup-job-9b768_test(7f6646d8-8fe9-4659-aaf1-b26822e0f9d6): RunContainerError: sandbox container \"1c33e5bb371fff9b5360d696d0bdc5350ab424ef7e1d8ae7d8e4964da6b64cc8\" is not running"
K3S: E0627 08:27:55.345769 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-cleanup\" with RunContainerError: \"sandbox container \\\"1c33e5bb371fff9b5360d696d0bdc5350ab424ef7e1d8ae7d8e4964da6b64cc8\\\" is not running\"" pod="test/storage-cleanup-job-9b768" podUID="7f6646d8-8fe9-4659-aaf1-b26822e0f9d6"
K3S: I0627 08:27:55.984006 55 scope.go:117] "RemoveContainer" containerID="9bcaf19106ef3c5e8c3e74f5a1ec363eeffe9d1c11991b43568d0bed4941337e"
K3S: E0627 08:27:55.984125 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 20s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:56.005994 55 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/coredns-697968c856-92jdv" podStartSLOduration=15.019436048 podStartE2EDuration="20.005982367s" podCreationTimestamp="2025-06-27 08:27:36 +0000 UTC" firstStartedPulling="2025-06-27 08:27:37.867311933 +0000 UTC m=+13.914927509" lastFinishedPulling="2025-06-27 08:27:42.853858246 +0000 UTC m=+18.901473828" observedRunningTime="2025-06-27 08:27:56.005809487 +0000 UTC m=+32.053425072" watchObservedRunningTime="2025-06-27 08:27:56.005982367 +0000 UTC m=+32.053597952"
K3S: I0627 08:27:56.064437 55 scope.go:117] "RemoveContainer" containerID="feda796405a649165ebe40ca7137af39ed4a3d9ab50c484fe284647aa1aa65bc"
K3S: I0627 08:27:56.847653 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: I0627 08:27:56.847653 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:56.847659 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:56.998205 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="703febb2c0bcd8efdbf63305e67070cf88767336238d0ecb29b4e8b7eb86606e"
K3S: I0627 08:27:57.001274 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1041b9dfb009ca11372b5faaed8c9be0866b4bc21c6c3cb94969b7b63e02c357"
K3S: I0627 08:27:57.001296 55 scope.go:117] "RemoveContainer" containerID="408a940c4cbc78223a6030a619cc15d9cf018879aa1a8b57a40f44982dfffeca"
K3S: I0627 08:27:57.004653 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e198829ed327133bf3f3852b7350cfbe8014cb860c5c353f16767681d4a3d23d"
K3S: I0627 08:27:57.007469 55 scope.go:117] "RemoveContainer" containerID="f9c3747fc3bc773aa9efa298d4e30f83c736abedc840b692224673486204e84a"
K3S: I0627 08:27:57.009063 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba7aec7259f7ea440636c0cb5dbdf79dd49a1357e1204e8adae1928f82435dec"
K3S: I0627 08:27:57.184415 55 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnvn6\" (UniqueName: \"kubernetes.io/projected/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-kube-api-access-pnvn6\") pod \"7f6646d8-8fe9-4659-aaf1-b26822e0f9d6\" (UID: \"7f6646d8-8fe9-4659-aaf1-b26822e0f9d6\") "
K3S: I0627 08:27:57.184441 55 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"storage\" (UniqueName: \"kubernetes.io/host-path/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-storage-volume-test\") pod \"7f6646d8-8fe9-4659-aaf1-b26822e0f9d6\" (UID: \"7f6646d8-8fe9-4659-aaf1-b26822e0f9d6\") "
K3S: I0627 08:27:57.184504 55 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-storage-volume-test" (OuterVolumeSpecName: "storage") pod "7f6646d8-8fe9-4659-aaf1-b26822e0f9d6" (UID: "7f6646d8-8fe9-4659-aaf1-b26822e0f9d6"). InnerVolumeSpecName "storage-volume-test". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
K3S: I0627 08:27:57.186147 55 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-kube-api-access-pnvn6" (OuterVolumeSpecName: "kube-api-access-pnvn6") pod "7f6646d8-8fe9-4659-aaf1-b26822e0f9d6" (UID: "7f6646d8-8fe9-4659-aaf1-b26822e0f9d6"). InnerVolumeSpecName "kube-api-access-pnvn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
K3S: I0627 08:27:57.285169 55 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnvn6\" (UniqueName: \"kubernetes.io/projected/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-kube-api-access-pnvn6\") on node \"4b25df6ebe50\" DevicePath \"\""
K3S: I0627 08:27:57.285195 55 reconciler_common.go:299] "Volume detached for volume \"storage-volume-test\" (UniqueName: \"kubernetes.io/host-path/7f6646d8-8fe9-4659-aaf1-b26822e0f9d6-storage-volume-test\") on node \"4b25df6ebe50\" DevicePath \"\""
K3S: E0627 08:27:57.291016 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 20s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: E0627 08:27:57.292375 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 20s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: E0627 08:27:57.305376 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 20s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:57.456082 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"703febb2c0bcd8efdbf63305e67070cf88767336238d0ecb29b4e8b7eb86606e\" is not running" containerID="9666916fd7a16e7140fe85a4d21455bf164d53dc2d335055e579d3fd6977b857"
K3S: E0627 08:27:57.456166 55 kuberuntime_manager.go:1358] "Unhandled Error" err="init container &Container{Name:fix-permissions,Image:public.ecr.aws/docker/library/busybox:latest,Command:[sh -c mkdir -p /opt/storage/shared/deployments && chown -R 1000:1000 /opt/storage/shared],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:storage,ReadOnly:false,MountPath:/opt/storage,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-8rlns,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod storage-5486697b4d-t86qd_test(c8dbfece-6c57-43e2-b495-03e779a19f20): RunContainerError: sandbox container \"703febb2c0bcd8efdbf63305e67070cf88767336238d0ecb29b4e8b7eb86606e\" is not running"
K3S: E0627 08:27:57.457260 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"fix-permissions\" with RunContainerError: \"sandbox container \\\"703febb2c0bcd8efdbf63305e67070cf88767336238d0ecb29b4e8b7eb86606e\\\" is not running\"" pod="test/storage-5486697b4d-t86qd" podUID="c8dbfece-6c57-43e2-b495-03e779a19f20"
K3S: E0627 08:27:57.940887 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"0a05e04c6fd0fc111ed0d77871cada5d05dd8bf187c6c65a7d786e14bb277b50\" is not running" containerID="d4884e7710e17e472054bebaa76134573c6d236cfa3c4cfddd7764360af8ef75"
K3S: E0627 08:27:57.940982 55 kuberuntime_manager.go:1358] "Unhandled Error" err="init container &Container{Name:init-permissions,Image:registry.access.redhat.com/ubi8,Command:[sh -c chmod g+rws /opt/keycloak/data],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:storage,ReadOnly:false,MountPath:/opt/keycloak/data,SubPath:keycloak/data,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-bcmmv,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keycloak-deployment-769c677ffc-ps2f4_test(9b972221-934f-49b0-8e50-a4707f900998): RunContainerError: sandbox container \"0a05e04c6fd0fc111ed0d77871cada5d05dd8bf187c6c65a7d786e14bb277b50\" is not running"
K3S: E0627 08:27:57.942079 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init-permissions\" with RunContainerError: \"sandbox container \\\"0a05e04c6fd0fc111ed0d77871cada5d05dd8bf187c6c65a7d786e14bb277b50\\\" is not running\"" pod="test/keycloak-deployment-769c677ffc-ps2f4" podUID="9b972221-934f-49b0-8e50-a4707f900998"
K3S: I0627 08:27:58.013297 55 scope.go:117] "RemoveContainer" containerID="83d57cf3e03839e44f062c137005ceff3f84181ef6afe5ffa7a8fe5186437970"
K3S: E0627 08:27:58.013409 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 20s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: I0627 08:27:58.016372 55 scope.go:117] "RemoveContainer" containerID="483d75b4c5e43ab774eb1b5497e4ae5d11309c95994aaedcc79a555cdff72fe7"
K3S: E0627 08:27:58.016576 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 20s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: I0627 08:27:58.020508 55 scope.go:117] "RemoveContainer" containerID="9bcaf19106ef3c5e8c3e74f5a1ec363eeffe9d1c11991b43568d0bed4941337e"
K3S: E0627 08:27:58.020609 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 20s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"
K3S: I0627 08:27:58.164305 55 scope.go:117] "RemoveContainer" containerID="9666916fd7a16e7140fe85a4d21455bf164d53dc2d335055e579d3fd6977b857"
K3S: I0627 08:27:58.201032 55 scope.go:117] "RemoveContainer" containerID="d4884e7710e17e472054bebaa76134573c6d236cfa3c4cfddd7764360af8ef75"
K3S: E0627 08:27:58.283484 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"fix-permissions\" with CrashLoopBackOff: \"back-off 10s restarting failed container=fix-permissions pod=storage-5486697b4d-t86qd_test(c8dbfece-6c57-43e2-b495-03e779a19f20)\"" pod="test/storage-5486697b4d-t86qd" podUID="c8dbfece-6c57-43e2-b495-03e779a19f20"
K3S: I0627 08:27:58.846482 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","pod251b8724-2fba-43a7-94a1-e000139f6723"] err="os: process not initialized"
K3S: I0627 08:27:58.846518 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","besteffort","pod7f6c53b6-16fc-44b3-a91c-f54a6e453779"] err="os: process not initialized"
K3S: I0627 08:27:58.846573 55 pod_container_manager_linux.go:207] "Failed to kill all the processes attached to cgroup" cgroupName=["kubepods","burstable","poda2fabe93-9955-4f3e-83a6-8b3d7e95e816"] err="os: process not initialized"
K3S: E0627 08:27:58.925001 55 log.go:32] "StartContainer from runtime service failed" err="rpc error: code = Unknown desc = sandbox container \"1dc34867fe06c2dadbbe2f4a2f923f4ff947abadc52ed567e2a5f2240670a4ea\" is not running" containerID="eafd27bae0981a09d8f1be86f1a3224cfd79fffe3ef482ddd43784c0c12fce65"
K3S: E0627 08:27:58.925085 55 kuberuntime_manager.go:1358] "Unhandled Error" err="init container &Container{Name:init-permissions,Image:registry.access.redhat.com/ubi8,Command:[sh -c chmod g+rws /opt/keycloak/data],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:storage,ReadOnly:false,MountPath:/opt/keycloak/data,SubPath:keycloak/data,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-bcmmv,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keycloak-deployment-769c677ffc-ps2f4_test(9b972221-934f-49b0-8e50-a4707f900998): RunContainerError: sandbox container \"1dc34867fe06c2dadbbe2f4a2f923f4ff947abadc52ed567e2a5f2240670a4ea\" is not running"
K3S: E0627 08:27:58.926296 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init-permissions\" with RunContainerError: \"sandbox container \\\"1dc34867fe06c2dadbbe2f4a2f923f4ff947abadc52ed567e2a5f2240670a4ea\\\" is not running\"" pod="test/keycloak-deployment-769c677ffc-ps2f4" podUID="9b972221-934f-49b0-8e50-a4707f900998"
K3S: I0627 08:27:59.035466 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dc34867fe06c2dadbbe2f4a2f923f4ff947abadc52ed567e2a5f2240670a4ea"
K3S: I0627 08:27:59.037239 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4279377afe1c0e859a25c2fc9afb94f26c70c93ca48b04876cdb4b469d2477a"
K3S: I0627 08:27:59.041369 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b66735821f867f505b38b4dbb33f3d5030696a478a2d7fc153c3e643e584888"
K3S: I0627 08:27:59.046860 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b02e5c9f92e9e7c2a3d67ae2fe549a36668fe3ffbfd5a1e767956c187b9eb43"
K3S: I0627 08:27:59.051189 55 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea35c69ffaef7cef01da890104a56a06aa11a282297473ed0846d1a5fe0cd4ce"
K3S: I0627 08:27:59.156116 55 scope.go:117] "RemoveContainer" containerID="eafd27bae0981a09d8f1be86f1a3224cfd79fffe3ef482ddd43784c0c12fce65"
K3S: E0627 08:27:59.289009 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init-permissions\" with CrashLoopBackOff: \"back-off 10s restarting failed container=init-permissions pod=keycloak-deployment-769c677ffc-ps2f4_test(9b972221-934f-49b0-8e50-a4707f900998)\"" pod="test/keycloak-deployment-769c677ffc-ps2f4" podUID="9b972221-934f-49b0-8e50-a4707f900998"
K3S: E0627 08:27:59.358034 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"metrics-server\" with CrashLoopBackOff: \"back-off 20s restarting failed container=metrics-server pod=metrics-server-6f4c6675d5-s7pp2_kube-system(ec1dcbc4-20fa-4372-b85c-be5348d10a84)\"" pod="kube-system/metrics-server-6f4c6675d5-s7pp2" podUID="ec1dcbc4-20fa-4372-b85c-be5348d10a84"
K3S: E0627 08:27:59.411868 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with CrashLoopBackOff: \"back-off 20s restarting failed container=coredns pod=coredns-697968c856-92jdv_kube-system(6b220854-264f-4415-8c7d-d3ca42872cd3)\"" pod="kube-system/coredns-697968c856-92jdv" podUID="6b220854-264f-4415-8c7d-d3ca42872cd3"
K3S: E0627 08:27:59.422362 55 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"local-path-provisioner\" with CrashLoopBackOff: \"back-off 20s restarting failed container=local-path-provisioner pod=local-path-provisioner-774c6665dc-hc695_kube-system(3a982885-c4ad-416d-9fff-ac92189b24fd)\"" pod="kube-system/local-path-provisioner-774c6665dc-hc695" podUID="3a982885-c4ad-416d-9fff-ac92189b24fd"Additional Information
wsl.exe -v
WSL version: 2.5.7.0
Kernel version: 6.6.87.1-1
WSLg version: 1.0.66
MSRDC version: 1.2.6074
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.26100.4349
WSL distribution: Ubuntu 24.04
Windows: 11 Enterprise
Version: 24H2
OS build: 26100.4349
Experience: Windows Feature Experience Pack 1000.26100.107.0
All available Windows updates installed
Followed informations on this GitHub page regarding the cgroup problems on WSL systems: https://github.com/spurin/wsl-cgroupsv2
docker run -it --rm spurin/wsl-cgroupsv2:latest returns Success: cgroup type is cgroup2fs on my machine.
I was able to create a green test by changing the constructor contents of the K3sContainer.java and by removing its cgroup configuration:
public K3sContainer(DockerImageName dockerImageName) {
super(dockerImageName);
dockerImageName.assertCompatibleWith(new DockerImageName[]{DockerImageName.parse("rancher/k3s")});
this.addExposedPorts(new int[]{KUBE_SECURE_PORT, RANCHER_WEBHOOK_PORT});
this.setPrivilegedMode(true);
// removed the two lines below
// this.withCreateContainerCmdModifier((it) -> it.getHostConfig().withCgroupnsMode("host"));
// this.addFileSystemBind("/sys/fs/cgroup", "/sys/fs/cgroup", BindMode.READ_WRITE);
Map<String, String> tmpFsMapping = new HashMap();
tmpFsMapping.put("/run", "");
tmpFsMapping.put("/var/run", "");
this.setTmpFsMapping(tmpFsMapping);
this.setCommand(new String[]{"server", "--disable=traefik", "--tls-san=" + this.getHost()});
this.setWaitStrategy(Wait.forLogMessage(".*Node controller sync successful.*", 1));
}