Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2002/e2e-tests/logs/haproxy-5-7.log Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 No resources found + kubectl patch pxc -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: resource(s) were provided, but no name was specified No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "pxc-operator" not found waiting for namespace/pxc-operator to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2002-42929599-9-cluster9" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-6c8dc776-ggxl9 condition met pod/percona-xtradb-cluster-operator-6c8dc776-ggxl9 condition met waiting for pod/percona-xtradb-cluster-operator-6c8dc776-ggxl9 to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces haproxy-1159 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "haproxy-1159" not found waiting for namespace/haproxy-1159 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "haproxy-1159" not found ----------------------------------------------------------------------------------- create namespace haproxy-1159 ----------------------------------------------------------------------------------- namespace/haproxy-1159 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2002-42929599-9-cluster9" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/gcp-cs-secret created secret/azure-secret created error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- install chaos-mesh ----------------------------------------------------------------------------------- "chaos-mesh" already exists with the same configuration, skipping NAME: chaos-mesh LAST DEPLOYED: Tue Nov 4 08:20:06 2025 NAMESPACE: haproxy-1159 STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. Make sure chaos-mesh components are running kubectl get pods --namespace haproxy-1159 -l app.kubernetes.io/instance=chaos-mesh ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- runtimeclass.node.k8s.io/docker-rc created ----------------------------------------------------------------------------------- create first PXC cluster with HAProxy ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/haproxy created ----------------------------------------------------------------------------------- check if all 3 Pods started ----------------------------------------------------------------------------------- error: no matching resources found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/haproxy-haproxy-0 condition met waiting for pod/haproxy-haproxy-0 to become Ready.Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/haproxy-pxc-0 condition met waiting for pod/haproxy-pxc-0 to become Ready.Ok pod/haproxy-pxc-1 condition met waiting for pod/haproxy-pxc-1 to become Ready.Ok pod/haproxy-pxc-2 condition met waiting for pod/haproxy-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- Unable to use a TTY - input is not a terminal or the right kind of file ERROR: ld.so: object '/usr/lib64/libjemalloc.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object '/usr/lib64/libjemalloc.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Unable to use a TTY - input is not a terminal or the right kind of file ERROR: ld.so: object '/usr/lib64/libjemalloc.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object '/usr/lib64/libjemalloc.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ----------------------------------------------------------------------------------- checking all haproxy pods point to the same writer ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/haproxy-pxc-0 condition met waiting for pod/haproxy-pxc-0 to become Ready.Ok pod/haproxy-pxc-1 condition met waiting for pod/haproxy-pxc-1 to become Ready.Ok pod/haproxy-pxc-2 condition met waiting for pod/haproxy-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/haproxy to be ready pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- delete active writer and checking all haproxy pods still point to the same writer ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- fail pxc-pod-0 pod for 60s ----------------------------------------------------------------------------------- podchaos.chaos-mesh.org/chaos-pod-failure created NAME READY STATUS RESTARTS AGE chaos-controller-manager-5d489b565d-4wfjk 1/1 Running 0 6m37s chaos-controller-manager-5d489b565d-6q69l 1/1 Running 0 6m38s chaos-controller-manager-5d489b565d-hgqq4 1/1 Running 0 6m37s chaos-daemon-gxvpj 1/1 Running 0 6m39s chaos-daemon-mgs9s 1/1 Running 0 6m39s chaos-daemon-slrsk 1/1 Running 0 6m39s chaos-dns-server-769bdfb58d-b87xx 1/1 Running 0 6m38s haproxy-haproxy-0 3/3 Running 0 6m12s haproxy-haproxy-1 3/3 Running 0 4m56s haproxy-haproxy-2 3/3 Running 0 4m35s haproxy-pxc-0 0/1 CrashLoopBackOff 2 (7s ago) 6m12s haproxy-pxc-1 1/1 Running 0 4m58s haproxy-pxc-2 1/1 Running 0 4m15s pxc-client-857d976497-qgdt5 2/2 Running 0 6m17s pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-857d976497-qgdt5 condition met waiting for pod/pxc-client-857d976497-qgdt5 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/haproxy-pxc-0 condition met waiting for pod/haproxy-pxc-0 to become Ready.Ok pod/haproxy-pxc-1 condition met waiting for pod/haproxy-pxc-1 to become Ready.Ok pod/haproxy-pxc-2 condition met waiting for pod/haproxy-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- check advanced options are enabled in haproxy statefulset ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pdb/haproxy-haproxy- ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare statefulset/haproxy-haproxy- ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- default haproxy-replicas service ----------------------------------------------------------------------------------- The request is invalid: the server rejected our request due to an error in our request ----------------------------------------------------------------------------------- compare service/haproxy-haproxy-replicas- ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- disable haproxy-replicas service ----------------------------------------------------------------------------------- waiting for svc/haproxy-haproxy-replicas to be deletedError from server (NotFound): services "haproxy-haproxy-replicas" not found ----------------------------------------------------------------------------------- enable haproxy-replicas service ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare service/haproxy-haproxy-replicas- ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/haproxy-haproxy-0 condition met waiting for pod/haproxy-haproxy-0 to become Ready.Ok pod/haproxy-haproxy-1 condition met waiting for pod/haproxy-haproxy-1 to become Ready.Ok pod/haproxy-haproxy-2 condition met waiting for pod/haproxy-haproxy-2 to become Ready.Ok Unable to use a TTY - input is not a terminal or the right kind of file secret/haproxy-haproxy created ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/haproxy to be ready.......... ----------------------------------------------------------------------------------- compare statefulset/haproxy-haproxy--secret ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- return true if kubernetes version equal or greater than desired ----------------------------------------------------------------------------------- Unable to use a TTY - input is not a terminal or the right kind of file ----------------------------------------------------------------------------------- clean up ----------------------------------------------------------------------------------- release "chaos-mesh" uninstalled error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found podchaos.chaos-mesh.org/chaos-pod-failure patched podchaos.chaos-mesh.org "chaos-pod-failure" deleted from haproxy-1159 namespace No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found No resources found customresourcedefinition.apiextensions.k8s.io "awschaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "azurechaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "blockchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "dnschaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "gcpchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "httpchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "iochaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "jvmchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "kernelchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "networkchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "physicalmachinechaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "physicalmachines.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "podchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "podhttpchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "podiochaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "podnetworkchaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "remoteclusters.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "schedules.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "statuschecks.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "stresschaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "timechaos.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "workflownodes.chaos-mesh.org" deleted customresourcedefinition.apiextensions.k8s.io "workflows.chaos-mesh.org" deleted error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- + kubectl patch pxc -n haproxy-1159 haproxy --type=merge -p '{"metadata":{"finalizers":[]}}' perconaxtradbcluster.pxc.percona.com/haproxy patched perconaxtradbcluster.pxc.percona.com "haproxy" deleted from haproxy-1159 namespace No resources found No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------