Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2521/e2e-tests/logs/scaling-8-0.log Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "pxc-operator" not found waiting for namespace/pxc-operator to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2521-bb594380-1-cluster8" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com condition met E0611 03:35:03.671663 31108 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusters.pxc.percona.com&resourceVersion=1781148902783516000&timeoutSeconds=334&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com condition met E0611 03:35:06.439845 31563 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterbackups.pxc.percona.com&resourceVersion=1781148902783516000&timeoutSeconds=583&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com condition met E0611 03:35:09.695420 31933 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterrestores.pxc.percona.com&resourceVersion=1781148907783565000&timeoutSeconds=512&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-774dd4b69-4rrpx condition met E0611 03:35:32.656277 3098 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-774dd4b69-4rrpx&resourceVersion=1781148932297270000&timeoutSeconds=476&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" pod/percona-xtradb-cluster-operator-774dd4b69-4rrpx condition met E0611 03:35:38.959057 4124 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-774dd4b69-4rrpx&resourceVersion=1781148936401011000&timeoutSeconds=597&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/percona-xtradb-cluster-operator-774dd4b69-4rrpx to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces scaling-16649 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "scaling-16649" not found waiting for namespace/scaling-16649 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "scaling-16649" not found ----------------------------------------------------------------------------------- create namespace scaling-16649 ----------------------------------------------------------------------------------- namespace/scaling-16649 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2521-bb594380-1-cluster8" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/do-spaces-secret created secret/gcp-cs-secret created secret/azure-secret created ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/scaling created ----------------------------------------------------------------------------------- waiting pods to be running for scaling ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 3 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met waiting for pod/scaling-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- Waiting for 2 running pods for scaling-proxysql ----------------------------------------------------------------------------------- pod/scaling-proxysql-0 condition met E0611 03:40:39.504130 17633 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-0&resourceVersion=1781149239270451000&timeoutSeconds=422&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-0 to become Ready.Ok pod/scaling-proxysql-1 condition met E0611 03:40:55.067257 19936 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-1&resourceVersion=1781149254277608000&timeoutSeconds=355&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-1 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/scaling to be ready ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:41:54.052772 28578 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149310163283000&timeoutSeconds=345&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:42:24.462676 748 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149341287279000&timeoutSeconds=328&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- compare initial data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:43:25.749208 10110 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149401860798000&timeoutSeconds=526&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:43:45.543482 13085 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149421610532000&timeoutSeconds=470&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:44:06.070500 16099 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149442238593000&timeoutSeconds=374&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- scale up from 3 to 5 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if all 5 Pods started ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 5 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met E0611 03:44:34.231863 20283 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-0&resourceVersion=1781149471261551012&timeoutSeconds=511&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met E0611 03:44:48.103941 22211 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-1&resourceVersion=1781149486136447012&timeoutSeconds=481&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met E0611 03:45:01.633926 24263 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-2&resourceVersion=1781149500145140000&timeoutSeconds=330&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-2 to become Ready.Ok pod/scaling-pxc-3 condition met waiting for pod/scaling-pxc-3 to become Ready.Ok pod/scaling-pxc-4 condition met waiting for pod/scaling-pxc-4 to become Ready.Ok ----------------------------------------------------------------------------------- check if PVC created ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-06-11T03:47:26+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-06-11T03:47:28+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check data consistency for new Pods ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:47:31.493574 13836 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149650445788000&timeoutSeconds=582&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-bv6lm condition met E0611 03:47:42.927861 15320 reflector.go:227] "Failed to watch" err="Get \"https://34.121.67.239/api/v1/namespaces/scaling-16649/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-bv6lm&resourceVersion=1781149661140993000&timeoutSeconds=578&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-bv6lm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check new Pods exists in ProxySQL ----------------------------------------------------------------------------------- scaling-pxc-3.scaling-pxc.scaling-16649.svc.cluster.local scaling-pxc-3.scaling-pxc.scaling-16649.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-16649.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-16649.svc.cluster.local ----------------------------------------------------------------------------------- scale down from 5 to 3 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if Pod deleted ----------------------------------------------------------------------------------- waiting for pod/scaling-pxc-3 to be deleted.Error from server (NotFound): pods "scaling-pxc-3" not found waiting for pod/scaling-pxc-4 to be deletedError from server (NotFound): pods "scaling-pxc-4" not found ----------------------------------------------------------------------------------- check if PVC not deleted ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-06-11T03:48:58+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-06-11T03:49:01+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check if Pod deleted from ProxySQL ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling patched perconaxtradbcluster.pxc.percona.com "scaling" deleted from scaling-16649 namespace No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted namespace "pxc-operator" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. namespace "scaling-16649" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------