Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2525/e2e-tests/logs/scaling-8-0.log Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified namespace "pxc-operator" deleted waiting for namespace/pxc-operator to be deletedError from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2525-c81e47f0-4-cluster4" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com condition met E0806 03:33:28.209218 1011887 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusters.pxc.percona.com&resourceVersion=1785987205389077000&timeoutSeconds=508&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com condition met E0806 03:33:31.067780 1012312 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterbackups.pxc.percona.com&resourceVersion=1785987210389345000&timeoutSeconds=527&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com condition met E0806 03:33:34.771577 1012738 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterrestores.pxc.percona.com&resourceVersion=1785987210389345000&timeoutSeconds=361&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-6699755d74-7hb72 condition met E0806 03:33:59.758960 1015980 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-6699755d74-7hb72&resourceVersion=1785987239400131000&timeoutSeconds=564&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" pod/percona-xtradb-cluster-operator-6699755d74-7hb72 condition met E0806 03:34:09.372954 1017051 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-6699755d74-7hb72&resourceVersion=1785987245277269000&timeoutSeconds=472&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/percona-xtradb-cluster-operator-6699755d74-7hb72 to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces scaling-25244 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "scaling-25244" not found waiting for namespace/scaling-25244 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "scaling-25244" not found ----------------------------------------------------------------------------------- create namespace scaling-25244 ----------------------------------------------------------------------------------- namespace/scaling-25244 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2525-c81e47f0-4-cluster4" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/do-spaces-secret created secret/gcp-cs-secret created secret/azure-secret created ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/scaling created ----------------------------------------------------------------------------------- waiting pods to be running for scaling ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 3 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met waiting for pod/scaling-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- Waiting for 2 running pods for scaling-proxysql ----------------------------------------------------------------------------------- pod/scaling-proxysql-0 condition met E0806 03:39:29.885836 1058695 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-0&resourceVersion=1785987565990024000&timeoutSeconds=550&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-0 to become Ready.Ok pod/scaling-proxysql-1 condition met E0806 03:39:38.298606 1059563 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-1&resourceVersion=1785987575990130000&timeoutSeconds=310&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-1 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/scaling to be ready ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:40:33.705887 1066688 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987631038245000&timeoutSeconds=380&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:40:59.427413 1069825 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987656150078000&timeoutSeconds=452&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- compare initial data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:41:50.119182 1076390 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987706828572000&timeoutSeconds=537&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:42:03.437581 1078264 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987721140055000&timeoutSeconds=309&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:42:16.279930 1080001 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987735248302000&timeoutSeconds=380&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- scale up from 3 to 5 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if all 5 Pods started ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 5 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met E0806 03:42:36.306326 1082371 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-0&resourceVersion=1785987754175791013&timeoutSeconds=402&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met E0806 03:42:46.165515 1083633 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-1&resourceVersion=1785987763340166000&timeoutSeconds=404&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met E0806 03:42:56.994193 1084942 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-2&resourceVersion=1785987772651615013&timeoutSeconds=532&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-2 to become Ready.Ok pod/scaling-pxc-3 condition met waiting for pod/scaling-pxc-3 to become Ready.Ok pod/scaling-pxc-4 condition met waiting for pod/scaling-pxc-4 to become Ready.Ok ----------------------------------------------------------------------------------- check if PVC created ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-08-06T03:45:40+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-08-06T03:45:43+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check data consistency for new Pods ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:45:51.525167 1107441 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987947392126000&timeoutSeconds=353&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-qrflx condition met E0806 03:46:13.455073 1110231 reflector.go:227] "Failed to watch" err="Get \"https://34.170.55.135/api/v1/namespaces/scaling-25244/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-qrflx&resourceVersion=1785987969302360000&timeoutSeconds=502&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-qrflx to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check new Pods exists in ProxySQL ----------------------------------------------------------------------------------- scaling-pxc-3.scaling-pxc.scaling-25244.svc.cluster.local scaling-pxc-3.scaling-pxc.scaling-25244.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-25244.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-25244.svc.cluster.local ----------------------------------------------------------------------------------- scale down from 5 to 3 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if Pod deleted ----------------------------------------------------------------------------------- waiting for pod/scaling-pxc-3 to be deleted.Error from server (NotFound): pods "scaling-pxc-3" not found waiting for pod/scaling-pxc-4 to be deletedError from server (NotFound): pods "scaling-pxc-4" not found ----------------------------------------------------------------------------------- check if PVC not deleted ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-08-06T03:47:30+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-08-06T03:47:35+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check if Pod deleted from ProxySQL ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling patched perconaxtradbcluster.pxc.percona.com "scaling" deleted from scaling-25244 namespace No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted namespace "pxc-operator" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. namespace "scaling-25244" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------