Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2538/e2e-tests/logs/scaling-8-0.log Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified namespace "pxc-operator" deleted waiting for namespace/pxc-operator to be deletedError from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2538-70c04858-1-cluster3" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com condition met E0702 03:27:44.853642 11123 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusters.pxc.percona.com&resourceVersion=1782962860338380000&timeoutSeconds=326&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com condition met E0702 03:27:47.976111 11584 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterbackups.pxc.percona.com&resourceVersion=1782962865338462000&timeoutSeconds=585&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com condition met E0702 03:27:50.854486 11994 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterrestores.pxc.percona.com&resourceVersion=1782962870338471000&timeoutSeconds=389&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-59848dbdcc-7q2vs condition met E0702 03:28:19.370162 16033 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-59848dbdcc-7q2vs&resourceVersion=1782962899011683000&timeoutSeconds=377&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" pod/percona-xtradb-cluster-operator-59848dbdcc-7q2vs condition met E0702 03:28:30.693866 17666 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-59848dbdcc-7q2vs&resourceVersion=1782962906343842000&timeoutSeconds=325&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/percona-xtradb-cluster-operator-59848dbdcc-7q2vs to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces scaling-16880 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "scaling-16880" not found waiting for namespace/scaling-16880 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "scaling-16880" not found ----------------------------------------------------------------------------------- create namespace scaling-16880 ----------------------------------------------------------------------------------- namespace/scaling-16880 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2538-70c04858-1-cluster3" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/do-spaces-secret created secret/gcp-cs-secret created secret/azure-secret created ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/scaling created ----------------------------------------------------------------------------------- waiting pods to be running for scaling ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 3 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met waiting for pod/scaling-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- Waiting for 2 running pods for scaling-proxysql ----------------------------------------------------------------------------------- pod/scaling-proxysql-0 condition met E0702 03:33:58.945732 1443 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-0&resourceVersion=1782963238275677000&timeoutSeconds=582&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-0 to become Ready.Ok pod/scaling-proxysql-1 condition met E0702 03:34:13.365619 3584 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-proxysql-1&resourceVersion=1782963248275743000&timeoutSeconds=543&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-proxysql-1 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/scaling to be ready ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:35:06.596841 12038 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963305287198000&timeoutSeconds=539&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:35:39.630646 16471 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963334861960000&timeoutSeconds=302&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- compare initial data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:36:37.252246 25105 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963394108958000&timeoutSeconds=495&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:36:52.808594 27421 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963410168302000&timeoutSeconds=591&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:37:10.744824 29970 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963427080349000&timeoutSeconds=446&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- scale up from 3 to 5 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if all 5 Pods started ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 5 running pods for scaling-pxc ----------------------------------------------------------------------------------- pod/scaling-pxc-0 condition met E0702 03:37:41.194933 1863 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-0&resourceVersion=1782963457378943005&timeoutSeconds=349&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-0 to become Ready.Ok pod/scaling-pxc-1 condition met E0702 03:37:57.369326 4230 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-1&resourceVersion=1782963475896559005&timeoutSeconds=524&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-1 to become Ready.Ok pod/scaling-pxc-2 condition met E0702 03:38:08.881984 5994 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dscaling-pxc-2&resourceVersion=1782963485900074000&timeoutSeconds=536&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/scaling-pxc-2 to become Ready.Ok pod/scaling-pxc-3 condition met waiting for pod/scaling-pxc-3 to become Ready.Ok pod/scaling-pxc-4 condition met waiting for pod/scaling-pxc-4 to become Ready.Ok ----------------------------------------------------------------------------------- check if PVC created ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-07-02T03:40:44+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-07-02T03:40:46+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check data consistency for new Pods ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:40:51.598907 30032 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963648652187000&timeoutSeconds=447&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-r2hqm condition met E0702 03:41:04.148794 32203 reflector.go:227] "Failed to watch" err="Get \"https://34.28.66.196/api/v1/namespaces/scaling-16880/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-r2hqm&resourceVersion=1782963663141898000&timeoutSeconds=330&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-r2hqm to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check new Pods exists in ProxySQL ----------------------------------------------------------------------------------- scaling-pxc-3.scaling-pxc.scaling-16880.svc.cluster.local scaling-pxc-3.scaling-pxc.scaling-16880.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-16880.svc.cluster.local scaling-pxc-4.scaling-pxc.scaling-16880.svc.cluster.local ----------------------------------------------------------------------------------- scale down from 5 to 3 ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling configured ----------------------------------------------------------------------------------- check if Pod deleted ----------------------------------------------------------------------------------- waiting for pod/scaling-pxc-3 to be deleted.Error from server (NotFound): pods "scaling-pxc-3" not found waiting for pod/scaling-pxc-4 to be deletedError from server (NotFound): pods "scaling-pxc-4" not found ----------------------------------------------------------------------------------- check if PVC not deleted ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-3- ----------------------------------------------------------------------------------- [2026-07-02T03:42:12+0000] compare_kubectl: pvc/datadir-scaling-pxc-3 OK ----------------------------------------------------------------------------------- compare pvc/datadir-scaling-pxc-4- ----------------------------------------------------------------------------------- [2026-07-02T03:42:14+0000] compare_kubectl: pvc/datadir-scaling-pxc-4 OK ----------------------------------------------------------------------------------- check if Pod deleted from ProxySQL ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/scaling patched perconaxtradbcluster.pxc.percona.com "scaling" deleted from scaling-16880 namespace No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted namespace "scaling-16880" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------