Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2538/e2e-tests/logs/storage-8-0.log Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified namespace "pxc-operator" deleted waiting for namespace/pxc-operator to be deletedError from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2538-70c04858-1-cluster9" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com condition met E0702 04:22:43.355796 23529 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusters.pxc.percona.com&resourceVersion=1782966160256968000&timeoutSeconds=593&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com condition met E0702 04:22:45.560028 23925 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterbackups.pxc.percona.com&resourceVersion=1782966165257025000&timeoutSeconds=514&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com condition met E0702 04:22:48.221888 24301 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/apis/apiextensions.k8s.io/v1/customresourcedefinitions?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dperconaxtradbclusterrestores.pxc.percona.com&resourceVersion=1782966165257025000&timeoutSeconds=314&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-59848dbdcc-2pxm4 condition met E0702 04:23:09.256933 27308 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-59848dbdcc-2pxm4&resourceVersion=1782966188899579000&timeoutSeconds=560&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" pod/percona-xtradb-cluster-operator-59848dbdcc-2pxm4 condition met E0702 04:23:14.189015 27976 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-59848dbdcc-2pxm4&resourceVersion=1782966191536719000&timeoutSeconds=334&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/percona-xtradb-cluster-operator-59848dbdcc-2pxm4 to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces storage-28723 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "storage-28723" not found waiting for namespace/storage-28723 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "storage-28723" not found ----------------------------------------------------------------------------------- create namespace storage-28723 ----------------------------------------------------------------------------------- namespace/storage-28723 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2538-70c04858-1-cluster9" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/do-spaces-secret created secret/gcp-cs-secret created secret/azure-secret created deployment.apps/pxc-client created secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created daemonset.apps/hostpath-helper created ----------------------------------------------------------------------------------- check emptydir ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- apply cr config ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets unchanged secret/some-name-ssl unchanged secret/some-name-ssl-internal unchanged deployment.apps/pxc-client unchanged perconaxtradbcluster.pxc.percona.com/emptydir created ----------------------------------------------------------------------------------- waiting pods to be running for emptydir ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 3 running pods for emptydir-pxc ----------------------------------------------------------------------------------- pod/emptydir-pxc-0 condition met waiting for pod/emptydir-pxc-0 to become Ready.Ok pod/emptydir-pxc-1 condition met waiting for pod/emptydir-pxc-1 to become Ready.Ok pod/emptydir-pxc-2 condition met waiting for pod/emptydir-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- Waiting for 2 running pods for emptydir-proxysql ----------------------------------------------------------------------------------- pod/emptydir-proxysql-0 condition met E0702 04:27:34.756197 635 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Demptydir-proxysql-0&resourceVersion=1782966453645061000&timeoutSeconds=552&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/emptydir-proxysql-0 to become Ready.Ok pod/emptydir-proxysql-1 condition met E0702 04:27:47.914066 2579 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Demptydir-proxysql-1&resourceVersion=1782966463645441000&timeoutSeconds=431&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/emptydir-proxysql-1 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/emptydir to be ready ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:28:38.487440 10390 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966515189955000&timeoutSeconds=577&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:29:07.483565 14778 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966544158468000&timeoutSeconds=467&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- compare initial data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:30:02.644494 23365 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966599637600000&timeoutSeconds=359&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:30:12.557681 24970 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966610585285000&timeoutSeconds=542&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:30:21.275243 26250 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966619451314000&timeoutSeconds=440&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check if statefulset created with expected config ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare statefulset/emptydir-pxc- ----------------------------------------------------------------------------------- [2026-07-02T04:30:31+0000] compare_kubectl: statefulset/emptydir-pxc OK ----------------------------------------------------------------------------------- compare statefulset/emptydir-proxysql- ----------------------------------------------------------------------------------- [2026-07-02T04:30:34+0000] compare_kubectl: statefulset/emptydir-proxysql OK ----------------------------------------------------------------------------------- delete cluster ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com "emptydir" deleted from storage-28723 namespace ----------------------------------------------------------------------------------- check hostpath ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- apply cr config ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets unchanged secret/some-name-ssl unchanged secret/some-name-ssl-internal unchanged deployment.apps/pxc-client unchanged perconaxtradbcluster.pxc.percona.com/hostpath created ----------------------------------------------------------------------------------- waiting pods to be running for hostpath ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Waiting for 3 running pods for hostpath-pxc ----------------------------------------------------------------------------------- pod/hostpath-pxc-0 condition met waiting for pod/hostpath-pxc-0 to become Ready.Ok pod/hostpath-pxc-1 condition met waiting for pod/hostpath-pxc-1 to become Ready.Ok pod/hostpath-pxc-2 condition met waiting for pod/hostpath-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- Waiting for 2 running pods for hostpath-proxysql ----------------------------------------------------------------------------------- pod/hostpath-proxysql-0 condition met E0702 04:33:47.342753 23147 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dhostpath-proxysql-0&resourceVersion=1782966826935068000&timeoutSeconds=502&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/hostpath-proxysql-0 to become Ready.Ok pod/hostpath-proxysql-1 condition met E0702 04:33:55.265636 24436 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dhostpath-proxysql-1&resourceVersion=1782966831935555000&timeoutSeconds=347&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/hostpath-proxysql-1 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/hostpath to be ready ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:34:28.692177 28752 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966866940592000&timeoutSeconds=498&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:34:44.149487 31002 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966882160375000&timeoutSeconds=343&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- compare initial data ----------------------------------------------------------------------------------- pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:35:29.458139 5176 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966927852786000&timeoutSeconds=513&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:35:39.111697 6473 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966937020925000&timeoutSeconds=573&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-67fc4995bb-npxw9 condition met E0702 04:35:50.164786 8217 reflector.go:227] "Failed to watch" err="Get \"https://34.60.240.116/api/v1/namespaces/storage-28723/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-67fc4995bb-npxw9&resourceVersion=1782966947817486000&timeoutSeconds=350&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-67fc4995bb-npxw9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check if statefulset created with expected config ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- compare statefulset/hostpath-pxc- ----------------------------------------------------------------------------------- [2026-07-02T04:36:01+0000] compare_kubectl: statefulset/hostpath-pxc OK ----------------------------------------------------------------------------------- compare statefulset/hostpath-proxysql- ----------------------------------------------------------------------------------- [2026-07-02T04:36:03+0000] compare_kubectl: statefulset/hostpath-proxysql OK ----------------------------------------------------------------------------------- delete cluster ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com "hostpath" deleted from storage-28723 namespace ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted namespace "storage-28723" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------