Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2476/e2e-tests/logs/users-5-7.log Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.36) and server (1.33) exceeds the supported minor version skew of +/-1 No resources found + kubectl patch pxc -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: resource(s) were provided, but no name was specified No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified namespace "pxc-operator" deleted waiting for namespace/pxc-operator to be deletedError from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2476-a8b01a39-5-cluster11" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-8548fd5788-rf9gc condition met E0516 21:39:41.907713 2112 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-8548fd5788-rf9gc&resourceVersion=1778967581547207000&timeoutSeconds=538&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" pod/percona-xtradb-cluster-operator-8548fd5788-rf9gc condition met E0516 21:39:47.896518 3013 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/pxc-operator/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpercona-xtradb-cluster-operator-8548fd5788-rf9gc&resourceVersion=1778967585406201000&timeoutSeconds=396&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/percona-xtradb-cluster-operator-8548fd5788-rf9gc to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces users-553 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "users-553" not found waiting for namespace/users-553 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "users-553" not found ----------------------------------------------------------------------------------- create namespace users-553 ----------------------------------------------------------------------------------- namespace/users-553 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2476-a8b01a39-5-cluster11" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/do-spaces-secret created secret/gcp-cs-secret created secret/azure-secret created ----------------------------------------------------------------------------------- create PXC cluster with 1-password secret ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/some-name created ----------------------------------------------------------------------------------- check if all 3 Pods started ----------------------------------------------------------------------------------- error: no matching resources found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/some-name-proxysql-0 condition met waiting for pod/some-name-proxysql-0 to become Ready.Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/some-name-pxc-0 condition met waiting for pod/some-name-pxc-0 to become Ready.Ok pod/some-name-pxc-1 condition met waiting for pod/some-name-pxc-1 to become Ready.Ok pod/some-name-pxc-2 condition met waiting for pod/some-name-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:44:42.873014 2021 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967881263534000&timeoutSeconds=589&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:44:48.980501 2930 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967888235573000&timeoutSeconds=495&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:45:26.667739 8080 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967925752893000&timeoutSeconds=380&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:45:33.785591 9145 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967932771527000&timeoutSeconds=340&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:45:42.484657 10256 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967940869464000&timeoutSeconds=569&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Unable to use a TTY - input is not a terminal or the right kind of file ----------------------------------------------------------------------------------- test missing passwords were created and present in internal secrets ----------------------------------------------------------------------------------- Checking root Running compare for root pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:45:56.137565 12105 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967954327931000&timeoutSeconds=470&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Checking xtrabackup Running compare for xtrabackup pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:46:03.457279 12986 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967962109322000&timeoutSeconds=442&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Checking monitor Running compare for monitor pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:46:09.878407 13671 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967969136725000&timeoutSeconds=599&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Checking proxyadmin Running compare for proxyadmin Checking operator Running compare for operator pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:46:17.605981 14373 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967976824479000&timeoutSeconds=335&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Checking replication Running compare for replication pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:46:26.693604 15362 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778967984485318000&timeoutSeconds=597&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- test root ----------------------------------------------------------------------------------- secret/my-cluster-secrets patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:47:19.533937 21101 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968038200540000&timeoutSeconds=586&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- test proxyadmin ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/some-name patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready. secret/my-cluster-secrets patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready..... ----------------------------------------------------------------------------------- test xtrabackup ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/some-name patched secret/my-cluster-secrets patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready...................... ----------------------------------------------------------------------------------- test monitor ----------------------------------------------------------------------------------- secret/my-cluster-secrets patched Skipping dual password feature doesn't work for 5.7. PXC 5.7 doesn't support it! ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready..... pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:53:30.450281 32302 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968409660129000&timeoutSeconds=380&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- test operator ----------------------------------------------------------------------------------- secret/my-cluster-secrets patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready.. pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:54:35.647032 7385 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968474873057000&timeoutSeconds=478&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- change secret name ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/some-name patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready........................ ----------------------------------------------------------------------------------- test new operator ----------------------------------------------------------------------------------- secret/my-cluster-secrets-2 patched ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready.... pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:58:52.108447 31808 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968730157691000&timeoutSeconds=348&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- test new users sync ----------------------------------------------------------------------------------- pod/pxc-client-97cb9c68b-84f4q condition met E0516 21:58:59.081758 32664 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968737622471000&timeoutSeconds=539&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-97cb9c68b-84f4q condition met E0516 22:00:26.981380 6557 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968826213798000&timeoutSeconds=500&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- check secret without operator ----------------------------------------------------------------------------------- secret/my-cluster-secrets-2 configured Warning: resource secrets/my-cluster-secrets-2 is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. pod/pxc-client-97cb9c68b-84f4q condition met E0516 22:01:33.356099 10582 reflector.go:227] "Failed to watch" err="Get \"https://136.112.216.179/api/v1/namespaces/users-553/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dpxc-client-97cb9c68b-84f4q&resourceVersion=1778968892616244000&timeoutSeconds=391&watch=true\": context canceled" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" type="*unstructured.Unstructured" waiting for pod/pxc-client-97cb9c68b-84f4q to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok perconaxtradbcluster.pxc.percona.com/some-name configured ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/some-name to be ready...................................../mnt/jenkins/workspace/cloud-pxc-operator_PR-2476/e2e-tests/users/../functions: line 152: 21849 Terminated sleep 5