Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2193/e2e-tests/logs/proxy-switch-8-0.log Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 No resources found + kubectl patch pxc -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: resource(s) were provided, but no name was specified No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "pxc-operator" not found waiting for namespace/pxc-operator to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2193-1eb37b20-11-cluster7" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-75d958d548-zhj4x condition met pod/percona-xtradb-cluster-operator-75d958d548-zhj4x condition met waiting for pod/percona-xtradb-cluster-operator-75d958d548-zhj4x to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces proxy-switch-7671 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "proxy-switch-7671" not found waiting for namespace/proxy-switch-7671 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "proxy-switch-7671" not found ----------------------------------------------------------------------------------- create namespace proxy-switch-7671 ----------------------------------------------------------------------------------- namespace/proxy-switch-7671 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2193-1eb37b20-11-cluster7" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/gcp-cs-secret created secret/azure-secret created ----------------------------------------------------------------------------------- create PXC cluster with HAProxy ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/proxy-switch created ----------------------------------------------------------------------------------- check if all 3 Pods started ----------------------------------------------------------------------------------- error: no matching resources found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- Error from server (NotFound): pods "proxy-switch-haproxy-0" not found waiting for pod/proxy-switch-haproxy-0 to become Ready..................................Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-pxc-0 condition met waiting for pod/proxy-switch-pxc-0 to become Ready.Ok pod/proxy-switch-pxc-1 condition met waiting for pod/proxy-switch-pxc-1 to become Ready.Ok pod/proxy-switch-pxc-2 condition met waiting for pod/proxy-switch-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Unable to use a TTY - input is not a terminal or the right kind of file ----------------------------------------------------------------------------------- check cluster is ready with HAProxy ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-pxc-0 condition met waiting for pod/proxy-switch-pxc-0 to become Ready.Ok pod/proxy-switch-pxc-1 condition met waiting for pod/proxy-switch-pxc-1 to become Ready.Ok pod/proxy-switch-pxc-2 condition met waiting for pod/proxy-switch-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-haproxy-0 condition met waiting for pod/proxy-switch-haproxy-0 to become Ready.Ok pod/proxy-switch-haproxy-1 condition met waiting for pod/proxy-switch-haproxy-1 to become Ready.Ok pod/proxy-switch-haproxy-2 condition met waiting for pod/proxy-switch-haproxy-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/proxy-switch to be ready ----------------------------------------------------------------------------------- write data and check connectivity through HAProxy ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ERROR 1062 (23000) at line 1: Duplicate entry '100500' for key 'myApp.PRIMARY' command terminated with exit code 1 pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- verify data exists on all PXC nodes ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- switch from HAProxy to ProxySQL ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/proxy-switch patched ----------------------------------------------------------------------------------- wait for ProxySQL to be ready and HAProxy to be removed ----------------------------------------------------------------------------------- waiting for sts/proxy-switch-haproxy to be deletedError from server (NotFound): statefulsets.apps "proxy-switch-haproxy" not found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-proxysql-0 condition met waiting for pod/proxy-switch-proxysql-0 to become Ready.Ok pod/proxy-switch-proxysql-1 condition met waiting for pod/proxy-switch-proxysql-1 to become Ready.Ok pod/proxy-switch-proxysql-2 condition met waiting for pod/proxy-switch-proxysql-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/proxy-switch to be ready ----------------------------------------------------------------------------------- verify HAProxy pods are deleted ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- verify ProxySQL service exists ----------------------------------------------------------------------------------- NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE proxy-switch-proxysql ClusterIP 34.118.238.34 3306/TCP,33062/TCP,6070/TCP 64s ----------------------------------------------------------------------------------- check connectivity through ProxySQL after switch ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- write new data through ProxySQL ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- verify new data exists on all PXC nodes ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- verify ProxySQL is routing to primary ----------------------------------------------------------------------------------- ProxySQL is routing to primary: proxy-switch-pxc-0 ----------------------------------------------------------------------------------- switch back to HAProxy from ProxySQL ----------------------------------------------------------------------------------- perconaxtradbcluster.pxc.percona.com/proxy-switch patched ----------------------------------------------------------------------------------- wait for HAProxy to be ready and ProxySQL to be removed ----------------------------------------------------------------------------------- waiting for sts/proxy-switch-proxysql to be deleted.Error from server (NotFound): statefulsets.apps "proxy-switch-proxysql" not found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-haproxy-0 condition met waiting for pod/proxy-switch-haproxy-0 to become Ready.Ok pod/proxy-switch-haproxy-1 condition met waiting for pod/proxy-switch-haproxy-1 to become Ready.Ok pod/proxy-switch-haproxy-2 condition met waiting for pod/proxy-switch-haproxy-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/proxy-switch to be ready ----------------------------------------------------------------------------------- verify ProxySQL pods are deleted ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- check cluster is ready with HAProxy ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-pxc-0 condition met waiting for pod/proxy-switch-pxc-0 to become Ready.Ok pod/proxy-switch-pxc-1 condition met waiting for pod/proxy-switch-pxc-1 to become Ready.Ok pod/proxy-switch-pxc-2 condition met waiting for pod/proxy-switch-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/proxy-switch-haproxy-0 condition met waiting for pod/proxy-switch-haproxy-0 to become Ready.Ok pod/proxy-switch-haproxy-1 condition met waiting for pod/proxy-switch-haproxy-1 to become Ready.Ok pod/proxy-switch-haproxy-2 condition met waiting for pod/proxy-switch-haproxy-2 to become Ready.Ok ----------------------------------------------------------------------------------- wait cluster consistency ----------------------------------------------------------------------------------- waiting for pxc/proxy-switch to be ready ----------------------------------------------------------------------------------- verify HAProxy service exists ----------------------------------------------------------------------------------- NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE proxy-switch-haproxy ClusterIP 34.118.237.46 3306/TCP,3309/TCP,33062/TCP,33060/TCP,8404/TCP 116s ----------------------------------------------------------------------------------- write data and check connectivity through HAProxy ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- verify data exists on all PXC nodes ----------------------------------------------------------------------------------- pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-59944c5bbf-xxjx9 condition met waiting for pod/pxc-client-59944c5bbf-xxjx9 to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok ----------------------------------------------------------------------------------- clean up ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- destroy cluster/operator and all other resources ----------------------------------------------------------------------------------- + kubectl patch pxc -n proxy-switch-7671 proxy-switch --type=merge -p '{"metadata":{"finalizers":[]}}' perconaxtradbcluster.pxc.percona.com/proxy-switch patched perconaxtradbcluster.pxc.percona.com "proxy-switch" deleted from proxy-switch-7671 namespace No resources found No resources found validatingwebhookconfiguration.admissionregistration.k8s.io "percona-xtradbcluster-webhook" deleted ----------------------------------------------------------------------------------- test passed -----------------------------------------------------------------------------------