Log: /mnt/jenkins/workspace/cloud-pxc-operator_PR-2349/e2e-tests/logs/scheduled-backup-8-0.log Warning: version difference between client (1.35) and server (1.31) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.35) and server (1.31) exceeds the supported minor version skew of +/-1 No resources found + kubectl patch pxc -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: resource(s) were provided, but no name was specified No resources found No resources found No resources found error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces pxc-operator ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "pxc-operator" not found waiting for namespace/pxc-operator to be deletedError from server (NotFound): namespaces "pxc-operator" not found ----------------------------------------------------------------------------------- create namespace pxc-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified namespace/pxc-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2349-b5e2b8a7-1-cluster6" modified. ----------------------------------------------------------------------------------- start PXC operator ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterbackups.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusterrestores.pxc.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaxtradbclusters.pxc.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-xtradb-cluster-operator unchanged serviceaccount/percona-xtradb-cluster-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-xtradb-cluster-operator unchanged deployment.apps/percona-xtradb-cluster-operator created service/percona-xtradb-cluster-operator created pod/percona-xtradb-cluster-operator-7c94dbdc94-9gk9d condition met pod/percona-xtradb-cluster-operator-7c94dbdc94-9gk9d condition met waiting for pod/percona-xtradb-cluster-operator-7c94dbdc94-9gk9d to become Ready.Ok error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces scheduled-backup-2987 ----------------------------------------------------------------------------------- Error from server (NotFound): namespaces "scheduled-backup-2987" not found waiting for namespace/scheduled-backup-2987 to be deletederror: resource(s) were provided, but no name was specified Error from server (NotFound): namespaces "scheduled-backup-2987" not found ----------------------------------------------------------------------------------- create namespace scheduled-backup-2987 ----------------------------------------------------------------------------------- namespace/scheduled-backup-2987 created Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2349-b5e2b8a7-1-cluster6" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/gcp-cs-secret created secret/azure-secret created "hashicorp" already exists with the same configuration, skipping "minio" already exists with the same configuration, skipping Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "hashicorp" chart repository ...Successfully got an update from the "percona" chart repository ...Successfully got an update from the "minio" chart repository ...Successfully got an update from the "chaos-mesh" chart repository Update Complete. ⎈Happy Helming!⎈ ----------------------------------------------------------------------------------- install Minio ----------------------------------------------------------------------------------- Error: uninstall: Release not loaded: minio-service: release: not found NAME: minio-service LAST DEPLOYED: Thu Jan 22 03:51:32 2026 NAMESPACE: scheduled-backup-2987 STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: MinIO can be accessed via port 9000 on the following DNS name from within your cluster: minio-service.scheduled-backup-2987.cluster.local To access MinIO from localhost, run the below commands: 1. export POD_NAME=$(kubectl get pods --namespace scheduled-backup-2987 -l "release=minio-service" -o jsonpath="{.items[0].metadata.name}") 2. kubectl port-forward $POD_NAME 9000 --namespace scheduled-backup-2987 Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/ You can now access MinIO server on http://localhost:9000. Follow the below steps to connect to MinIO server with mc client: 1. Download the MinIO mc client - https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart 2. export MC_HOST_minio-service-local=http://$(kubectl get secret --namespace scheduled-backup-2987 minio-service -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret --namespace scheduled-backup-2987 minio-service -o jsonpath="{.data.rootPassword}" | base64 --decode)@localhost:9000 3. mc ls minio-service-local pod/minio-service-55fcc5d75f-48tsw condition met waiting for pod/minio-service-55fcc5d75f-48tsw to become Ready.Ok make_bucket: operator-testing pod "aws-cli" deleted from scheduled-backup-2987 namespace All commands and output from this session will be recorded in container logs, including credentials and sensitive information passed through the command prompt. If you don't see a command prompt, try pressing enter. warning: couldn't attach to pod/aws-cli, falling back to streaming logs: Internal error occurred: unable to upgrade connection: container aws-cli not found in pod aws-cli_scheduled-backup-2987 priorityclass.scheduling.k8s.io/high-priority created ----------------------------------------------------------------------------------- create first PXC cluster ----------------------------------------------------------------------------------- secret/my-cluster-secrets created secret/some-name-ssl created secret/some-name-ssl-internal created deployment.apps/pxc-client created perconaxtradbcluster.pxc.percona.com/scheduled-backup created ----------------------------------------------------------------------------------- check if all 3 Pods started ----------------------------------------------------------------------------------- error: no matching resources found ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/scheduled-backup-proxysql-0 condition met waiting for pod/scheduled-backup-proxysql-0 to become Ready.Ok ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/scheduled-backup-pxc-0 condition met waiting for pod/scheduled-backup-pxc-0 to become Ready.Ok pod/scheduled-backup-pxc-1 condition met waiting for pod/scheduled-backup-pxc-1 to become Ready.Ok pod/scheduled-backup-pxc-2 condition met waiting for pod/scheduled-backup-pxc-2 to become Ready.Ok ----------------------------------------------------------------------------------- write data ----------------------------------------------------------------------------------- pod/pxc-client-c75dc5c46-xrkfw condition met waiting for pod/pxc-client-c75dc5c46-xrkfw to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-c75dc5c46-xrkfw condition met waiting for pod/pxc-client-c75dc5c46-xrkfw to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-c75dc5c46-xrkfw condition met waiting for pod/pxc-client-c75dc5c46-xrkfw to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-c75dc5c46-xrkfw condition met waiting for pod/pxc-client-c75dc5c46-xrkfw to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok pod/pxc-client-c75dc5c46-xrkfw condition met waiting for pod/pxc-client-c75dc5c46-xrkfw to become ReadyDefaulted container "pxc-client" out of: pxc-client, backup .Ok Unable to use a TTY - input is not a terminal or the right kind of file ----------------------------------------------------------------------------------- add backups schedule for pvc storage ----------------------------------------------------------------------------------- Context "gke_cloud-dev-112233_us-central1-a_jen-pxc-2349-b5e2b8a7-1-cluster6" modified. perconaxtradbcluster.pxc.percona.com/scheduled-backup configured node/gke-jen-pxc-2349-b5e2b8a-default-pool-baa7e952-1qff labeled perconaxtradbcluster.pxc.percona.com/scheduled-backup configured ----------------------------------------------------------------------------------- wait for running cluster ----------------------------------------------------------------------------------- pod/scheduled-backup-pxc-0 condition met waiting for pod/scheduled-backup-pxc-0 to become Ready.Ok error: timed out waiting for the condition on pods/scheduled-backup-pxc-1 waiting for pod/scheduled-backup-pxc-1 to become Ready.Name: scheduled-backup-pxc-1 Namespace: scheduled-backup-2987 Priority: 0 Service Account: default Node: Labels: app.kubernetes.io/component=pxc app.kubernetes.io/instance=scheduled-backup app.kubernetes.io/managed-by=percona-xtradb-cluster-operator app.kubernetes.io/name=percona-xtradb-cluster app.kubernetes.io/part-of=percona-xtradb-cluster apps.kubernetes.io/pod-index=1 controller-revision-hash=scheduled-backup-pxc-6b7dc488cf statefulset.kubernetes.io/pod-name=scheduled-backup-pxc-1 Annotations: cloud.google.com/cluster_autoscaler_unhelpable_since: 2026-01-22T03:59:30+0000 cloud.google.com/cluster_autoscaler_unhelpable_until: Inf kubectl.kubernetes.io/default-container: pxc percona.com/configuration-hash: d41d8cd98f00b204e9800998ecf8427e percona.com/ssl-hash: cfd6a52398268173b51d7cdb1331c09a percona.com/ssl-internal-hash: 751e59a1c1d94ac67f9d6bb6e756acdc Status: Pending IP: IPs: Controlled By: StatefulSet/scheduled-backup-pxc Init Containers: pxc-init: Image: perconalab/percona-xtradb-cluster-operator:PR-2349-b5e2b8a7 Port: Host Port: Command: /pxc-init-entrypoint.sh Limits: cpu: 50m memory: 50M Requests: cpu: 50m memory: 50M Environment: Mounts: /var/lib/mysql from datadir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vqhrb (ro) Containers: pxc: Image: perconalab/percona-xtradb-cluster-operator:main-pxc8.0 Ports: 3306/TCP (mysql), 4444/TCP (sst), 4567/TCP (write-set), 4568/TCP (ist), 33062/TCP (mysql-admin), 33060/TCP (mysqlx) Host Ports: 0/TCP (mysql), 0/TCP (sst), 0/TCP (write-set), 0/TCP (ist), 0/TCP (mysql-admin), 0/TCP (mysqlx) Command: /var/lib/mysql/pxc-entrypoint.sh Args: mysqld Liveness: exec [/var/lib/mysql/liveness-check.sh] delay=300s timeout=5s period=10s #success=1 #failure=3 Readiness: exec [/var/lib/mysql/readiness-check.sh] delay=15s timeout=15s period=30s #success=1 #failure=5 Environment Variables from: scheduled-backup-env-vars-pxc Secret Optional: true Environment: PXC_SERVICE: scheduled-backup-pxc-unready MONITOR_HOST: % MYSQL_ROOT_PASSWORD: Optional: false XTRABACKUP_PASSWORD: Optional: false MONITOR_PASSWORD: Optional: false CLUSTER_HASH: 1389204 OPERATOR_ADMIN_PASSWORD: Optional: false LIVENESS_CHECK_TIMEOUT: 5 READINESS_CHECK_TIMEOUT: 15 DEFAULT_AUTHENTICATION_PLUGIN: caching_sha2_password MYSQL_NOTIFY_SOCKET: /var/lib/mysql/notify.sock MYSQL_STATE_FILE: /var/lib/mysql/mysql.state Mounts: /etc/my.cnf.d from auto-config (rw) /etc/mysql/init-file from mysql-init-file (rw) /etc/mysql/mysql-users-secret from mysql-users-secret-file (rw) /etc/mysql/ssl from ssl (rw) /etc/mysql/ssl-internal from ssl-internal (rw) /etc/mysql/vault-keyring-secret from vault-keyring-secret (rw) /etc/percona-xtradb-cluster.conf.d from config (rw) /tmp from tmp (rw) /var/lib/mysql from datadir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vqhrb (ro) Conditions: Type Status PodScheduled False Volumes: datadir: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: datadir-scheduled-backup-pxc-1 ReadOnly: false tmp: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: config: Type: ConfigMap (a volume populated by a ConfigMap) Name: scheduled-backup-pxc Optional: true ssl-internal: Type: Secret (a volume populated by a Secret) SecretName: some-name-ssl-internal Optional: true ssl: Type: Secret (a volume populated by a Secret) SecretName: some-name-ssl Optional: false auto-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: auto-scheduled-backup-pxc Optional: true vault-keyring-secret: Type: Secret (a volume populated by a Secret) SecretName: scheduled-backup-vault Optional: true mysql-users-secret-file: Type: Secret (a volume populated by a Secret) SecretName: internal-scheduled-backup Optional: false mysql-init-file: Type: Secret (a volume populated by a Secret) SecretName: scheduled-backup-mysql-init Optional: true kube-api-access-vqhrb: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt Optional: false DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 14s default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Warning FailedScheduling 11s (x2 over 13s) default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Normal NotTriggerScaleUp 14s cluster-autoscaler pod didn't trigger scale-up: Error from server: Get "https://10.213.0.32:10250/containerLogs/pxc-operator/percona-xtradb-cluster-operator-7c94dbdc94-9gk9d/percona-xtradb-cluster-operator": http2: client connection lost