Log: /mnt/jenkins/workspace/cloud-psmdb-operator_PR-2227/e2e-tests/logs/demand-backup-incremental-sharded-gcp-s3.log Warning: version difference between client (1.35) and server (1.32) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.35) and server (1.32) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.35) and server (1.32) exceeds the supported minor version skew of +/-1 ----------------------------------------------------------------------------------- get and delete old CRDs and RBAC ----------------------------------------------------------------------------------- grep: warning: stray \ before - grep: warning: stray \ before - error: the server doesn't have a resource type "perconaservermongodbbackups" + kubectl patch perconaservermongodbbackups.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbbackups" error: the server doesn't have a resource type "perconaservermongodbrestores" + kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbrestores" error: the server doesn't have a resource type "perconaservermongodbs" + kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbs" Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found ----------------------------------------------------------------------------------- destroy chaos-mesh ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces psmdb-operator ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- create namespace psmdb-operator ----------------------------------------------------------------------------------- namespace/psmdb-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-2227-94dbdcd6-1-cluster14" modified. ----------------------------------------------------------------------------------- start PSMDB operator: perconalab/percona-server-mongodb-operator:PR-2227-94dbdcd6 ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaservermongodbbackups.psmdb.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaservermongodbrestores.psmdb.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaservermongodbs.psmdb.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-server-mongodb-operator created serviceaccount/percona-server-mongodb-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-server-mongodb-operator created deployment.apps/percona-server-mongodb-operator created waiting for pod/percona-server-mongodb-operator-84cfb447fb-c268p to be ready.OK Print operator info from log 2026-02-03T00:14:51.792Z INFO setup Manager starting up {"gitCommit": "94dbdcd604e08c657ed02d302eed8841e81611a6", "gitBranch": "PR-2227-94dbdcd6", "buildTime": "", "goVersion": "go1.25.6", "os": "linux", "arch": "amd64"} ----------------------------------------------------------------------------------- destroy chaos-mesh ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces demand-backup-incremental-sharded-gcp-s3-10699 ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- create namespace demand-backup-incremental-sharded-gcp-s3-10699 ----------------------------------------------------------------------------------- namespace/demand-backup-incremental-sharded-gcp-s3-10699 created Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-2227-94dbdcd6-1-cluster14" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/gcp-cs-secret created secret/azure-secret created secret/gcp-cs-sa-key-secret created ----------------------------------------------------------------------------------- Testing on sharded cluster ----------------------------------------------------------------------------------- [2026-02-03T00:15:31+0000] Creating PSMDB cluster secret/some-users created perconaservermongodb.psmdb.percona.com/some-name created deployment.apps/psmdb-client created [2026-02-03T00:15:37+0000] check if all pods started waiting for pod/some-name-rs0-0 to be ready........OK waiting for pod/some-name-rs0-1 to be ready......OK waiting for pod/some-name-rs0-2 to be ready......OK Waiting for cluster readyness........................................... waiting for pod/some-name-cfg-0 to be ready.OK waiting for pod/some-name-cfg-1 to be ready.OK waiting for pod/some-name-cfg-2 to be ready.OK Waiting for cluster readyness waiting for pod/some-name-mongos-0 to be ready.OK waiting for pod/some-name-mongos-1 to be ready.OK waiting for pod/some-name-mongos-2 to be ready.OK Waiting for cluster readyness waiting for cluster readyness.OK Percona Server for MongoDB shell version v4.4.29-28 connecting to: mongodb://some-name-mongos.demand-backup-incremental-sharded-gcp-s3-10699.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("23ca9be7-5989-421f-8149-1fa7a32ce7e8") } Percona Server for MongoDB server version: v8.0.17-6 WARNING: shell and server versions do not match Successfully added user: { "user" : "myApp", "roles" : [ { "db" : "myApp", "role" : "readWrite" } ] } bye Percona Server for MongoDB shell version v4.4.29-28 connecting to: mongodb://some-name-mongos.demand-backup-incremental-sharded-gcp-s3-10699.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("6c3dec83-0613-4ccb-8151-eb25b990febd") } Percona Server for MongoDB server version: v8.0.17-6 WARNING: shell and server versions do not match switched to db myApp WriteResult({ "nInserted" : 1 }) bye [2026-02-03T00:19:25+0000] running db.test.command() in myApp error: Internal error occurred: error sending request: Post "https://10.221.0.43:10250/exec/demand-backup-incremental-sharded-gcp-s3-10699/psmdb-client-54c7db9946-pkpmv/psmdb-client?command=bash&command=-c&command=printf+%27use+myApp%5Cn+db.test.find%28%29%5Cn%27+%7C+mongo+mongodb%3A%2F%2FmyApp%3AmyPass%40some-name-mongos.demand-backup-incremental-sharded-gcp-s3-10699.svc.cluster.local%3A27017%2Fadmin+&error=1&output=1": dial timeout, backstop 2c2,15 < { "_id" : , "x" : 100501 } --- > Error: error: { > "ok" : 0, > "errmsg" : "Encountered non-retryable error during query :: caused by :: Could not find host matching read preference { mode: \"primary\" } for set rs0", > "code" : 133, > "codeName" : "FailedToSatisfyReadPreference", > "$clusterTime" : { > "clusterTime" : Timestamp(1770078015, 1), > "signature" : { > "hash" : BinData(0,"hVCsffOh+2kcKflBfXrMR3CK2SY="), > "keyId" : NumberLong("7602426296735367181") > } > }, > "operationTime" : Timestamp(1770078015, 1) > }