Log: /mnt/jenkins/workspace/cloud-psmdb-operator_PR-2044/e2e-tests/logs/demand-backup-physical-sharded-gcp-native.log grep: warning: stray \ before - Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1 ----------------------------------------------------------------------------------- get and delete old CRDs and RBAC ----------------------------------------------------------------------------------- grep: warning: stray \ before - grep: warning: stray \ before - error: the server doesn't have a resource type "perconaservermongodbbackups" + kubectl patch perconaservermongodbbackups.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbbackups" error: the server doesn't have a resource type "perconaservermongodbrestores" + kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbrestores" error: the server doesn't have a resource type "perconaservermongodbs" + kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}' error: the server doesn't have a resource type "perconaservermongodbs" Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found ----------------------------------------------------------------------------------- destroy chaos-mesh ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces psmdb-operator ----------------------------------------------------------------------------------- egrep: warning: egrep is obsolescent; using grep -E error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- create namespace psmdb-operator ----------------------------------------------------------------------------------- namespace/psmdb-operator created Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-2044-6293f902-6-cluster5" modified. ----------------------------------------------------------------------------------- start PSMDB operator: perconalab/percona-server-mongodb-operator:PR-2044-6293f902 ----------------------------------------------------------------------------------- customresourcedefinition.apiextensions.k8s.io/perconaservermongodbbackups.psmdb.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaservermongodbrestores.psmdb.percona.com serverside-applied customresourcedefinition.apiextensions.k8s.io/perconaservermongodbs.psmdb.percona.com serverside-applied clusterrole.rbac.authorization.k8s.io/percona-server-mongodb-operator created serviceaccount/percona-server-mongodb-operator created clusterrolebinding.rbac.authorization.k8s.io/service-account-percona-server-mongodb-operator created deployment.apps/percona-server-mongodb-operator created waiting for pod/percona-server-mongodb-operator-55597d55d4-rthb8 to be ready.OK Print operator info from log 2025-11-17T12:05:04.706Z INFO setup Manager starting up {"gitCommit": "6293f90271409106d42d5ed27016494a08db3282", "gitBranch": "PR-2044-6293f902", "buildTime": "", "goVersion": "go1.25.4", "os": "linux", "arch": "amd64"} ----------------------------------------------------------------------------------- destroy chaos-mesh ----------------------------------------------------------------------------------- error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- cleaned up all old namespaces ----------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- cleaned up old namespaces demand-backup-physical-sharded-gcp-native-5662 ----------------------------------------------------------------------------------- egrep: warning: egrep is obsolescent; using grep -E error: resource(s) were provided, but no name was specified ----------------------------------------------------------------------------------- create namespace demand-backup-physical-sharded-gcp-native-5662 ----------------------------------------------------------------------------------- namespace/demand-backup-physical-sharded-gcp-native-5662 created Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-2044-6293f902-6-cluster5" modified. ----------------------------------------------------------------------------------- create secrets for cloud storages ----------------------------------------------------------------------------------- secret/minio-secret created secret/aws-s3-secret created secret/gcp-cs-secret created secret/azure-secret created secret/gcp-cs-sa-key-secret created ----------------------------------------------------------------------------------- Testing on sharded cluster ----------------------------------------------------------------------------------- Creating PSMDB cluster secret/some-users created perconaservermongodb.psmdb.percona.com/some-name created deployment.apps/psmdb-client created Check if all pods started waiting for pod/some-name-rs0-0 to be ready........OK waiting for pod/some-name-rs0-1 to be ready.......OK waiting for pod/some-name-rs0-2 to be ready.........OK Waiting for cluster readyness......................... waiting for pod/some-name-cfg-0 to be ready.OK waiting for pod/some-name-cfg-1 to be ready.OK waiting for pod/some-name-cfg-2 to be ready.OK Waiting for cluster readyness waiting for pod/some-name-mongos-0 to be ready.OK waiting for pod/some-name-mongos-1 to be ready.OK waiting for pod/some-name-mongos-2 to be ready.OK Waiting for cluster readyness waiting for cluster readyness.OK Percona Server for MongoDB shell version v4.4.29-28 connecting to: mongodb://some-name-mongos.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("98a98890-a3ce-423b-8d9f-6c6da3a7f375") } Percona Server for MongoDB server version: v8.0.12-4 WARNING: shell and server versions do not match Successfully added user: { "user" : "myApp", "roles" : [ { "db" : "myApp", "role" : "readWrite" } ] } bye Percona Server for MongoDB shell version v4.4.29-28 connecting to: mongodb://some-name-mongos.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("d948ae7d-21e5-4755-b0b6-20ed84d20b73") } Percona Server for MongoDB server version: v8.0.12-4 WARNING: shell and server versions do not match switched to db myApp WriteResult({ "nInserted" : 1 }) bye egrep: warning: egrep is obsolescent; using grep -E Waiting 60 seconds for stable timestamp in wiredtiger Running GCP storage backups ----------------------------------------------------------------------------------- run backup backup-gcp-cs-sa-sharded ----------------------------------------------------------------------------------- perconaservermongodbbackup.psmdb.percona.com/backup-gcp-cs-sa-sharded created waiting for backup-gcp-cs-sa-sharded to reach ready state...............OK [2025-11-17T12:10:43+0000] checking if https://storage.googleapis.com/operator-testing/psmdb-demand-backup-physical-sharded/2025-11-17T12:10:15Z/rs0/filelist.pbm exists % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 9126 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 9126 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 HTTP/2 200 content-type: application/octet-stream x-guploader-uploadid: AOCedOH8Hd2HqlSaLNLHRvf3p1vYDU9T4dciG1BxPEhXqNtnh36K8aUm_nOi4wfWVHI-y1u5mrsiwp0 expires: Mon, 17 Nov 2025 13:10:43 GMT date: Mon, 17 Nov 2025 12:10:43 GMT cache-control: public, max-age=3600 last-modified: Mon, 17 Nov 2025 12:10:35 GMT etag: "020652a586b578f3bc028662e8bde8dc" x-goog-generation: 1763381435971784 x-goog-metageneration: 1 x-goog-stored-content-encoding: identity x-goog-stored-content-length: 9126 x-goog-hash: crc32c=orfNCw== x-goog-hash: md5=AgZSpYa1ePO8AoZi6L3o3A== x-goog-expiration: Tue, 18 Nov 2025 12:10:35 GMT x-goog-storage-class: REGIONAL accept-ranges: bytes content-length: 9126 server: UploadServer alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 [2025-11-17T12:10:43+0000] checking if https://storage.googleapis.com/operator-testing/psmdb-demand-backup-physical-sharded/2025-11-17T12:10:15Z/cfg/filelist.pbm exists % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 17484 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 HTTP/2 200 content-type: application/octet-stream x-guploader-uploadid: AOCedOFXUuXdEaz3ghxxGKEWHHi-OoKsMwFnjSi2v5X8ExY0-OIHQ463TpVuu2JlzPLo3JWU2qU2aDQ expires: Mon, 17 Nov 2025 13:10:43 GMT date: Mon, 17 Nov 2025 12:10:43 GMT cache-control: public, max-age=3600 last-modified: Mon, 17 Nov 2025 12:10:38 GMT etag: "1489f3f7b47631846473cb6928639042" x-goog-generation: 1763381438279607 x-goog-metageneration: 1 x-goog-stored-content-encoding: identity x-goog-stored-content-length: 17484 x-goog-hash: crc32c=Lehfng== x-goog-hash: md5=FInz97R2MYRkc8tpKGOQQg== x-goog-expiration: Tue, 18 Nov 2025 12:10:38 GMT x-goog-storage-class: REGIONAL accept-ranges: bytes content-length: 17484 server: UploadServer alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Drop collection Percona Server for MongoDB shell version v4.4.29-28 connecting to: mongodb://some-name-mongos.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("75d58d27-2f72-4b4f-aedf-df88cf44906e") } Percona Server for MongoDB server version: v8.0.12-4 WARNING: shell and server versions do not match switched to db myApp true bye Check backup and restore -- gcp-cs-sa perconaservermongodbrestore.psmdb.percona.com/restore-backup-gcp-cs-sa-sharded created Waiting for the psmdb-restore/restore-backup-gcp-cs-sa-sharded object to be created.OK Waiting psmdb-restore/restore-backup-gcp-cs-sa-sharded to reach state "requested" .....OK after 4 minutes [2025-11-17T12:15:01+0000] compare_kubectl: statefulset/some-name-rs0 OK Waiting for the psmdb-restore/restore-backup-gcp-cs-sa-sharded object to be created.OK Waiting psmdb-restore/restore-backup-gcp-cs-sa-sharded to reach state "ready" .................................................. ----------------------------------------------------------------------------------- pbm-agent logs from some-name-rs0-0 ----------------------------------------------------------------------------------- 2025/11/17 12:33:41 [entrypoint] starting `pbm-agent` 2025-11-17T12:33:53.000+0000 I % _____ %%% | __ \ ###%%%%%%%%%%%%* | |__) |__ _ __ ___ ___ _ __ __ _ ### ##%% %%%% | ___/ _ \ '__/ __/ _ \| '_ \ / _` | #### ##% %%%% | | | __/ | | (_| (_) | | | | (_| | ### #### %%% |_| \___|_| \___\___/|_| |_|\__,_| ,((### ### %%% _____ _ (((( (### #### %%%% / ____| | | ((( ((# ###### | (___ __ _ _ _ __ _ __| | (((( (((# #### \___ \ / _` | | | |/ _` |/ _` | /(( ,((( *### ____) | (_| | |_| | (_| | (_| | //// ((( #### |_____/ \__, |\__,_|\__,_|\__,_| /// (((( #### | | /////////////(((((((((((((((((######## |_| Join @ squad.percona.com/mongodb ** Join Percona Squad! ** Participate in monthly SWAG raffles, get early access to new product features, invite-only ”ask me anything” sessions with database performance experts. Interested? Fill in the form at squad.percona.com/mongodb 2025-11-17T12:33:53.000+0000 I log options: log-path=/dev/stderr, log-level:D, log-json:false 2025-11-17T12:33:53.000+0000 I pbm-agent: Version: 2.12.0 Platform: linux/amd64 GitCommit: 2f7100a2a1da2bb7074e15d757693999cac7574e GitBranch: release-2.12.0 BuildTime: 2025-10-31_09:21_UTC GoVersion: go1.25.3 2025-11-17T12:33:53.000+0000 I starting PITR routine 2025-11-17T12:33:53.000+0000 I node: rs0/some-name-rs0-0.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017 2025-11-17T12:33:53.000+0000 I conn level ReadConcern: majority; WriteConcern: majority 2025-11-17T12:33:53.000+0000 I listening for the commands Defaulted container "mongod" out of: mongod, mongo-init (init), pbm-init (init) ----------------------------------------------------------------------------------- pbm-agent logs from some-name-rs0-1 ----------------------------------------------------------------------------------- 2025/11/17 12:34:36 [entrypoint] starting `pbm-agent` 2025-11-17T12:34:39.000+0000 I % _____ %%% | __ \ ###%%%%%%%%%%%%* | |__) |__ _ __ ___ ___ _ __ __ _ ### ##%% %%%% | ___/ _ \ '__/ __/ _ \| '_ \ / _` | #### ##% %%%% | | | __/ | | (_| (_) | | | | (_| | ### #### %%% |_| \___|_| \___\___/|_| |_|\__,_| ,((### ### %%% _____ _ (((( (### #### %%%% / ____| | | ((( ((# ###### | (___ __ _ _ _ __ _ __| | (((( (((# #### \___ \ / _` | | | |/ _` |/ _` | /(( ,((( *### ____) | (_| | |_| | (_| | (_| | //// ((( #### |_____/ \__, |\__,_|\__,_|\__,_| /// (((( #### | | /////////////(((((((((((((((((######## |_| Join @ squad.percona.com/mongodb ** Join Percona Squad! ** Participate in monthly SWAG raffles, get early access to new product features, invite-only ”ask me anything” sessions with database performance experts. Interested? Fill in the form at squad.percona.com/mongodb 2025-11-17T12:34:39.000+0000 I log options: log-path=/dev/stderr, log-level:D, log-json:false 2025-11-17T12:34:39.000+0000 I pbm-agent: Version: 2.12.0 Platform: linux/amd64 GitCommit: 2f7100a2a1da2bb7074e15d757693999cac7574e GitBranch: release-2.12.0 BuildTime: 2025-10-31_09:21_UTC GoVersion: go1.25.3 2025-11-17T12:34:39.000+0000 I starting PITR routine 2025-11-17T12:34:39.000+0000 I node: rs0/some-name-rs0-1.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017 2025-11-17T12:34:39.000+0000 I conn level ReadConcern: majority; WriteConcern: majority 2025-11-17T12:34:39.000+0000 I listening for the commands Defaulted container "mongod" out of: mongod, mongo-init (init), pbm-init (init) ----------------------------------------------------------------------------------- pbm-agent logs from some-name-rs0-2 ----------------------------------------------------------------------------------- 2025/11/17 12:13:34 [entrypoint] starting `pbm-agent` 2025-11-17T12:13:38.000+0000 I % _____ %%% | __ \ ###%%%%%%%%%%%%* | |__) |__ _ __ ___ ___ _ __ __ _ ### ##%% %%%% | ___/ _ \ '__/ __/ _ \| '_ \ / _` | #### ##% %%%% | | | __/ | | (_| (_) | | | | (_| | ### #### %%% |_| \___|_| \___\___/|_| |_|\__,_| ,((### ### %%% _____ _ (((( (### #### %%%% / ____| | | ((( ((# ###### | (___ __ _ _ _ __ _ __| | (((( (((# #### \___ \ / _` | | | |/ _` |/ _` | /(( ,((( *### ____) | (_| | |_| | (_| | (_| | //// ((( #### |_____/ \__, |\__,_|\__,_|\__,_| /// (((( #### | | /////////////(((((((((((((((((######## |_| Join @ squad.percona.com/mongodb ** Join Percona Squad! ** Participate in monthly SWAG raffles, get early access to new product features, invite-only ”ask me anything” sessions with database performance experts. Interested? Fill in the form at squad.percona.com/mongodb 2025-11-17T12:13:38.000+0000 I log options: log-path=/dev/stderr, log-level:D, log-json:false 2025-11-17T12:13:38.000+0000 I pbm-agent: Version: 2.12.0 Platform: linux/amd64 GitCommit: 2f7100a2a1da2bb7074e15d757693999cac7574e GitBranch: release-2.12.0 BuildTime: 2025-10-31_09:21_UTC GoVersion: go1.25.3 2025-11-17T12:13:38.000+0000 I starting PITR routine 2025-11-17T12:13:38.000+0000 I node: rs0/some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017 2025-11-17T12:13:38.000+0000 I conn level ReadConcern: majority; WriteConcern: majority 2025-11-17T12:13:38.000+0000 I listening for the commands 2025-11-17T12:15:00.000+0000 I got command restore [name: 2025-11-17T12:15:00.00103081Z, snapshot: 2025-11-17T12:10:15Z] , opid: 691b11c49a5d489af543d987 2025-11-17T12:15:00.000+0000 I got epoch {1763381472 1} 2025-11-17T12:15:00.000+0000 I [restore/2025-11-17T12:15:00.00103081Z] oplog slicer disabled 2025-11-17T12:15:00.000+0000 I [restore/2025-11-17T12:15:00.00103081Z] backup: 2025-11-17T12:10:15Z 2025-11-17T12:15:00.000+0000 I [restore/2025-11-17T12:15:00.00103081Z] recovery started 2025-11-17T12:15:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] port: 27776 2025-11-17T12:15:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:15:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:15:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:15:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] wait for cluster status 2025-11-17T12:17:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:17:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:17:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:19:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:19:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:19:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:21:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:21:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:21:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:23:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:23:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:23:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:25:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:25:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:25:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:27:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:27:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:27:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:29:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:29:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:29:31.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:31:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:31:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:31:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:33:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025/11/17 12:33:30 [ERROR] writing log: db: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: some-name-cfg-0.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: Unknown, Last error: dial tcp: lookup some-name-cfg-0.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local on 34.118.224.10:53: no such host }, { Addr: some-name-cfg-1.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: RSSecondary, Tag sets: nodeName=gke-jen-psmdb-2044-6293f-default-pool-297d5f86-n3rt,zone=us-central1-a,podName=some-name-cfg-1,serviceName=some-name,region=us-central1, Average RTT: 1069622 }, { Addr: some-name-cfg-2.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: RSSecondary, Tag sets: nodeName=gke-jen-psmdb-2044-6293f-default-pool-297d5f86-csd6,serviceName=some-name,zone=us-central1-a,podName=some-name-cfg-2,region=us-central1, Average RTT: 1164620 }, ] }, , entry: 2025-11-17T12:33:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:33:30.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:33:30.000+0000 E [pitr] init: get conf: get: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: some-name-cfg-0.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: Unknown, Last error: dial tcp: lookup some-name-cfg-0.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local: i/o timeout }, { Addr: some-name-cfg-1.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: RSSecondary, Tag sets: serviceName=some-name,nodeName=gke-jen-psmdb-2044-6293f-default-pool-297d5f86-n3rt,region=us-central1,zone=us-central1-a,podName=some-name-cfg-1, Average RTT: 610925 }, { Addr: some-name-cfg-2.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017, Type: Unknown, Last error: dial tcp: lookup some-name-cfg-2.some-name-cfg.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local: i/o timeout }, ] } 2025-11-17T12:33:31.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:35:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:35:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:35:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:37:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:37:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:37:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:39:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:39:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:39:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:41:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:41:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:41:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:43:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:43:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:43:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:45:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:45:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:45:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:47:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:47:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:47:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:49:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:49:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:49:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:51:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:51:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:51:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:53:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:53:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:53:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:55:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:55:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:55:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:57:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:57:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:57:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:59:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:59:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T12:59:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:01:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:01:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:01:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:03:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:03:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:03:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:05:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/node.some-name-rs0-2.some-name-rs0.demand-backup-physical-sharded-gcp-native-5662.svc.cluster.local:27017.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:05:00.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/rs.rs0/rs.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] 2025-11-17T13:05:01.000+0000 D [restore/2025-11-17T12:15:00.00103081Z] uploading ".pbm.restore/2025-11-17T12:15:00.00103081Z/cluster.hb" [size hint: 10 (10.00B); part size: 10485760 (10.00MB)] Defaulted container "mongod" out of: mongod, mongo-init (init), pbm-init (init) 2025-11-17T13:04:40.117Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "8d22bbeb-8273-4db1-9d47-df9c601ad4c6", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:04:40.117Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "8d22bbeb-8273-4db1-9d47-df9c601ad4c6", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:04:41.397Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "8d22bbeb-8273-4db1-9d47-df9c601ad4c6", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:04:46.398Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:04:46.400Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "pod": "some-name-rs0-0"} 2025-11-17T13:04:46.400Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "pod": "some-name-rs0-1"} 2025-11-17T13:04:46.400Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "pod": "some-name-rs0-2"} 2025-11-17T13:04:46.400Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:04:46.400Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:04:46.400Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:04:48.051Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "721e9480-ddf0-467a-9092-02dcb42dc907", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:04:53.051Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:04:53.052Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "pod": "some-name-rs0-0"} 2025-11-17T13:04:53.052Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "pod": "some-name-rs0-1"} 2025-11-17T13:04:53.052Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "pod": "some-name-rs0-2"} 2025-11-17T13:04:53.052Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:04:53.052Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:04:53.053Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:04:54.325Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "2b8537d0-f001-4840-a09b-4a4fc0a3464c", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:04:59.326Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:04:59.328Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "pod": "some-name-rs0-0"} 2025-11-17T13:04:59.328Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "pod": "some-name-rs0-1"} 2025-11-17T13:04:59.328Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "pod": "some-name-rs0-2"} 2025-11-17T13:04:59.328Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:04:59.328Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:04:59.328Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:00.811Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ccbc0751-d399-48c0-b9f7-c28ceeaaa958", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:05.812Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:05.813Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "pod": "some-name-rs0-2"} 2025-11-17T13:05:05.813Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "pod": "some-name-rs0-0"} 2025-11-17T13:05:05.813Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "pod": "some-name-rs0-1"} 2025-11-17T13:05:05.813Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:05.813Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:05.813Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:06.958Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "75db56f6-7f82-4715-ad8f-b2e00cbc9283", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:11.958Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:11.960Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "pod": "some-name-rs0-1"} 2025-11-17T13:05:11.960Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "pod": "some-name-rs0-2"} 2025-11-17T13:05:11.960Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "pod": "some-name-rs0-0"} 2025-11-17T13:05:11.960Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:11.960Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:11.960Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:13.219Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "615f2ac6-e9f4-4ed3-ae37-8363c6ef9a78", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:18.219Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:18.221Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "pod": "some-name-rs0-0"} 2025-11-17T13:05:18.221Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "pod": "some-name-rs0-1"} 2025-11-17T13:05:18.221Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "pod": "some-name-rs0-2"} 2025-11-17T13:05:18.221Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:18.221Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:18.221Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:19.577Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "730e728f-f439-45c7-a3e6-a2833470123b", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:24.578Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:24.580Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "pod": "some-name-rs0-0"} 2025-11-17T13:05:24.580Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "pod": "some-name-rs0-1"} 2025-11-17T13:05:24.580Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "pod": "some-name-rs0-2"} 2025-11-17T13:05:24.580Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:24.580Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:24.580Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:25.694Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "1c2d7ddb-8281-45fe-aa10-c26401120b81", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:30.696Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:30.697Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "pod": "some-name-rs0-0"} 2025-11-17T13:05:30.697Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "pod": "some-name-rs0-1"} 2025-11-17T13:05:30.697Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "pod": "some-name-rs0-2"} 2025-11-17T13:05:30.697Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:30.697Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:30.697Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:32.022Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "7ab80dc7-25ca-4996-a3e4-6542cced3822", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:37.023Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:37.025Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "pod": "some-name-rs0-1"} 2025-11-17T13:05:37.025Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "pod": "some-name-rs0-2"} 2025-11-17T13:05:37.025Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "pod": "some-name-rs0-0"} 2025-11-17T13:05:37.025Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:37.025Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:37.025Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:38.219Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "f0882ee6-ee0b-4b36-9a84-1263b1f737f3", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:43.221Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:43.223Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "pod": "some-name-rs0-0"} 2025-11-17T13:05:43.223Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "pod": "some-name-rs0-1"} 2025-11-17T13:05:43.223Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "pod": "some-name-rs0-2"} 2025-11-17T13:05:43.223Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:43.223Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:43.224Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:44.257Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "5402c4e6-2460-4160-9ae6-75f8aa4eeec7", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:49.258Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:49.259Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "pod": "some-name-rs0-0"} 2025-11-17T13:05:49.259Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "pod": "some-name-rs0-1"} 2025-11-17T13:05:49.259Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "pod": "some-name-rs0-2"} 2025-11-17T13:05:49.259Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:49.259Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:49.259Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:50.644Z DEBUG PBM restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "ce7336ef-24f2-4b9c-9f92-d16a3930316f", "status": {"type":"physical","opid":"","name":"2025-11-17T12:15:00.00103081Z","replsets":[{"name":"cfg","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null},{"name":"rs0","start_ts":0,"status":"","size":0,"size_uncompressed":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"node":"","conditions":null}],"compression":"","store":{"type":""},"size":0,"size_uncompressed":0,"mongodb_version":"","fcv":"","start_ts":0,"last_transition_ts":0,"first_write_ts":{"T":0,"I":0},"last_write_ts":{"T":0,"I":0},"hb":{"T":0,"I":0},"status":"","conditions":null,"n":null,"pbm_version":"","balancer":""}} 2025-11-17T13:05:55.645Z DEBUG checking if restore is allowed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "cluster": "some-name", "namespace": "demand-backup-physical-sharded-gcp-native-5662"} 2025-11-17T13:05:55.647Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "pod": "some-name-rs0-2"} 2025-11-17T13:05:55.647Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "pod": "some-name-rs0-0"} 2025-11-17T13:05:55.647Z DEBUG Pod is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "pod": "some-name-rs0-1"} 2025-11-17T13:05:55.647Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "sts": "some-name-rs0", "replset": "rs0"} 2025-11-17T13:05:55.647Z DEBUG Statefulset is ready for physical restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "sts": "some-name-cfg", "replset": "cfg"} 2025-11-17T13:05:55.647Z DEBUG Check restore status {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "command": ["/opt/percona/pbm", "describe-restore", "2025-11-17T12:15:00.00103081Z", "--config", "/etc/pbm/pbm_config.yaml", "--out", "json"], "pod": "some-name-rs0-0"} 2025-11-17T13:05:56.673Z ERROR failed to make restore {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "restore": "restore-backup-gcp-cs-sa-sharded", "backup": "backup-gcp-cs-sa-sharded", "error": "unmarshal PBM describe-restore output: unexpected end of JSON input", "errorVerbose": "unexpected end of JSON input\nunmarshal PBM describe-restore output\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodbrestore.(*ReconcilePerconaServerMongoDBRestore).reconcilePhysicalRestore\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodbrestore/physical.go:225\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodbrestore.(*ReconcilePerconaServerMongoDBRestore).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodbrestore/psmdb_restore_controller.go:260\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.3/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.3/pkg/internal/controller/controller.go:461\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.3/pkg/internal/controller/controller.go:421\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.3/pkg/internal/controller/controller.go:296\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1693"} 2025-11-17T13:05:56.673Z INFO Restore state changed {"controller": "psmdbrestore-controller", "controllerGroup": "psmdb.percona.com", "controllerKind": "PerconaServerMongoDBRestore", "PerconaServerMongoDBRestore": {"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"}, "namespace": "demand-backup-physical-sharded-gcp-native-5662", "name": "restore-backup-gcp-cs-sa-sharded", "reconcileID": "76e92b47-7faf-4802-a4a4-cfca3aa7a801", "previous": "requested", "current": "error"} apiVersion: psmdb.percona.com/v1 kind: PerconaServerMongoDBRestore metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"psmdb.percona.com/v1","kind":"PerconaServerMongoDBRestore","metadata":{"annotations":{},"name":"restore-backup-gcp-cs-sa-sharded","namespace":"demand-backup-physical-sharded-gcp-native-5662"},"spec":{"backupName":"backup-gcp-cs-sa-sharded","clusterName":"some-name"}} creationTimestamp: "2025-11-17T12:10:48Z" generation: 1 name: restore-backup-gcp-cs-sa-sharded namespace: demand-backup-physical-sharded-gcp-native-5662 resourceVersion: "1763381700033455014" uid: 3cba9682-6394-4514-8d1d-95e6c219c850 spec: backupName: backup-gcp-cs-sa-sharded clusterName: some-name status: pbmName: "2025-11-17T12:15:00.00103081Z" state: requested Name: restore-backup-gcp-cs-sa-sharded Namespace: demand-backup-physical-sharded-gcp-native-5662 Labels: Annotations: API Version: psmdb.percona.com/v1 Kind: PerconaServerMongoDBRestore Metadata: Creation Timestamp: 2025-11-17T12:10:48Z Generation: 1 Resource Version: 1763381700033455014 UID: 3cba9682-6394-4514-8d1d-95e6c219c850 Spec: Backup Name: backup-gcp-cs-sa-sharded Cluster Name: some-name Status: Pbm Name: 2025-11-17T12:15:00.00103081Z State: requested Events: Restore object restore-backup-gcp-cs-sa-sharded is in requested state after 50 minutes. something went wrong with operator or kubernetes cluster