Log: /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/logs/scheduled-backup.log


-----------------------------------------------------------------------------------
get and delete old CRDs and RBAC
-----------------------------------------------------------------------------------

error: the server doesn't have a resource type "perconaservermongodbbackups"
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbbackups"
error: the server doesn't have a resource type "perconaservermongodbrestores"
+ kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbrestores"
error: the server doesn't have a resource type "perconaservermongodbs"
+ kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbs"
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found


-----------------------------------------------------------------------------------
destroy chaos-mesh
-----------------------------------------------------------------------------------

error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified


-----------------------------------------------------------------------------------
cleaned up all old namespaces
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
cleaned up old namespaces scheduled-backup-4631
-----------------------------------------------------------------------------------

error: resource(s) were provided, but no name was specified


-----------------------------------------------------------------------------------
create namespace scheduled-backup-4631
-----------------------------------------------------------------------------------

namespace/scheduled-backup-4631 created
Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-1352-9793733e-5-cluster1" modified.


-----------------------------------------------------------------------------------
start PSMDB operator
-----------------------------------------------------------------------------------

customresourcedefinition.apiextensions.k8s.io/perconaservermongodbbackups.psmdb.percona.com serverside-applied
customresourcedefinition.apiextensions.k8s.io/perconaservermongodbrestores.psmdb.percona.com serverside-applied
customresourcedefinition.apiextensions.k8s.io/perconaservermongodbs.psmdb.percona.com serverside-applied
role.rbac.authorization.k8s.io/percona-server-mongodb-operator created
serviceaccount/percona-server-mongodb-operator created
rolebinding.rbac.authorization.k8s.io/service-account-percona-server-mongodb-operator created
deployment.apps/percona-server-mongodb-operator created
waiting for pod/percona-server-mongodb-operator-6b6bcb669-z7xfq to be ready.OK


-----------------------------------------------------------------------------------
install Minio
-----------------------------------------------------------------------------------

Error: uninstall: Release not loaded: minio-service: release: not found
"minio" has been removed from your repositories
"minio" has been added to your repositories
NAME: minio-service
LAST DEPLOYED: Thu Oct  5 15:14:03 2023
NAMESPACE: scheduled-backup-4631
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Minio can be accessed via port 9000 on the following DNS name from within your cluster:
minio-service.scheduled-backup-4631.svc.cluster.local

To access Minio from localhost, run the below commands:

  1. export POD_NAME=$(kubectl get pods --namespace scheduled-backup-4631 -l "release=minio-service" -o jsonpath="{.items[0].metadata.name}")

  2. kubectl port-forward $POD_NAME 9000 --namespace scheduled-backup-4631

Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/

You can now access Minio server on http://localhost:9000. Follow the below steps to connect to Minio server with mc client:

  1. Download the Minio mc client - https://docs.minio.io/docs/minio-client-quickstart-guide

  2. Get the ACCESS_KEY=$(kubectl get secret minio-service -o jsonpath="{.data.accesskey}" | base64 --decode) and the SECRET_KEY=$(kubectl get secret minio-service -o jsonpath="{.data.secretkey}" | base64 --decode)

  3. mc alias set minio-service-local http://localhost:9000 "$ACCESS_KEY" "$SECRET_KEY" --api s3v4

  4. mc ls minio-service-local

Alternately, you can use your browser or the Minio SDK to access the server - https://docs.minio.io/categories/17
waiting for pod/minio-service-5fc4dc898-fc8q9 to be ready.....OK
make_bucket: operator-testing
pod "aws-cli" deleted
If you don't see a command prompt, try pressing enter.
Error attaching, falling back to logs: Internal error occurred: error attaching to container: container is in CONTAINER_EXITED state


-----------------------------------------------------------------------------------
add labels
-----------------------------------------------------------------------------------

node/gke-jen-psmdb-1352-97937-default-pool-9e7f5a5f-gx2x labeled


-----------------------------------------------------------------------------------
create PriorityClass
-----------------------------------------------------------------------------------

priorityclass.scheduling.k8s.io/high-priority created


-----------------------------------------------------------------------------------
create secrets and start client
-----------------------------------------------------------------------------------

secret/some-users created
deployment.apps/psmdb-client created


-----------------------------------------------------------------------------------
create secrets for cloud storages
-----------------------------------------------------------------------------------

secret/minio-secret created
secret/aws-s3-secret created
secret/gcp-cs-secret created
secret/azure-secret created


-----------------------------------------------------------------------------------
create first PSMDB cluster some-name-rs0
-----------------------------------------------------------------------------------

perconaservermongodb.psmdb.percona.com/some-name created


-----------------------------------------------------------------------------------
check if all 3 Pods started
-----------------------------------------------------------------------------------

waiting for pod/some-name-rs0-0 to be ready......OK
waiting for pod/some-name-rs0-1 to be ready......OK
waiting for pod/some-name-rs0-2 to be ready......OK
Waiting for cluster readyness.


-----------------------------------------------------------------------------------
check if service and statefulset created with expected config
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
create user myApp
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("2ee1780a-4e4f-4285-a452-1c69688ead2b") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
Successfully added user: {
	"user" : "myApp",
	"roles" : [
		{
			"db" : "myApp",
			"role" : "readWrite"
		}
	]
}
bye


-----------------------------------------------------------------------------------
write data, read from all
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("1cb6e46d-2b71-456d-853d-f8f9f6283546") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
some-name-rs0-0
some-name-rs0-1
some-name-rs0-2


-----------------------------------------------------------------------------------
add backups schedule, wait for the first backup
-----------------------------------------------------------------------------------

perconaservermongodb.psmdb.percona.com/some-name configured
perconaservermongodb.psmdb.percona.com/some-name configured
cron-some-name-20231005151800-74l28..........................
cron-some-name-20231005151800-z5l2b.
cron-some-name-20231005151800-mzq6c.
cron-some-name-20231005151800-qj98p.


-----------------------------------------------------------------------------------
check backup and restore -- minio
-----------------------------------------------------------------------------------

If you don't see a command prompt, try pressing enter.
Error attaching, falling back to logs: unable to upgrade connection: container aws-cli not found in pod aws-cli_scheduled-backup-4631
2023-10-05 15:18:26         56 myApp.test.gz
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("7f0191a3-87c9-454f-8b9d-741cece672fc") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
perconaservermongodbrestore.psmdb.percona.com/restore-cron-some-name-20231005151800-qj98p created
waiting psmdb-restore/cron-some-name-20231005151800-qj98p to reach ready state......
+ '[' 1 -eq 1 ']'
+ wait_cluster_consistency some-name
+ local cluster_name=some-name
+ local wait_time=32
+ retry=0
+ sleep 7
+ echo -n 'waiting for cluster readyness'
waiting for cluster readyness++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.CiGBfj8rIS
+++ mktemp
++ local LAST_ERR=/tmp/tmp.5yeOc2gYeP
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.CiGBfj8rIS
++ cat /tmp/tmp.5yeOc2gYeP
++ rm /tmp/tmp.CiGBfj8rIS /tmp/tmp.5yeOc2gYeP
++ return 0
+ [[ ready == \r\e\a\d\y ]]
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.crzgvqqe3P
+++ mktemp
++ local LAST_ERR=/tmp/tmp.IgUIfXc8mR
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.crzgvqqe3P
++ cat /tmp/tmp.IgUIfXc8mR
++ rm /tmp/tmp.crzgvqqe3P /tmp/tmp.IgUIfXc8mR
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.0wXNpLjAD9
++ mktemp
+ local LAST_ERR=/tmp/tmp.8uEIYPre4u
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.0wXNpLjAD9
+ cat /tmp/tmp.8uEIYPre4u
+ rm /tmp/tmp.0wXNpLjAD9 /tmp/tmp.8uEIYPre4u
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.JrTri1jcJZ
+++ mktemp
++ local LAST_ERR=/tmp/tmp.m4Qxr9YSMF
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.JrTri1jcJZ
++ cat /tmp/tmp.m4Qxr9YSMF
++ rm /tmp/tmp.JrTri1jcJZ /tmp/tmp.m4Qxr9YSMF
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.Cvwul2uTNP
++ mktemp
+ local LAST_ERR=/tmp/tmp.i3IWMwzVVS
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.Cvwul2uTNP
+ cat /tmp/tmp.i3IWMwzVVS
+ rm /tmp/tmp.Cvwul2uTNP /tmp/tmp.i3IWMwzVVS
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.GdL113rhPS
+++ mktemp
++ local LAST_ERR=/tmp/tmp.JjthxX4RDm
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.GdL113rhPS
++ cat /tmp/tmp.JjthxX4RDm
++ rm /tmp/tmp.GdL113rhPS /tmp/tmp.JjthxX4RDm
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.aFBERAAWkz
++ mktemp
+ local LAST_ERR=/tmp/tmp.IfusllWPzw
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.aFBERAAWkz
+ cat /tmp/tmp.IfusllWPzw
+ rm /tmp/tmp.aFBERAAWkz /tmp/tmp.IfusllWPzw
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ '[' -z '' ']'
+ desc 'check backup and restore -- aws-s3'
+ set +o xtrace


-----------------------------------------------------------------------------------
check backup and restore -- aws-s3
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("e51847e3-7979-4c76-983c-ef1d41fa41bd") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
perconaservermongodbrestore.psmdb.percona.com/restore-cron-some-name-20231005151800-74l28 created
waiting psmdb-restore/cron-some-name-20231005151800-74l28 to reach ready state.............
+ '[' 1 -eq 1 ']'
+ wait_cluster_consistency some-name
+ local cluster_name=some-name
+ local wait_time=32
+ retry=0
+ sleep 7
+ echo -n 'waiting for cluster readyness'
waiting for cluster readyness++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.5YDiaU6x8h
+++ mktemp
++ local LAST_ERR=/tmp/tmp.3jV0Gq6Q5I
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.5YDiaU6x8h
++ cat /tmp/tmp.3jV0Gq6Q5I
++ rm /tmp/tmp.5YDiaU6x8h /tmp/tmp.3jV0Gq6Q5I
++ return 0
+ [[ ready == \r\e\a\d\y ]]
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.19V3SNrzAJ
+++ mktemp
++ local LAST_ERR=/tmp/tmp.bqusC4MSfX
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.19V3SNrzAJ
++ cat /tmp/tmp.bqusC4MSfX
++ rm /tmp/tmp.19V3SNrzAJ /tmp/tmp.bqusC4MSfX
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.xuPjq8ZTMZ
++ mktemp
+ local LAST_ERR=/tmp/tmp.nItYzIPkTP
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.xuPjq8ZTMZ
+ cat /tmp/tmp.nItYzIPkTP
+ rm /tmp/tmp.xuPjq8ZTMZ /tmp/tmp.nItYzIPkTP
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.2rNMD8dzJa
+++ mktemp
++ local LAST_ERR=/tmp/tmp.rUmpHkkQQi
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.2rNMD8dzJa
++ cat /tmp/tmp.rUmpHkkQQi
++ rm /tmp/tmp.2rNMD8dzJa /tmp/tmp.rUmpHkkQQi
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.flLKcOYRZh
++ mktemp
+ local LAST_ERR=/tmp/tmp.J0FIlvFp6h
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.flLKcOYRZh
+ cat /tmp/tmp.J0FIlvFp6h
+ rm /tmp/tmp.flLKcOYRZh /tmp/tmp.J0FIlvFp6h
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 mongodb ''
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.jdcmCmy6xw
+++ mktemp
++ local LAST_ERR=/tmp/tmp.FrwyEutbIu
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.jdcmCmy6xw
++ cat /tmp/tmp.FrwyEutbIu
++ rm /tmp/tmp.jdcmCmy6xw /tmp/tmp.FrwyEutbIu
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.qP8f2d4Kpl
++ mktemp
+ local LAST_ERR=/tmp/tmp.uoJcAMOLZs
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.qP8f2d4Kpl
+ cat /tmp/tmp.uoJcAMOLZs
+ rm /tmp/tmp.qP8f2d4Kpl /tmp/tmp.uoJcAMOLZs
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ desc 'check backup and restore -- gcp-cs'
+ set +o xtrace


-----------------------------------------------------------------------------------
check backup and restore -- gcp-cs
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("dc0ff413-d0d1-4eba-b098-aac8986a25bd") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
perconaservermongodbrestore.psmdb.percona.com/restore-cron-some-name-20231005151800-z5l2b created
waiting psmdb-restore/cron-some-name-20231005151800-z5l2b to reach ready state........
+ '[' 1 -eq 1 ']'
+ wait_cluster_consistency some-name
+ local cluster_name=some-name
+ local wait_time=32
+ retry=0
+ sleep 7
+ echo -n 'waiting for cluster readyness'
waiting for cluster readyness++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.ALyaLoZLiq
+++ mktemp
++ local LAST_ERR=/tmp/tmp.RA8cl6P5XI
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.ALyaLoZLiq
++ cat /tmp/tmp.RA8cl6P5XI
++ rm /tmp/tmp.ALyaLoZLiq /tmp/tmp.RA8cl6P5XI
++ return 0
+ [[ ready == \r\e\a\d\y ]]
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.zDJjbyCj1w
+++ mktemp
++ local LAST_ERR=/tmp/tmp.wWVuiFLWvu
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.zDJjbyCj1w
++ cat /tmp/tmp.wWVuiFLWvu
++ rm /tmp/tmp.zDJjbyCj1w /tmp/tmp.wWVuiFLWvu
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.PRSchUgT35
++ mktemp
+ local LAST_ERR=/tmp/tmp.i37vvJwhMi
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.PRSchUgT35
+ cat /tmp/tmp.i37vvJwhMi
+ rm /tmp/tmp.PRSchUgT35 /tmp/tmp.i37vvJwhMi
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.cPdT4IM4gL
+++ mktemp
++ local LAST_ERR=/tmp/tmp.L8S1DUELUZ
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.cPdT4IM4gL
++ cat /tmp/tmp.L8S1DUELUZ
++ rm /tmp/tmp.cPdT4IM4gL /tmp/tmp.L8S1DUELUZ
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.LPRNVc462i
++ mktemp
+ local LAST_ERR=/tmp/tmp.ym5cenEIlE
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.LPRNVc462i
+ cat /tmp/tmp.ym5cenEIlE
+ rm /tmp/tmp.LPRNVc462i /tmp/tmp.ym5cenEIlE
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 mongodb ''
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.YiFJfXLNTK
+++ mktemp
++ local LAST_ERR=/tmp/tmp.AdoIkmsfWl
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.YiFJfXLNTK
++ cat /tmp/tmp.AdoIkmsfWl
++ rm /tmp/tmp.YiFJfXLNTK /tmp/tmp.AdoIkmsfWl
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.fneYNTc6wT
++ mktemp
+ local LAST_ERR=/tmp/tmp.7AnQRw6zgo
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.fneYNTc6wT
+ cat /tmp/tmp.7AnQRw6zgo
+ rm /tmp/tmp.fneYNTc6wT /tmp/tmp.7AnQRw6zgo
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ desc 'check backup and restore -- azure-blob'
+ set +o xtrace


-----------------------------------------------------------------------------------
check backup and restore -- azure-blob
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017,some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("d023e6ac-2a7a-4cea-a25c-9f30229665b5") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
perconaservermongodbrestore.psmdb.percona.com/restore-cron-some-name-20231005151800-mzq6c created
waiting psmdb-restore/cron-some-name-20231005151800-mzq6c to reach ready state..........
+ '[' 1 -eq 1 ']'
+ wait_cluster_consistency some-name
+ local cluster_name=some-name
+ local wait_time=32
+ retry=0
+ sleep 7
+ echo -n 'waiting for cluster readyness'
waiting for cluster readyness++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.BzxOSWVkZJ
+++ mktemp
++ local LAST_ERR=/tmp/tmp.2SbpQMGJbM
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.BzxOSWVkZJ
++ cat /tmp/tmp.2SbpQMGJbM
++ rm /tmp/tmp.BzxOSWVkZJ /tmp/tmp.2SbpQMGJbM
++ return 0
+ [[ ready == \r\e\a\d\y ]]
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.9kGnaezE5M
+++ mktemp
++ local LAST_ERR=/tmp/tmp.oCCb2wOIAq
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.9kGnaezE5M
++ cat /tmp/tmp.oCCb2wOIAq
++ rm /tmp/tmp.9kGnaezE5M /tmp/tmp.oCCb2wOIAq
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.5usjXs9V54
++ mktemp
+ local LAST_ERR=/tmp/tmp.xv43ww9wZx
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.5usjXs9V54
+ cat /tmp/tmp.xv43ww9wZx
+ rm /tmp/tmp.5usjXs9V54 /tmp/tmp.xv43ww9wZx
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.AuuUlpbtIU
+++ mktemp
++ local LAST_ERR=/tmp/tmp.d2TXtWGj3l
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.AuuUlpbtIU
++ cat /tmp/tmp.d2TXtWGj3l
++ rm /tmp/tmp.AuuUlpbtIU /tmp/tmp.d2TXtWGj3l
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.Ntj5n1A8dX
++ mktemp
+ local LAST_ERR=/tmp/tmp.xYXKwAKi6t
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.Ntj5n1A8dX
+ cat /tmp/tmp.xYXKwAKi6t
+ rm /tmp/tmp.Ntj5n1A8dX /tmp/tmp.xYXKwAKi6t
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.hh5BmVSNtS
+++ mktemp
++ local LAST_ERR=/tmp/tmp.RL4Bq3C6eb
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.hh5BmVSNtS
++ cat /tmp/tmp.RL4Bq3C6eb
++ rm /tmp/tmp.hh5BmVSNtS /tmp/tmp.RL4Bq3C6eb
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.XrTsIkICnM
++ mktemp
+ local LAST_ERR=/tmp/tmp.mW9AobJajm
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.XrTsIkICnM
+ cat /tmp/tmp.mW9AobJajm
+ rm /tmp/tmp.XrTsIkICnM /tmp/tmp.mW9AobJajm
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ desc 'add physical backup schedule, wait for the first backup'
+ set +o xtrace


-----------------------------------------------------------------------------------
add physical backup schedule, wait for the first backup
-----------------------------------------------------------------------------------

perconaservermongodb.psmdb.percona.com/some-name configured
perconaservermongodb.psmdb.percona.com/some-name configured
cron-some-name-20231005152800-kkgrp..


-----------------------------------------------------------------------------------
check backup and restore -- minio
-----------------------------------------------------------------------------------

perconaservermongodbrestore.psmdb.percona.com/restore-cron-some-name-20231005152800-kkgrp created
waiting psmdb-restore/cron-some-name-20231005152800-kkgrp to reach ready state...................................................
+ '[' 1 -eq 1 ']'
+ wait_cluster_consistency some-name
+ local cluster_name=some-name
+ local wait_time=32
+ retry=0
+ sleep 7
+ echo -n 'waiting for cluster readyness'
waiting for cluster readyness++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.IMMgis7EqD
+++ mktemp
++ local LAST_ERR=/tmp/tmp.J7nk2kWGJ8
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.IMMgis7EqD
++ cat /tmp/tmp.J7nk2kWGJ8
++ rm /tmp/tmp.IMMgis7EqD /tmp/tmp.J7nk2kWGJ8
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 1 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.UzD1lt5kog
+++ mktemp
++ local LAST_ERR=/tmp/tmp.tAV7f0V8md
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.UzD1lt5kog
++ cat /tmp/tmp.tAV7f0V8md
++ rm /tmp/tmp.UzD1lt5kog /tmp/tmp.tAV7f0V8md
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 2 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.aUe1SEU7ZQ
+++ mktemp
++ local LAST_ERR=/tmp/tmp.mOjPzCLCRh
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.aUe1SEU7ZQ
++ cat /tmp/tmp.mOjPzCLCRh
++ rm /tmp/tmp.aUe1SEU7ZQ /tmp/tmp.mOjPzCLCRh
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 3 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.TBrvAuahRu
+++ mktemp
++ local LAST_ERR=/tmp/tmp.FfVW4Vkzef
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.TBrvAuahRu
++ cat /tmp/tmp.FfVW4Vkzef
++ rm /tmp/tmp.TBrvAuahRu /tmp/tmp.FfVW4Vkzef
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 4 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.6xtFvUazW4
+++ mktemp
++ local LAST_ERR=/tmp/tmp.QgHz0dnH8S
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.6xtFvUazW4
++ cat /tmp/tmp.QgHz0dnH8S
++ rm /tmp/tmp.6xtFvUazW4 /tmp/tmp.QgHz0dnH8S
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 5 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.u1788Thttq
+++ mktemp
++ local LAST_ERR=/tmp/tmp.Opg7vp5VAn
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.u1788Thttq
++ cat /tmp/tmp.Opg7vp5VAn
++ rm /tmp/tmp.u1788Thttq /tmp/tmp.Opg7vp5VAn
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 6 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.QXokXOt9yt
+++ mktemp
++ local LAST_ERR=/tmp/tmp.fTqJ6uuwhG
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.QXokXOt9yt
++ cat /tmp/tmp.fTqJ6uuwhG
++ rm /tmp/tmp.QXokXOt9yt /tmp/tmp.fTqJ6uuwhG
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 7 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.909CgCfcMS
+++ mktemp
++ local LAST_ERR=/tmp/tmp.Yhb2xI7Wtx
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.909CgCfcMS
++ cat /tmp/tmp.Yhb2xI7Wtx
++ rm /tmp/tmp.909CgCfcMS /tmp/tmp.Yhb2xI7Wtx
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 8 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.U1Aq9ygeaq
+++ mktemp
++ local LAST_ERR=/tmp/tmp.jyva6Tzafv
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.U1Aq9ygeaq
++ cat /tmp/tmp.jyva6Tzafv
++ rm /tmp/tmp.U1Aq9ygeaq /tmp/tmp.jyva6Tzafv
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 9 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.bFXriTInbg
+++ mktemp
++ local LAST_ERR=/tmp/tmp.ADFGfdQt9N
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.bFXriTInbg
++ cat /tmp/tmp.ADFGfdQt9N
++ rm /tmp/tmp.bFXriTInbg /tmp/tmp.ADFGfdQt9N
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 10 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.h7vIMy6mdY
+++ mktemp
++ local LAST_ERR=/tmp/tmp.xehp5voPPX
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.h7vIMy6mdY
++ cat /tmp/tmp.xehp5voPPX
++ rm /tmp/tmp.h7vIMy6mdY /tmp/tmp.xehp5voPPX
++ return 0
+ [[ initializing == \r\e\a\d\y ]]
+ let retry+=1
+ '[' 11 -ge 32 ']'
+ echo -n .
.+ sleep 10
++ kubectl_bin get psmdb some-name -o 'jsonpath={.status.state}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.XB9ONuQn8g
+++ mktemp
++ local LAST_ERR=/tmp/tmp.v35nKqjCIr
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get psmdb some-name -o 'jsonpath={.status.state}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.XB9ONuQn8g
++ cat /tmp/tmp.v35nKqjCIr
++ rm /tmp/tmp.XB9ONuQn8g /tmp/tmp.v35nKqjCIr
++ return 0
+ [[ ready == \r\e\a\d\y ]]
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.B4rOXFGOZB
+++ mktemp
++ local LAST_ERR=/tmp/tmp.75wUtYyFN2
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.B4rOXFGOZB
++ cat /tmp/tmp.75wUtYyFN2
++ rm /tmp/tmp.B4rOXFGOZB /tmp/tmp.75wUtYyFN2
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.mxmv2HvqA3
++ mktemp
+ local LAST_ERR=/tmp/tmp.DFgAsxulpq
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-0.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.mxmv2HvqA3
+ cat /tmp/tmp.DFgAsxulpq
+ rm /tmp/tmp.mxmv2HvqA3 /tmp/tmp.DFgAsxulpq
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 mongodb ''
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
++ local LAST_OUT=/tmp/tmp.zSTH0aZ6i8
+++ mktemp
++ local LAST_ERR=/tmp/tmp.tIU2tbcKfR
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.zSTH0aZ6i8
++ cat /tmp/tmp.tIU2tbcKfR
++ rm /tmp/tmp.zSTH0aZ6i8 /tmp/tmp.tIU2tbcKfR
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.sGy1L6PQbe
++ mktemp
+ local LAST_ERR=/tmp/tmp.HV0Mc3aVRy
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-1.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.sGy1L6PQbe
+ cat /tmp/tmp.HV0Mc3aVRy
+ rm /tmp/tmp.sGy1L6PQbe /tmp/tmp.HV0Mc3aVRy
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ compare_mongo_cmd find myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local command=find
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local postfix=
+ local suffix=
+ local database=myApp
+ local collection=test
+ run_mongo 'use myApp\n db.test.find()' myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 mongodb ''
+ egrep -v 'I NETWORK|W NETWORK|F NETWORK|Error saving history file|Percona Server for MongoDB|connecting to:|Unable to reach primary for set|Implicit session:|versions do not match'
+ local 'command=use myApp\n db.test.find()'
+ local uri=myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631
+ local driver=mongodb
+ local suffix=.svc.cluster.local
++ kubectl_bin get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
+++ mktemp
+ /usr/bin/sed -re 's/ObjectId\("[0-9a-f]+"\)//; s/-[0-9]+.svc/-xxx.svc/'
++ local LAST_OUT=/tmp/tmp.CdDJ8Qr2wQ
+++ mktemp
++ local LAST_ERR=/tmp/tmp.o9k6K3TMc3
++ local exit_status=0
++ local timeout=4
+++ seq 0 2
++ for i in '$(seq 0 2)'
++ set +e
++ kubectl get pods --selector=name=psmdb-client -o 'jsonpath={.items[].metadata.name}'
++ exit_status=0
++ set -e
++ '[' 0 '!=' 0 -a -n 0 ']'
++ break
++ cat /tmp/tmp.CdDJ8Qr2wQ
++ cat /tmp/tmp.o9k6K3TMc3
++ rm /tmp/tmp.CdDJ8Qr2wQ /tmp/tmp.o9k6K3TMc3
++ return 0
+ local client_container=psmdb-client-9b7b5fdcc-6gv6s
+ local mongo_flag=
+ [[ myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631 == *cfg* ]]
+ replica_set=rs0
+ kubectl_bin exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
++ mktemp
+ local LAST_OUT=/tmp/tmp.81anwfT4Lu
++ mktemp
+ local LAST_ERR=/tmp/tmp.ZZ4QJ42olW
+ local exit_status=0
+ local timeout=4
++ seq 0 2
+ for i in '$(seq 0 2)'
+ set +e
+ kubectl exec psmdb-client-9b7b5fdcc-6gv6s -- bash -c 'printf '\''use myApp\n db.test.find()\n'\'' | mongo mongodb://myApp:myPass@some-name-rs0-2.some-name-rs0.scheduled-backup-4631.svc.cluster.local/admin?ssl=false\&replicaSet=rs0 '
+ exit_status=0
+ set -e
+ '[' 0 '!=' 0 -a -n 0 ']'
+ break
+ cat /tmp/tmp.81anwfT4Lu
+ cat /tmp/tmp.ZZ4QJ42olW
+ rm /tmp/tmp.81anwfT4Lu /tmp/tmp.ZZ4QJ42olW
+ return 0
+ diff /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/scheduled-backup/compare/find.json /tmp/tmp.SpUSatvxsN/find
+ sleep 60
+ unlabel_node
+ desc 'remove labels'
+ set +o xtrace


-----------------------------------------------------------------------------------
remove labels
-----------------------------------------------------------------------------------

node/gke-jen-psmdb-1352-97937-default-pool-9e7f5a5f-gx2x labeled


-----------------------------------------------------------------------------------
destroy cluster/operator and all other resources
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
get and delete old CRDs and RBAC
-----------------------------------------------------------------------------------

customresourcedefinition.apiextensions.k8s.io "perconaservermongodbbackups.psmdb.percona.com" deleted
customresourcedefinition.apiextensions.k8s.io "perconaservermongodbrestores.psmdb.percona.com" deleted
customresourcedefinition.apiextensions.k8s.io "perconaservermongodbs.psmdb.percona.com" deleted
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n scheduled-backup-4631 cron-some-name-20231005151800-qj98p --type=merge -p '{"metadata":{"finalizers":[]}}'
perconaservermongodbbackup.psmdb.percona.com/cron-some-name-20231005151800-qj98p patched
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n scheduled-backup-4631 cron-some-name-20231005151800-z5l2b --type=merge -p '{"metadata":{"finalizers":[]}}'
perconaservermongodbbackup.psmdb.percona.com/cron-some-name-20231005151800-z5l2b patched
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n scheduled-backup-4631 cron-some-name-20231005152800-kkgrp --type=merge -p '{"metadata":{"finalizers":[]}}'
perconaservermongodbbackup.psmdb.percona.com/cron-some-name-20231005152800-kkgrp patched
customresourcedefinition.apiextensions.k8s.io/perconaservermongodbbackups.psmdb.percona.com condition met
error: the server doesn't have a resource type "perconaservermongodbrestores"
+ kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbrestores"
error: the server doesn't have a resource type "perconaservermongodbs"
+ kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbs"
role.rbac.authorization.k8s.io "percona-server-mongodb-operator" deleted
serviceaccount "percona-server-mongodb-operator" deleted
rolebinding.rbac.authorization.k8s.io "service-account-percona-server-mongodb-operator" deleted


-----------------------------------------------------------------------------------
test passed
-----------------------------------------------------------------------------------