Log: /mnt/jenkins/workspace/cloud-psmdb-operator_PR-1352/e2e-tests/logs/monitoring-2-0.log


-----------------------------------------------------------------------------------
deploy cert manager
-----------------------------------------------------------------------------------

namespace/cert-manager created
namespace/cert-manager labeled
namespace/cert-manager configured
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
configmap/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
role.rbac.authorization.k8s.io/cert-manager:leaderelection created
role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
Warning: resource namespaces/cert-manager is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
pod/cert-manager-cainjector-6fd77d5fdc-hk7rv condition met
pod/cert-manager-df6d8fb9-nzfw9 condition met
pod/cert-manager-webhook-576cc5ddb7-mwlf7 condition met


-----------------------------------------------------------------------------------
get and delete old CRDs and RBAC
-----------------------------------------------------------------------------------

error: the server doesn't have a resource type "perconaservermongodbbackups"
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbbackups"
error: the server doesn't have a resource type "perconaservermongodbrestores"
+ kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbrestores"
error: the server doesn't have a resource type "perconaservermongodbs"
+ kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbs"
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "null" not found


-----------------------------------------------------------------------------------
destroy chaos-mesh
-----------------------------------------------------------------------------------

error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified
error: resource(s) were provided, but no name was specified


-----------------------------------------------------------------------------------
cleaned up all old namespaces
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
cleaned up old namespaces monitoring-2-0-27394
-----------------------------------------------------------------------------------

namespace "cert-manager" deleted


-----------------------------------------------------------------------------------
create namespace monitoring-2-0-27394
-----------------------------------------------------------------------------------

namespace/monitoring-2-0-27394 created
Context "gke_cloud-dev-112233_us-central1-a_jen-psmdb-1352-9793733e-5-cluster1" modified.


-----------------------------------------------------------------------------------
start PSMDB operator
-----------------------------------------------------------------------------------

customresourcedefinition.apiextensions.k8s.io/perconaservermongodbbackups.psmdb.percona.com serverside-applied
customresourcedefinition.apiextensions.k8s.io/perconaservermongodbrestores.psmdb.percona.com serverside-applied
customresourcedefinition.apiextensions.k8s.io/perconaservermongodbs.psmdb.percona.com serverside-applied
role.rbac.authorization.k8s.io/percona-server-mongodb-operator created
serviceaccount/percona-server-mongodb-operator created
rolebinding.rbac.authorization.k8s.io/service-account-percona-server-mongodb-operator created
deployment.apps/percona-server-mongodb-operator created
waiting for pod/percona-server-mongodb-operator-6b6bcb669-t9pjs to be ready.OK


-----------------------------------------------------------------------------------
install PMM Server
-----------------------------------------------------------------------------------

Error: uninstall: Release not loaded: monitoring: release: not found
Error: no repo named "stable" found
"stable" has been added to your repositories
NAME: monitoring
LAST DEPLOYED: Thu Oct  5 14:38:44 2023
NAMESPACE: monitoring-2-0-27394
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
PMM server can be accessed via HTTPS (port 443) on the following DNS name from within your cluster:

endpoint: https://monitoring-service.monitoring-2-0-27394.svc.cluster.local:443
login:    admin
password: admin
error: unable to upgrade connection: container not found ("monitoring")
error: unable to upgrade connection: container not found ("monitoring")
error: unable to upgrade connection: container not found ("monitoring")
error: unable to upgrade connection: container not found ("monitoring")
Retry 0
command terminated with exit code 1


-----------------------------------------------------------------------------------
create secrets and start client
-----------------------------------------------------------------------------------

secret/some-users created
secret/some-users unchanged
deployment.apps/psmdb-client created


-----------------------------------------------------------------------------------
create first PSMDB cluster monitoring
-----------------------------------------------------------------------------------

perconaservermongodb.psmdb.percona.com/monitoring created
waiting for pod/monitoring-rs0-0 to be ready............OK
waiting for pod/monitoring-rs0-1 to be ready.......OK
waiting for pod/monitoring-rs0-2 to be ready.....OK
Waiting for cluster readyness...........


-----------------------------------------------------------------------------------
check if pmm-client container is not enabled
-----------------------------------------------------------------------------------

Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://monitoring-mongos.monitoring-2-0-27394.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2023-10-05T14:43:44.396Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread1","msg":"Started a new thread for the timer service"}
Implicit session: session { "id" : UUID("aa1a081b-97d2-4499-8167-918bb7b075a4") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
Successfully added user: {
	"user" : "myApp",
	"roles" : [
		{
			"db" : "myApp",
			"role" : "readWrite"
		}
	]
}
bye
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://monitoring-mongos.monitoring-2-0-27394.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2023-10-05T14:43:51.317Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread1","msg":"Started a new thread for the timer service"}
Implicit session: session { "id" : UUID("d86e0515-d508-47e8-abf9-244488aeaf27") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
{
	"ok" : 1,
	"$clusterTime" : {
		"clusterTime" : Timestamp(1696517031, 4),
		"signature" : {
			"hash" : BinData(0,"mc0+x6KlSyg2dBeg8ij2whJrOOg="),
			"keyId" : NumberLong("7286484752935157784")
		}
	},
	"operationTime" : Timestamp(1696517031, 2)
}
bye
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://monitoring-mongos.monitoring-2-0-27394.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2023-10-05T14:43:55.864Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread1","msg":"Started a new thread for the timer service"}
Implicit session: session { "id" : UUID("8e58542e-c658-4807-944f-08c189feac30") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://monitoring-mongos.monitoring-2-0-27394.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2023-10-05T14:44:01.821Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread1","msg":"Started a new thread for the timer service"}
Implicit session: session { "id" : UUID("fda33237-4b1d-4948-b1ad-ee5144497a0b") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://monitoring-mongos.monitoring-2-0-27394.svc.cluster.local:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2023-10-05T14:44:08.743Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread1","msg":"Started a new thread for the timer service"}
Implicit session: session { "id" : UUID("7bb6adfd-0ca2-428c-aa2b-f78396a026bb") }
Percona Server for MongoDB server version: v5.0.20-17
WARNING: shell and server versions do not match
switched to db myApp
WriteResult({ "nInserted" : 1 })
bye


-----------------------------------------------------------------------------------
add PMM_SERVER_API_KEY for secret some-users
-----------------------------------------------------------------------------------

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   155  100   119  100    36    253     76 --:--:-- --:--:-- --:--:--   329
secret/some-users patched


-----------------------------------------------------------------------------------
check if all 3 Pods started
-----------------------------------------------------------------------------------

waiting for pod/monitoring-rs0-0 to be ready.OK
waiting for pod/monitoring-rs0-1 to be ready.OK
waiting for pod/monitoring-rs0-2 to be ready.OK
Waiting for cluster readyness.............................................................


-----------------------------------------------------------------------------------
check if pmm-client container enabled
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
check mongod metrics
-----------------------------------------------------------------------------------

"1696511423"
"1696511423"
"2"
"2"


-----------------------------------------------------------------------------------
check mongo config  metrics
-----------------------------------------------------------------------------------

"1696511423"
"1696511423"
"2"
"2"


-----------------------------------------------------------------------------------
check mongos metrics
-----------------------------------------------------------------------------------

"1696511827"
"1696511827"


-----------------------------------------------------------------------------------
check QAN data
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
check for passwords leak
-----------------------------------------------------------------------------------

secrets=YmFja3VwMTIzNDU2 Y2x1c3RlckFkbWluMTIzNDU2 Y2x1c3Rlck1vbml0b3IxMjM0NTY= ZGF0YWJhc2VBZG1pbjEyMzQ1Ng== dXNlckFkbWluMTIzNDU2 YmFja3VwMTIzNDU2 Y2x1c3RlckFkbWluMTIzNDU2 Y2x1c3Rlck1vbml0b3IxMjM0NTY= ZGF0YWJhc2VBZG1pbjEyMzQ1Ng== dXNlckFkbWluMTIzNDU2
passwords=backup123456 clusterAdmin123456 clusterMonitor123456 databaseAdmin123456 userAdmin123456 backup123456 clusterAdmin123456 clusterMonitor123456 databaseAdmin123456 userAdmin123456 YmFja3VwMTIzNDU2 Y2x1c3RlckFkbWluMTIzNDU2 Y2x1c3Rlck1vbml0b3IxMjM0NTY= ZGF0YWJhc2VBZG1pbjEyMzQ1Ng== dXNlckFkbWluMTIzNDU2 YmFja3VwMTIzNDU2 Y2x1c3RlckFkbWluMTIzNDU2 Y2x1c3Rlck1vbml0b3IxMjM0NTY= ZGF0YWJhc2VBZG1pbjEyMzQ1Ng== dXNlckFkbWluMTIzNDU2
pods=monitoring-0 monitoring-cfg-0 monitoring-cfg-1 monitoring-cfg-2 monitoring-mongos-0 monitoring-mongos-1 monitoring-mongos-2 monitoring-rs0-0 monitoring-rs0-1 monitoring-rs0-2 percona-server-mongodb-operator-6b6bcb669-t9pjs psmdb-client-5756fb77b9-hl6hm
logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-0-monitoring.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-cfg-0-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-cfg-1-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-cfg-2-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-mongos-0-mongos.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-mongos-1-mongos.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-mongos-2-mongos.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-rs0-0-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-rs0-1-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-monitoring-rs0-2-mongod.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-percona-server-mongodb-operator-6b6bcb669-t9pjs-percona-server-mongodb-operator.txt

logs saved in: /tmp/tmp.w4JyrBWZOL/logs_output-psmdb-client-5756fb77b9-hl6hm-psmdb-client.txt

release "monitoring" uninstalled


-----------------------------------------------------------------------------------
destroy cluster/operator and all other resources
-----------------------------------------------------------------------------------



-----------------------------------------------------------------------------------
get and delete old CRDs and RBAC
-----------------------------------------------------------------------------------

customresourcedefinition.apiextensions.k8s.io "perconaservermongodbbackups.psmdb.percona.com" deleted
customresourcedefinition.apiextensions.k8s.io "perconaservermongodbrestores.psmdb.percona.com" deleted
customresourcedefinition.apiextensions.k8s.io "perconaservermongodbs.psmdb.percona.com" deleted
error: the server doesn't have a resource type "perconaservermongodbbackups"
+ kubectl patch perconaservermongodbbackups.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbbackups"
error: the server doesn't have a resource type "perconaservermongodbrestores"
+ kubectl patch perconaservermongodbrestores.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbrestores"
error: the server doesn't have a resource type "perconaservermongodbs"
+ kubectl patch perconaservermongodbs.psmdb.percona.com -n sh --type=merge -p '{"metadata":{"finalizers":[]}}'
error: the server doesn't have a resource type "perconaservermongodbs"
role.rbac.authorization.k8s.io "percona-server-mongodb-operator" deleted
serviceaccount "percona-server-mongodb-operator" deleted
rolebinding.rbac.authorization.k8s.io "service-account-percona-server-mongodb-operator" deleted
customresourcedefinition.apiextensions.k8s.io "certificaterequests.cert-manager.io" deleted
customresourcedefinition.apiextensions.k8s.io "certificates.cert-manager.io" deleted
customresourcedefinition.apiextensions.k8s.io "challenges.acme.cert-manager.io" deleted
customresourcedefinition.apiextensions.k8s.io "clusterissuers.cert-manager.io" deleted
customresourcedefinition.apiextensions.k8s.io "issuers.cert-manager.io" deleted
customresourcedefinition.apiextensions.k8s.io "orders.acme.cert-manager.io" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-cainjector" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-issuers" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-clusterissuers" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-certificates" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-orders" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-challenges" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-ingress-shim" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-view" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-edit" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-approve:cert-manager-io" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-controller-certificatesigningrequests" deleted
clusterrole.rbac.authorization.k8s.io "cert-manager-webhook:subjectaccessreviews" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-cainjector" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-issuers" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-clusterissuers" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-certificates" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-orders" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-challenges" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-ingress-shim" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-approve:cert-manager-io" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-controller-certificatesigningrequests" deleted
clusterrolebinding.rbac.authorization.k8s.io "cert-manager-webhook:subjectaccessreviews" deleted
role.rbac.authorization.k8s.io "cert-manager-cainjector:leaderelection" deleted
role.rbac.authorization.k8s.io "cert-manager:leaderelection" deleted
rolebinding.rbac.authorization.k8s.io "cert-manager-cainjector:leaderelection" deleted
rolebinding.rbac.authorization.k8s.io "cert-manager:leaderelection" deleted
mutatingwebhookconfiguration.admissionregistration.k8s.io "cert-manager-webhook" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io "cert-manager-webhook" deleted


-----------------------------------------------------------------------------------
test passed
-----------------------------------------------------------------------------------