[google-cloud-sql-discuss] CloudSQL Proxy error: oauth2: cannot fetch token: Post https://oauth2.googleapis.com/token: dial tcp: i/o timeout

We have just migrated from a trial GCP account to a "real" one and are now unable to connect to our pre-existing MySQL CLoudSQL instance from GKE.

We have a pod (in a deployment) that has a cloudsql proxy container and a wordpress one (which I've replaced with a simple mysql container, running a while-true-sleep loop so we can exec in a test a command line mysql connection).  The client errors off with:

ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 107

And the cloudsql proxy log shows this:

2018/08/22 15:41:57 New connection for "PROJECT:us-central1:MYSQL_INSTANCE_ID"

2018/08/22 15:42:27 couldn't connect to "PROJECT:us-central1:MYSQL_INSTANCE_ID": Post https://www.googleapis.com/sql/v1beta4/projects/PROJECT/instances/MYSQL_INSTANCE_ID/createEphemeral?alt=json: oauth2: cannot fetch token: Post https://oauth2.googleapis.com/token: dial tcp: i/o timeout

Note the 30 seconds elapsed time.

The cloudsql-proxy is invoked with:

        command: ["/cloud_sql_proxy"]

        args: ["-instances=$(MYSQL_INSTANCE)",

               "-credential_file=/secrets/cloudsql/XXX-mysql-proxy-access.json",

               "-verbose=true"]

Where mysql-proxy-access.json contains the JSON credentials of a service account assigned the Cloud SQL Client role and $MYSQL_INSTANCE is PROJECT:us-central1:MYSQL_INSTANCE_ID=tcp:3306

I've checked to ensure that the contents of XXX-mysql-proxy-access.json match the key on the service account.

The cluster nodes are on v1.10.6-gke.1 and we are using the latest cloudsql proxy image (gcr.io/cloudsql-docker/gce-proxy:1.11).  We tried making the mysql instance reside in the same zone as the nodes (us-central1b) but nothing changed.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8a4eafb6-71ff-4abb-b2e8-18053fe5bba8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.