Thursday, May 31, 2018

[google-cloud-sql-discuss] Re: Cloud SQL proxy causes significant additional latency

Hello Daniel

It appears to be related to the github issue you mentioned

Currently, I am not aware of any workaround about the cloud proxy latency caused  by the connection handshakes. However, You mentioned that with connection pooling it is working fine, which is related to decrease in the number of new connection handshakes. Cloud SQL instance can be directly accessed from GKE, using the Cloud SQL external IP within the container with appropriate routing. In addition, you also use secrets to store the Cloud SQL information and use them in application. Here is one of the third-party link describing this process. it will also require to manage the whitelisting IPs for the Cloud SQL connections. 

By default for direct connection, Cloud SQL connections may not be encrypted. It requires to configure SSL as per this documentation. Also, I found this discussion thread providing more insight on the same. 

Our engineering team is making efforts for cloudsql-proxy, to reduce the latency spike almost every hour by refreshing the SSL certificate before it expires its lifetime of one hour. You can keep track of this git hub feature request or Google public feature request for further updates regarding this. 


On Wednesday, May 30, 2018 at 7:32:26 AM UTC-4, Daniel Alm wrote:
(This may be related to https://github.com/GoogleCloudPlatform/cloudsql-proxy/issues/87.)

I've been experimenting with using Cloud SQL Proxy for my PostgreSQL database vs. connecting directly via TCP. To illustrate, here's my Stackdriver Trace latency graph:

screen shot 2018-05-27 at 20 22 32

Around 18:15 I switched from directly connecting via TCP from GKE to Cloud SQL over to using a Cloud SQL proxy sidecar that my service connects to via a Unix socket (connecting to the proxy via TCP yields very similar results). As one can see, there are a few spikes in latency when the connection pools get warmed up, and then the latency distribution is significantly above the latency before, which was for direct connections.

My Stackdriver Trace spans also indicate that the increase in overall latency can be attributed to an increase in database call latency. Trivial database calls that normally take <=10ms via a direct connection take up to 100ms through the proxy.

Is this expected, can I do anything about it, and, if there is no remedy, what would be the best way to connect to the DB via TCP from GKE? I'm currently using a /16 netmask to allow connections grom GCE, but that's probably bad for production. Also, I'm not sure whether direct traffic to Cloud SQL is encrypted — if you could provide insights into that, that would be appreciated.

FYI, the GKE cluster and the Cloud SQL database are located in the same zone (europe-west3-a IIRC). The database is only ~500 MB right now (fits completely into memory and has appropriate indexes) and has low load. Also, as stated above, I am using connection pooling and have confirmed that that works.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/574a0fa0-0b54-4e64-9a3d-2c7594bffee2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment