We've spent some time looking at cloud sql as part of a larger-scoped migration from an existing datacenter to a kubernetes cluster running on Google Cloud platform. A key requirement for our production infrastructure is that we be able to automate deployment and management. In our existing systems we use puppet and other tools to manage databases and other components. We've made good headway on GCP using tools like make, gcloud, kubectl and in the future we will probably adopt helm and terraform.
However we've run into a few brick walls trying to automate deployment and configuration of second generation cloud SQL instances. I'd love to learn of good work-arounds for these issues, or to find that we've simply been doing things wrong. If it's not that simple then my goal would be to learn what the timeline is for seeing some resolution on the following:
1. Creating databases.
The `gcloud sql instances create` API seems incomplete when trying to create second gen. instances. The beta extensions add support for failover_replica and storage autoscaling, but not for initial storage size, maintenance window, etc. We ended up implementing a script to create instances through the REST json API and that worked fine, but it would be nice to know when gcloud will catch up.
2. Managing root users.
We could not figure out how to use the REST json API and an authorized service account to update the root user password for a new instance. Whenever we tried to submit a password update for root, for example, we received "This operation isn't valid for this instance." This same error results from trying `gcloud beta sql users list` or `gcloud sql instances set-root-password`. The documentation states that we should be using mysql client to manage users on second gen. instances. In order to connect with the mysql client we have to manually activate the instance root user in the console. If there's a way to make this work from our deployment script we'd really like to hear about it.
3. The connection model.
Even if we got past manually configuring the root user our options for connecting mysql clients to the database are not very appetizing. According to this page we would have to assign static IPs to instances and authorize those. We're not connecting directly from GCE in any case, so according to this other page we should be using Google's proxy container image.
The proxy appears to be one per instance. We have three large SQL databases to migrate. The examples of the proxy suggest running the container inside the pod with the app that needs access and addressing the database on localhost. If we need access to more than one database in a service I guess we would either need them on different ports or we'd have to pull the container out of the application's container network stack and wrap a clusterIP kubernetes service around it. Either way running three proxy services in the cluster in order to simply connect to a mysql database isn't a happy prospect.
When I read in the console documentation that private network ranges could not be added to an instance's authorized networks table I assumed that meant the feature was intended for access by external applications. It didn't occur to me that instances running in the private subnet of the same project the SQL instance is in would not be able to get packets to it. I suspect there might be a way to accomplish this using routes and the external interface of the project default gateway, but frankly this seems like an infrastructure issue we shouldn't be messing with. What I think should happen is that the project subnet(s) get access to cloud sql instances in the same project automatically.
For the moment our short term plans are to leave our large SQL databases in the existing datacenter and proxy requests to them across a VPN tunnel. We would very much like to migrate them, so any input on these issues would be really welcome.
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/554f3594-0fb0-48f7-ae61-d7fe4bd5adc3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
No comments:
Post a Comment