Re: [google-cloud-sql-discuss] 1 GB storage used when empty? 512MB RAM used when idle? (2G)

Hi Melvin,

f1-micro has 0.6GB of RAM. A lot of it is pre-allocated to the InnoDB buffer pool which is why it may show up as "used", but it doesn't necessarily mean MySQL is actively using all of it at any given time.

The 1GB used comes from the disk space pre-allocated for the InnoDB redo log.

Hope this helps,

 - Vadim

On Thu, Jan 28, 2016 at 7:23 PM Melvin Sommer <sommer.melvin@gmail.com> wrote:

Hey guys,

I'm using the Google Cloud products for the first time and have an SQL server set up for an app I'm currently deploying.

The SQL instance is a db-f1-micro tier, with 10GB storage reserved, of the second generation.

Now, I'm noticing that when the app is idle and there's no DB interaction, my admin panel shows 512MB RAM usage for the instance (whereas the tier should indicate a 125MB cap), and 1GB of storage space is used when it is actually empty.

Does any of this sound familiar? Do you have any pointers for me to get this issue resolved?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/47e88c57-8d4d-4e7e-8449-08c40e31cb44%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CABDU3p0OXx8CCQU5ih-G%3Dai_g7QhW5Ggauutc%3DKFe9x%3DQr3Qww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] 1 GB storage used when empty? 512MB RAM used when idle? (2G)

Hey guys,

I'm using the Google Cloud products for the first time and have an SQL server set up for an app I'm currently deploying.

The SQL instance is a db-f1-micro tier, with 10GB storage reserved, of the second generation.

Now, I'm noticing that when the app is idle and there's no DB interaction, my admin panel shows 512MB RAM usage for the instance (whereas the tier should indicate a 125MB cap), and 1GB of storage space is used when it is actually empty.

Does any of this sound familiar? Do you have any pointers for me to get this issue resolved?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/47e88c57-8d4d-4e7e-8449-08c40e31cb44%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

That did the trick indeed, thank you!

On Thursday, 28 January 2016 21:42:33 UTC+1, David Newgas wrote:

The service account has read permission on the bucket but not on the object. Bucket read allows you to list objects but you need object read to access their contents. Try running 'gsutil acl -u <service-acct>:r <object>'. If you set the default object acl then new objects will have the permission automatically.

David

On Thu, Jan 28, 2016 at 11:42 AM, <bj...@otainsight.com> wrote:

The EmailAddress from the output already had read permissions for the bucket.

Thanks for your help.

On Thursday, 28 January 2016 18:57:21 UTC+1, David Newgas wrote:

In fact, could you run "gcloud sql instances describe <target instance>" and note the serviceAccountEmailAddress in the output. Try granting that email address read access to your object and then importing again (rather than making the object public). I'm going to check in with the Cloud SQL team whether this is supposed to be necessary and whether we are missing it from our docs.

On Thu, Jan 28, 2016 at 9:50 AM, David Newgas <dne...@google.com> wrote:

Hi,

I suspect this is a issue with a Second Generation instances not having permissions to access the data in GCS. If acceptable security-wise, could you see if you can import from a publicly accessible object?

Also please note that if you need to load the data urgently you will be able to do so (but slower and at higher cost) by downloading the file to your workstation and loading it with a remote MySQL connection.

David

On Thu, Jan 28, 2016 at 1:08 AM, <bj...@otainsight.com> wrote:

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2a082ced-7169-40ea-9775-9c9247df3ffa%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/b14c879b-054f-41c9-961c-37360c5529eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

The service account has read permission on the bucket but not on the object. Bucket read allows you to list objects but you need object read to access their contents. Try running 'gsutil acl -u <service-acct>:r <object>'. If you set the default object acl then new objects will have the permission automatically.

David

On Thu, Jan 28, 2016 at 11:42 AM, <bjorn@otainsight.com> wrote:

The EmailAddress from the output already had read permissions for the bucket.

Thanks for your help.

On Thursday, 28 January 2016 18:57:21 UTC+1, David Newgas wrote:

In fact, could you run "gcloud sql instances describe <target instance>" and note the serviceAccountEmailAddress in the output. Try granting that email address read access to your object and then importing again (rather than making the object public). I'm going to check in with the Cloud SQL team whether this is supposed to be necessary and whether we are missing it from our docs.

On Thu, Jan 28, 2016 at 9:50 AM, David Newgas <dne...@google.com> wrote:

Hi,

I suspect this is a issue with a Second Generation instances not having permissions to access the data in GCS. If acceptable security-wise, could you see if you can import from a publicly accessible object?

Also please note that if you need to load the data urgently you will be able to do so (but slower and at higher cost) by downloading the file to your workstation and loading it with a remote MySQL connection.

David

On Thu, Jan 28, 2016 at 1:08 AM, <bj...@otainsight.com> wrote:

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2a082ced-7169-40ea-9775-9c9247df3ffa%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAJZK_bY8JQogYaSKqhtc1y07MYq8Ji9xRSmEMZz5OAfaQZWfhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

The EmailAddress from the output already had read permissions for the bucket.

Thanks for your help.

On Thursday, 28 January 2016 18:57:21 UTC+1, David Newgas wrote:

In fact, could you run "gcloud sql instances describe <target instance>" and note the serviceAccountEmailAddress in the output. Try granting that email address read access to your object and then importing again (rather than making the object public). I'm going to check in with the Cloud SQL team whether this is supposed to be necessary and whether we are missing it from our docs.

On Thu, Jan 28, 2016 at 9:50 AM, David Newgas <dne...@google.com> wrote:

Hi,

I suspect this is a issue with a Second Generation instances not having permissions to access the data in GCS. If acceptable security-wise, could you see if you can import from a publicly accessible object?

Also please note that if you need to load the data urgently you will be able to do so (but slower and at higher cost) by downloading the file to your workstation and loading it with a remote MySQL connection.

David

On Thu, Jan 28, 2016 at 1:08 AM, <bj...@otainsight.com> wrote:

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2a082ced-7169-40ea-9775-9c9247df3ffa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

In fact, could you run "gcloud sql instances describe <target instance>" and note the serviceAccountEmailAddress in the output. Try granting that email address read access to your object and then importing again (rather than making the object public). I'm going to check in with the Cloud SQL team whether this is supposed to be necessary and whether we are missing it from our docs.

On Thu, Jan 28, 2016 at 9:50 AM, David Newgas <dnewgas@google.com> wrote:

Hi,

I suspect this is a issue with a Second Generation instances not having permissions to access the data in GCS. If acceptable security-wise, could you see if you can import from a publicly accessible object?

Also please note that if you need to load the data urgently you will be able to do so (but slower and at higher cost) by downloading the file to your workstation and loading it with a remote MySQL connection.

David

On Thu, Jan 28, 2016 at 1:08 AM, <bjorn@otainsight.com> wrote:

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAJZK_baQFHBKsEmOspM3sr9kRFTSWqE2iQ%2BS_y1RFG5iwp5Oew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

Hi,

I suspect this is a issue with a Second Generation instances not having permissions to access the data in GCS. If acceptable security-wise, could you see if you can import from a publicly accessible object?

Also please note that if you need to load the data urgently you will be able to do so (but slower and at higher cost) by downloading the file to your workstation and loading it with a remote MySQL connection.

David

On Thu, Jan 28, 2016 at 1:08 AM, <bjorn@otainsight.com> wrote:

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAJZK_bbe-v8Ox%3D%2Bg0xVhf1ftNAoYH9KaU-J9PqZFOn4oR2RGUw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Server down, stuck in backup? HELP

Can you send an email to cloud-sql@google.com with your instance name and reference to this post? Thanks.

On Thursday, January 28, 2016 at 6:10:18 AM UTC-8, Graphics Geeks wrote:

I've left the server alive for now in case anyone from Google wants to inspect it to find out what happened.

It's still backing up.....12 hours later.....

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/39d3212f-59da-4d40-96f4-5d8541b8f09f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Server down, stuck in backup? HELP

I've left the server alive for now in case anyone from Google wants to inspect it to find out what happened.

It's still backing up.....12 hours later.....

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/71fe2d5b-0c82-4de3-b611-277210ba911f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Server down, stuck in backup? HELP

Ok, thankfully I had a replica server running that I could recover my data from.

In total I lost 9 hours of server uptime while I was asleep.

This has damaged my reputation and I'm extremely unhappy about this.

Google described this as a beta so I knew there would be some downtime. But not fatal errors that could have lost my entire database.

This was only my second day with Second Generation and I experienced 2 huge problems. There should be warning labels all over this thing.

WARNING - UNSTABLE ENVIRONMENT - RISK OF DATABASE LOSS

I'm now downgraded to first generation. I pray it's more stable.

On Thursday, January 28, 2016 at 10:39:11 AM UTC+1, Graphics Geeks wrote:

My SQL instance master server is down and seems to be stuck in backup.

It's been down at least half an hour HELP!!! What do I do???

Is my database lost??

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/648a7cea-95fb-4c8e-8ae2-0650e2d9b1d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Server down, stuck in backup? HELP

JUST CHECKED MY PHP SERVER - SQL HAS BEEN DOWN SINCE 3AM!!!

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/38fbf835-5407-4229-ad2e-92cb07be00a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Server down, stuck in backup? HELP

PLEASE HELP - IT SHOULD HAVE ENDED BACKUP AT 4AM AND IT'S 10AM NOW!!!!!

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e7aff9b2-ff88-4110-bbfc-c7c8da9f429f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Server down, stuck in backup? HELP

My SQL instance master server is down and seems to be stuck in backup.

It's been down at least half an hour HELP!!! What do I do???

Is my database lost??

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/71ae0190-2fc7-4e2a-af8f-9000d54c48b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Importing an export of an sql instance fails via command line, succeeds via cloud console

When I try to import an export of an sql instance via the command line, I get the following output:

Importing Cloud SQL instance...failed,

ERROR: (gcloud.sql.instances.import) UNKNOWN

The command I use is: 

> gcloud sql instances import <target instance> gs://<mybucket>/<file.gz>

However, when I import the file to the target instance using the cloud console, I don't get any errors.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8cd8147b-488a-4dd4-96e6-f05ebeaa3ec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Cloud SQL First Generation hang this morning?

Hello Allen,

Do you have query log enabled on your CloudSQL instance ? If yes, you can query the log table which might provide you with more insight. You can refer to this link for more information on slow query log. If its not enabled, you can visit the link for steps to enable it to debug the issue in future.

I hope that helps.

Faizan

On Tuesday, January 26, 2016 at 11:38:06 AM UTC-5, Allen Firstenberg wrote:

Our automated processes on GCE went to execute a DROP TABLE command on one of our Cloud SQL tables this morning around 3am ET. This is part of a regular set of commands that are issued this time every week. The Cloud SQL instance is a First Generation D4.

Most weeks it takes about 3 seconds.

This week it took over 2.5 hours - until 5:41:35 am ET.

The CPU usage graph on our cloud console also indicates no CPU activity on the Cloud SQL instance until this time as well.

The GCE status page reports no issues during this time.

Is there any way we can figure out what happened and how to prevent it from happening again?

Thanks!

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/cefa4826-9ff2-4940-a0b3-6b116cf99499%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Queries related to Cloud SQL concurrent connections and requests

Replies in-line:

On Tue, Jan 26, 2016 at 11:39 PM, Kartik Domadiya <kartik.domadiya@searce.com> wrote:

a) What is Pending Connection and how does it affect ?

        Not sure if this throws 1033 Instance has too many concurrent requests, when it exceeds 100. How do we handle this ? Is it like we can create 250 connections but use only 100 at a time ?

The key word here is "pending". This is really a limitation on the rate at which new connections can be opened. Due to our connection routing infrastructure we have a limit on the number of connections that can be in an internal startup phase at once.  As the FAQ explains: "Establishing a connection takes, on the server side, about 1.25 ms; because of the 100 pending connection limit, this means a maximum of 800 connection per second. If more than 100 clients try to connect simultaneously then some them will fail."

b) 250 Concurrent Connections.

        Should throw error Too Many Connections if it exceeds 250

This is correct. The key word here is "concurrent", as this is the number of connections that exist at a given moment, rather than anything to do with the rate at which they are opened.

 

c) Per app engine instance 12 concurrent connections per SQL Instance. How do we ensure that no more than 12 connections per app engine instance ? 

    Have gone through 

            http://stackoverflow.com/questions/10424442/what-are-the-connection-limits-for-google-cloud-sql-from-app-engine-and-how-to

            http://stackoverflow.com/questions/10158805/whats-a-good-approach-to-managing-the-db-connection-in-a-google-cloud-sql-gae/10438622#10438622

    But people face certain issues in that.

If you have one connection per request then you can ensure <= 12 connections per app engine instance by doing two things: (a) don't increase max-concurrent-requests above 12 and (b) ensuring you close db connections at the end of the request. If you use connection pooling, ensure you have less than 12 connections and they are closed cleanly when no longer used.

 

d) We got OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") error when     we tried a test with 1000+ requests concurrently.

This error message occurs for lots of different problems. My guess is you are hitting the 100 pending connection limit. Try ensuring you open less than 800 connections per second (but you can keep them open >1s, allowing for >800 concurrent connections).

 

We have 1500+ people(as of now and it will grow) using our system concurrently and looks like it will fail. So we are just confused if we can use Cloud SQL due to the above mentioned issues. But solutions should be available though. 

If you are using D0 or D1 but reaching limits of your instance, try moving to a larger tier.  Also you can try Cloud SQL second generation instances (in beta) which have some better performance characteristics.

Yours,

David

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAJZK_bbgL5MC5-AUwAv301A8woVOfG0P34RpEuuzEnbcHtog8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Queries related to Cloud SQL concurrent connections and requests

No Sandeep,
I am stuck in understanding the above mentioned three points If you can help.

On Wednesday, January 27, 2016 at 3:07:26 PM UTC+5:30, sande...@sruthika.net wrote:

Hi karthik,

Even my scenario is similar.I have users more than 10k!! what is the final solution??

On Wednesday, January 27, 2016 at 1:09:17 PM UTC+5:30, Kartik Domadiya wrote:

I need help in understanding few things related to Cloud SQL.

We have an app on App Engine using Datastore. Now we have requirement of using Cloud SQL with Datastore. The data is migrated from Datastore to Cloud SQL via cron jobs on Appengine.

I was just going through the below links to know the basic details related to performance and limitations.
https://cloud.google.com/sql/docs/diagnose-issues#data-issues
https://cloud.google.com/sql/pricing#v1-pricing
https://cloud.google.com/sql/faq#sizeqps

So for looks like First Generation Tier D0 - D1 will serve the purpose.

Few things which are confusing me:

a) What is Pending Connection and how does it affect ?

        Not sure if this throws 1033 Instance has too many concurrent requests, when it exceeds 100. How do we handle this ? Is it like we can create 250 connections but use only 100 at a time ?

b) 250 Concurrent Connections.

        Should throw error Too Many Connections if it exceeds 250

c) Per app engine instance 12 concurrent connections per SQL Instance. How do we ensure that no more than 12 connections per app engine instance ? 

    Have gone through 

            http://stackoverflow.com/questions/10424442/what-are-the-connection-limits-for-google-cloud-sql-from-app-engine-and-how-to

            http://stackoverflow.com/questions/10158805/whats-a-good-approach-to-managing-the-db-connection-in-a-google-cloud-sql-gae/10438622#10438622

    But people face certain issues in that.

d) We got OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") error when     we tried a test with 1000+ requests concurrently.

We have 1500+ people(as of now and it will grow) using our system concurrently and looks like it will fail. So we are just confused if we can use Cloud SQL due to the above mentioned issues. But solutions should be available though. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/69c69102-86ac-43a9-a16d-a093ccf47abe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Queries related to Cloud SQL concurrent connections and requests

Hi karthik,

Even my scenario is similar.I have users more than 10k!! what is the final solution??

On Wednesday, January 27, 2016 at 1:09:17 PM UTC+5:30, Kartik Domadiya wrote:

I need help in understanding few things related to Cloud SQL.

We have an app on App Engine using Datastore. Now we have requirement of using Cloud SQL with Datastore. The data is migrated from Datastore to Cloud SQL via cron jobs on Appengine.

I was just going through the below links to know the basic details related to performance and limitations.
https://cloud.google.com/sql/docs/diagnose-issues#data-issues
https://cloud.google.com/sql/pricing#v1-pricing
https://cloud.google.com/sql/faq#sizeqps

So for looks like First Generation Tier D0 - D1 will serve the purpose.

Few things which are confusing me:

a) What is Pending Connection and how does it affect ?

        Not sure if this throws 1033 Instance has too many concurrent requests, when it exceeds 100. How do we handle this ? Is it like we can create 250 connections but use only 100 at a time ?

b) 250 Concurrent Connections.

        Should throw error Too Many Connections if it exceeds 250

c) Per app engine instance 12 concurrent connections per SQL Instance. How do we ensure that no more than 12 connections per app engine instance ? 

    Have gone through 

            http://stackoverflow.com/questions/10424442/what-are-the-connection-limits-for-google-cloud-sql-from-app-engine-and-how-to

            http://stackoverflow.com/questions/10158805/whats-a-good-approach-to-managing-the-db-connection-in-a-google-cloud-sql-gae/10438622#10438622

    But people face certain issues in that.

d) We got OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") error when     we tried a test with 1000+ requests concurrently.

We have 1500+ people(as of now and it will grow) using our system concurrently and looks like it will fail. So we are just confused if we can use Cloud SQL due to the above mentioned issues. But solutions should be available though. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2b4d75f7-a5a7-42a9-9094-205be3ed95cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Queries related to Cloud SQL concurrent connections and requests

I need help in understanding few things related to Cloud SQL.

We have an app on App Engine using Datastore. Now we have requirement of using Cloud SQL with Datastore. The data is migrated from Datastore to Cloud SQL via cron jobs on Appengine.

I was just going through the below links to know the basic details related to performance and limitations.
https://cloud.google.com/sql/docs/diagnose-issues#data-issues
https://cloud.google.com/sql/pricing#v1-pricing
https://cloud.google.com/sql/faq#sizeqps

So for looks like First Generation Tier D0 - D1 will serve the purpose.

Few things which are confusing me:

a) What is Pending Connection and how does it affect ?

        Not sure if this throws 1033 Instance has too many concurrent requests, when it exceeds 100. How do we handle this ? Is it like we can create 250 connections but use only 100 at a time ?

b) 250 Concurrent Connections.

        Should throw error Too Many Connections if it exceeds 250

c) Per app engine instance 12 concurrent connections per SQL Instance. How do we ensure that no more than 12 connections per app engine instance ? 

    Have gone through 

            http://stackoverflow.com/questions/10424442/what-are-the-connection-limits-for-google-cloud-sql-from-app-engine-and-how-to

            http://stackoverflow.com/questions/10158805/whats-a-good-approach-to-managing-the-db-connection-in-a-google-cloud-sql-gae/10438622#10438622

    But people face certain issues in that.

d) We got OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") error when     we tried a test with 1000+ requests concurrently.

We have 1500+ people(as of now and it will grow) using our system concurrently and looks like it will fail. So we are just confused if we can use Cloud SQL due to the above mentioned issues. But solutions should be available though. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f0a046ff-d1b6-4c7c-8342-9fa7aab1de4f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Can't access read replica/slave?

I think this is the same post as http://stackoverflow.com/questions/35010302/google-cloud-sql-read-slave-replica-connection-and-query ?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/97540159-3392-4d78-86ae-4449a91afbf0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Testing Cloud SQL Failover

I work in the Cloud SQL team in Google. I can assure that nothing is leaked for your instance :) (okay, maybe the settings version, I will email you privately next time...)

On Tuesday, January 26, 2016 at 1:16:34 AM UTC-8, Peter Cheesewright wrote:

I had not realized that the settingsVersion changes - it all makes sense now. With the correct value the curl command works.

Thank you all for your help.

Herman, how are you able to see my settingsVersion? Have I left something open to the world?

On Wednesday, January 20, 2016 at 10:34:32 PM UTC, Herman Tai wrote:

Maybe you have done something between your last reply and now, but I am seeing your version is not at 29, so you get the error.

On Wednesday, January 20, 2016 at 2:47:03 AM UTC-8, Peter Cheesewright wrote:

Forgot to mention that my curl command now looks like this:

curl --header 'Content-Type: application/json' --header 'Authorization: Bearer OAUTH_ACCESS_TOKEN' -X POST "https://www.googleapis.com/sql/v1beta4/projects/numeric-datum-118114/instances/db-master/failover" --data '{"failoverContext":{"settingsVersion":14}}'

With the following error returned:

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "staleData",
    "message": "Condition does not match.",
    "locationType": "header",
    "location": "If-Match"
   }
  ],
  "code": 412,
  "message": "Condition does not match."
 }
}

On Wednesday, January 20, 2016 at 10:42:01 AM UTC, Peter Cheesewright wrote:

I have now managed to initiate failover via the tryit utility that Vishnu pointed to above. I did have to set the failoverContext.settingsVersion but also I had to change the instance ID that I was passing from "asia-east1:db-master" to just "db-master" for it to work (putting "asia-east1:db-master" resulted in the error "The client is not authorized to make this request.").

However I still cannot get the curl command to work. I have tried using the access token returned by

gcloud auth print-access-token

as David suggested but without success. (Why is "print-access-token" not documented in "gcloud help auth" ?)

Checking that token with 

https://apis-explorer.appspot.com/apis-explorer/#search/oauth2/oauth2/v2/oauth2.tokeninfo
gives the expected email address, the following for the 'scope' property:

"scope": "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/plus.me",

and the "access_type" as "offline".

Does this seem reasonable? I notice that the scope does not contain "https://www.googleapis.com/auth/sqlservice.admin" but does contain "https://www.googleapis.com/auth/cloud-platform" but then the documentation for the failover method says at least one of the 2 is required.

On Tuesday, January 19, 2016 at 10:59:10 PM UTC, Vishnu Fafat wrote:

Hi,

You will have to set failoverContext.settingsVersion in the Request body for calling failover on an instance at https://cloud.google.com/sql/docs/admin-api/v1beta4/instances/failover#try-it.

You can get the settingsVersion by listing instances in the project at https://cloud.google.com/sql/docs/admin-api/v1beta4/instances/list#try-it.

I will get the curl command documentation at https://cloud.google.com/sql/docs/high-availability#test fixed.

On Tue, Jan 19, 2016 at 2:26 PM, 'David Newgas' via Google Cloud SQL discuss <google-cloud...@googlegroups.com> wrote:

I still suggest trying gcloud auth print-access-token.

You might also want to try putting the access token into https://apis-explorer.appspot.com/apis-explorer/#search/oauth2/oauth2/v2/oauth2.tokeninfo and clicking execute. You should make sure that the scope includes those required by the API.  You should also check that the email returned has owner or editor permission on the project your instance is in.

On Tue, Jan 19, 2016 at 2:20 PM, Peter Cheesewright <pe...@microcosm.co.uk> wrote:

The token I'm using does start "ya29."

It's the token obtained from the output of

gcloud auth login

after completing the web flow.

Any other thoughts?

On Tuesday, January 19, 2016 at 5:36:48 PM UTC, David Newgas wrote:

I suspect you might be using a "authorization code" rather than an "access token". The authorization code exchanged as part of the "gcloud auth login" flow is just part of the  login process. Try using "gcloud auth print-access-token" to actually get an access token you can use. Google's access tokens (currently) start with "ya29." which should help you identify it.

David

On Tue, Jan 19, 2016 at 9:16 AM, Peter Cheesewright <pe...@microcosm.co.uk> wrote:

I've been trying to test SQL failover using the following command:

curl --header 'Content-Type: application/json' --header 'Authorization: Bearer OATH_ACCESS_TOKEN_HERE' -X POST "https://www.googleapis.com/sql/v1beta4/projects/numeric-datum-118114/instances/asia-east1:db-master/failover" --data "{}"

I started with the command here:

https://cloud.google.com/sql/docs/high-availability#test

but it seems to be a bit off the mark. Eventually I ended up with the curl command above but I keep getting the following response:

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

The actual access token I'm using in the curl command is the one returned by:

gcloud auth login

I'm assuming this is the wrong thing to do? 

How do I acquire an access token that allows me to make this call?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/5ed3f23d-b82a-481f-8885-7b986bce0966%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c2543eec-71ed-426f-a06f-6672c4a0012e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAJZK_baXxbZRhD0Ls2ntR4FVcNoD-EP1x-dOtwx%2B5597-w23GA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Vishnu Fafat

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8bfbb307-8406-4596-9d58-cd41d87c6341%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-cloud-sql-discuss] Google Cloud SQL (Second Generation) - Read Replica Instances Inoperable - Bug Report

Ok, yes it is in beta, I'm not upset about it. And thanks again....

Just would really love to get access to that read server so I can shift all the heavy duty reads over there...

On Tuesday, January 26, 2016 at 9:08:09 PM UTC+1, Vishnu Fafat wrote:

On Tue, Jan 26, 2016 at 11:47 AM, Graphics Geeks <graphic...@gmail.com> wrote:

Ok, I added a user on the master with a new password. And I made sure it had all the same privileges as the root.

I also flushed the privileges.

Doesn't seem to connect still? Does it take a day or something to be replicated over into the slave?

The changes are near-instantly propagated. Please send an email to clou...@google.com with the instance details and the commands that you are running to help you debug this issue. (Also, ignore its auto reply).

 

I totally appreciate your help. From the google architecture side of things though this all seems pretty crazy. 

Why not just have a configurable user list on the control panel? I know making forms is a pain, but come on - seriously?

The feature still needs work to be enabled for Second Generation instances.

 

On Tuesday, January 26, 2016 at 7:25:33 PM UTC+1, Vishnu Fafat wrote:

Hi,

The users from the master are replicated to the replica. So, you should be able to connect to the replica if you provide the right information. These are the steps I followed:

1. Add a user on master instance for all hosts. (CREATE USER 'abcdefg'@'%' IDENTIFIED BY 'abcdef';)

2. Flush privileges.

3. Connect to the replica instance.

On Tue, Jan 26, 2016 at 9:31 AM, Graphics Geeks <graphic...@gmail.com> wrote:

Hello Lee,

Thanks for helping me.

I'm a bit confused about this, since it doesn't seem from the interface that I can even set any users for a read replica instance.

And setting the root password is not allowed for the replica - it says it has to be the same as the master?

Kind regards

On Tuesday, January 26, 2016 at 6:00:57 PM UTC+1, Lee S. wrote:

You need to set the user passwords for the replica separately.  You can do this from the cloud console under 'access control > users'.  The system tables don't get replicated so the users have to be created on each replica separately.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/98c7fec1-e5cb-443f-8122-c2a53a072caa%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Vishnu Fafat

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/36317033-2900-4e20-adad-6a56f027dfd0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Vishnu Fafat

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9f913348-1f48-4339-b033-c589a2f0e66a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.