[google-cloud-sql-discuss] Upgrading Postgres from 9.6 to 13

I'd like to be able to upgrade my existing cloudsql postgres 9.6 instance to 11 to use some new pg 11 features.

I need to keep the same IP as the instance with 9.6. There are many applications.

I accept suggestion of less important strategies.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7109fa56-3593-4a56-9f34-ebcbaa38eed4n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Despite shutting down the instance, it continues to cost money

Hello,

As stated in this document, stopping the instance does not stop some charges such as storage (the data is still being stored even though the instance is stopped) and IP address. If you would like to avoid being charged in any way due to having this instance stopped, you would need to delete it. However, you could try contacting billing support to get more information about these charges you are seeing.

On Sunday, May 30, 2021 at 5:16:54 PM UTC-4 Nobuhito wrote:

Hi,

Thank you for your quick reply.

When I checked the details further, the SQL for writing was always executed, and the storage was constantly increasing. 

This write SQL is still running even if I stop the instance on Cloud SQL.

What is the SQL for this write?

How can I stop it?

Best regards,

2021年5月28日金曜日 1:00:14 UTC+9 Alvaro (Google Cloud Platform Support):

Hi,

I understand that your Cloud SQL instance is not running, but even if it is stopped some charges like storage and IP addresses still apply [1], so it is likely that the growing cost is coming from there. 

[1]

https://cloud.google.com/sql/docs/mysql/start-stop-restart-instance#stop

On Wednesday, May 26, 2021 at 10:34:08 AM UTC+2 Nobuhito wrote:

I registered for a 90-day free trial of google app engine.

After creating a project on google app engine, I created an instance on cloud sql and created a database with MySQL 8.0.

Currently the project is not running on google app engine. The instance on cloud sql is also stopped. However, SQL storage is still growing on its own, costing $ 82.65 in just four days.

Looking at the payment report, the following three are the main factors.

Cloud SQL for MySQL: Regional --RAM in JAPAN

Cloud SQL for MySQL: Regional --Standard storage in JAPAN

Cloud SQL for MySQL: Regional --vCPU in JAPAN

Both the project on google app engine and the instance on cloud sql are down, but the cost continues to be taken.

How can I stop it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e4f4f6f5-54df-4db1-9813-e4b830bb4089n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Despite shutting down the instance, it continues to cost money

Hi,

Thank you for your quick reply.

When I checked the details further, the SQL for writing was always executed, and the storage was constantly increasing. 

This write SQL is still running even if I stop the instance on Cloud SQL.

What is the SQL for this write?

How can I stop it?

Best regards,

2021年5月28日金曜日 1:00:14 UTC+9 Alvaro (Google Cloud Platform Support):

Hi,

I understand that your Cloud SQL instance is not running, but even if it is stopped some charges like storage and IP addresses still apply [1], so it is likely that the growing cost is coming from there. 

[1]

https://cloud.google.com/sql/docs/mysql/start-stop-restart-instance#stop

On Wednesday, May 26, 2021 at 10:34:08 AM UTC+2 Nobuhito wrote:

I registered for a 90-day free trial of google app engine.

After creating a project on google app engine, I created an instance on cloud sql and created a database with MySQL 8.0.

Currently the project is not running on google app engine. The instance on cloud sql is also stopped. However, SQL storage is still growing on its own, costing $ 82.65 in just four days.

Looking at the payment report, the following three are the main factors.

Cloud SQL for MySQL: Regional --RAM in JAPAN

Cloud SQL for MySQL: Regional --Standard storage in JAPAN

Cloud SQL for MySQL: Regional --vCPU in JAPAN

Both the project on google app engine and the instance on cloud sql are down, but the cost continues to be taken.

How can I stop it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d06f3feb-8f35-4c4b-a862-ebe3cf782c5an%40googlegroups.com.

[google-cloud-sql-discuss] Re: Trigger Importing Dump SQL File to Cloud SQL

Hello,

In order to get help from the community, I recommend that you post this how to question on StackOverFlow. You can check information about this in the Community Support article.

However, in this link you can find information on how to connect to Cloud SQL from Cloud Functions.

On Wednesday, May 26, 2021 at 10:30:00 AM UTC+2 Ahmed Mohamed wrote:

Hi everyone, 

The Scenario: 

I am receiving a dump SQL file every day on my Cloud storage bucket and I need to automate importing this file to Cloud SQL as soon as the file arrived in GCS. 

My suggestion is to use a Cloud function to trigger Cloud SQL when the file arrives in GCS. but I don't know how to achieve this process (importing to Cloud SQL) in the programmatic (Pythonic) way. 

 Does anybody have any idea that will help me achieve this?

Thanks, Ahmed

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/412bfba6-788f-4159-9bd3-667041e86046n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Despite shutting down the instance, it continues to cost money

Hi,

I understand that your Cloud SQL instance is not running, but even if it is stopped some charges like storage and IP addresses still apply [1], so it is likely that the growing cost is coming from there. 

[1]

https://cloud.google.com/sql/docs/mysql/start-stop-restart-instance#stop

On Wednesday, May 26, 2021 at 10:34:08 AM UTC+2 Nobuhito wrote:

I registered for a 90-day free trial of google app engine.

After creating a project on google app engine, I created an instance on cloud sql and created a database with MySQL 8.0.

Currently the project is not running on google app engine. The instance on cloud sql is also stopped. However, SQL storage is still growing on its own, costing $ 82.65 in just four days.

Looking at the payment report, the following three are the main factors.

Cloud SQL for MySQL: Regional --RAM in JAPAN

Cloud SQL for MySQL: Regional --Standard storage in JAPAN

Cloud SQL for MySQL: Regional --vCPU in JAPAN

Both the project on google app engine and the instance on cloud sql are down, but the cost continues to be taken.

How can I stop it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/60175244-7784-4e4f-acdc-04de6564f06cn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Export CloudSQL to Bigquery

Hi,

The easiest method is to use Federated queries[1], which allows you to read data from Cloud SQL as if it were in BigQuery.

The sample code you are looking for is scattered across different documents. For example, as as CSV export from SQL with Cloud Scheduler here[2]. And then load the CSV data from a bucket[3] into Big Query.

Then there's also sample code to write streams using Big Query's write API[4]. More about streaming data into BigQuery here[5].

As you can see, there are a lot of possibilities and all these code samples can be used on Cloud Functions. I think you need to decide first how you will export, plus also in what format and when. However, if appropriate to your workflow, none of this needs to be done if you just use Federated Queries. Federated Queries is the simplest method without impact on performance since BigQuery is an aggregator of sort that uses the infrastructure to run queries[6].

[1] https://cloud.google.com/bigquery/docs/working-with-connections

[2] https://cloud.google.com/sql/docs/mysql/import-export/exporting#automating_export_operations

[3] https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#loading_csv_data_into_a_table

[4] https://cloud.google.com/bigquery/docs/write-api#write_to_a_stream_in_committed_mode

[5] https://cloud.google.com/bigquery/streaming-data-into-bigquery

[6] https://cloud.google.com/bigquery/docs/introduction

On Wednesday, May 26, 2021 at 9:25:44 AM UTC-4 rgo...@ctepl.com wrote:

Hi,

I know we can use cloud functions.

But I am new to GCP.

Is there any sample code to schedule data load from cloudsql to BigQuery.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/bc463201-459c-4d5b-8ee7-d64bcd6bc85fn%40googlegroups.com.

[google-cloud-sql-discuss] Export CloudSQL to Bigquery

Hi,

I know we can use cloud functions.

But I am new to GCP.

Is there any sample code to schedule data load from cloudsql to BigQuery.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/3bdd247c-a870-4832-a7e5-13aa9f9697den%40googlegroups.com.

[google-cloud-sql-discuss] Trigger Importing Dump SQL File to Cloud SQL

Hi everyone, 

The Scenario: 

I am receiving a dump SQL file every day on my Cloud storage bucket and I need to automate importing this file to Cloud SQL as soon as the file arrived in GCS. 

My suggestion is to use a Cloud function to trigger Cloud SQL when the file arrives in GCS. but I don't know how to achieve this process (importing to Cloud SQL) in the programmatic (Pythonic) way. 

 Does anybody have any idea that will help me achieve this?

Thanks, Ahmed

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7ca76a2e-1e21-479d-85d9-b02152770fccn%40googlegroups.com.

[google-cloud-sql-discuss] Despite shutting down the instance, it continues to cost money

I registered for a 90-day free trial of google app engine.

After creating a project on google app engine, I created an instance on cloud sql and created a database with MySQL 8.0.

Currently the project is not running on google app engine. The instance on cloud sql is also stopped. However, SQL storage is still growing on its own, costing $ 82.65 in just four days.

Looking at the payment report, the following three are the main factors.

Cloud SQL for MySQL: Regional --RAM in JAPAN

Cloud SQL for MySQL: Regional --Standard storage in JAPAN

Cloud SQL for MySQL: Regional --vCPU in JAPAN

Both the project on google app engine and the instance on cloud sql are down, but the cost continues to be taken.

How can I stop it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f9b93681-2f69-467d-8ba3-47a8ee105917n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: How do I work around MySQL Bug #69721 (Block Nested Loop making things slower)

Hi, 

As seen on the public issue and the internal one, the feature for setting optimizer-switch=block_nested_loop=off is still under the evaluation of the Cloud SQL Engineering team. Please kindly note that there is no ETA or guarantee on the implementation. Thank you.

On Sunday, May 16, 2021 at 5:49:59 PM UTC-4 feja...@gmail.com wrote:

Hi,

Please how were you able to resolve this eventually, I am also stuck in the same situation.

Thanks

On Thursday, 4 April 2019 at 08:53:38 UTC+1 krzysz...@gmail.com wrote:

Thanks, I issued https://issuetracker.google.com/issues/129897196

On Thu, Apr 4, 2019 at 1:57 AM 'Harmit Rishi (Cloud Platform Support)'
via Google Cloud SQL discuss
<google-cloud...@googlegroups.com> wrote:
>
> Thank you for using Google Groups.
>
>
> Considering that Cloud SQL offers MySQL instances, I believe this issue should be brought to the attention of the engineering team behind the Cloud SQL product. Due to this, I encourage you to follow the steps to file a Public Issue Tracker here[1].
>
>
> Also, doing so will ensure that this issue is getting the appropriate exposure to the engineering team as Google Groups is meant for discussion purposes. Let me know if you are able to file a Public Issue Tracker for this.
>
>
> [1] Public Issue Tracker: https://cloud.google.com/support/docs/issue-trackers
>
> On Wednesday, February 27, 2019 at 2:26:49 PM UTC-5, Krzysztof Drys wrote:
>>
>> Recently we experienced problems with our MySQL instance. One hour our db was running fine (CPU usage at about 20%), next hour CPU is at 100% and queries are stuck at "creating sort index" phase. After restarting DB everything was normal again.
>>
>> We suspect that it was MySQL Bug #69721 (https://bugs.mysql.com/bug.php?id=69721). As suggested here (https://community.sugarcrm.com/thread/33079-queries-getting-stuck-on-creating-sort-index-slowing-sugar-down), one option to mitigate this bug is to set optimizer-switch=block_nested_loop=off.
>>
>> However, as far as I can see, it is not possible to set this option in Cloud MySQL. Do you know any other way to work around this bug?
>>
>> Thanks,
>> Krzysztof Dryś
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/cPeyrQ4Rbz0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to google-cloud-sql-d...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/68475f91-8f40-4dbb-80ff-64e76e5f0dc0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9a30318b-baaf-4c53-97ea-25a6d3b46e72n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Strange error log "Improper path /cloudsql/..."

Sorry I've just found the answer on Stackoverflow (question already posted yesterday) :

https://stackoverflow.com/questions/67595318/cloudrun-suddenly-got-improper-path-cloudsql-sql-connection-name-to-connect

Answer : 

This log was recently added in Cloud Run data path to provide more context for debugging CloudSQL connectivity issues. However, the original logic was overly aggressive, emitting this message even for properly working CloudSQL connections. Your application is working correctly and should not receive this warning.

Thank you for reporting this issue. The fix is ready and should roll out soon. You should not see this message anymore after the fix is out.

Here is also the issue  : 

https://issuetracker.google.com/issues/188462249

On Thursday, May 20, 2021 at 9:21:23 AM UTC+2 Thierry Falvo wrote:

Hello,

I'm using stack Cloud Run (NestJS/TypeOrm) + Cloud Sql. Everything is working fine since a long time, and with multiples services, projects and databases. 

However, since yesterday morning, I saw a strange error message in Cloud Run log when attempting to connect to Cloud SQL :

Improper path /cloudsql/INSTANCE_NAME to connect to Postgres Cloud SQL instance INSTANCE_NAME.

where INSTANCE_NAME is the correct name of my cloud sql instance.

But everything is still working as usual. No problem. DB Connection is OK. 

This new log error happened first after a service update. No changes in my configuration, nothing in my dependencies (node.js), except if child dependencies have been updated during container build...

Cloud run environment variables are OK (INSTANCE_NAME).

Have you ever seen this kind of message ?

Thank you for your help and feedback !

Regards

Thierry

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/bb443f22-dc83-48ad-9042-4772499c8953n%40googlegroups.com.

[google-cloud-sql-discuss] Strange error log "Improper path /cloudsql/..."

Hello,

I'm using stack Cloud Run (NestJS/TypeOrm) + Cloud Sql. Everything is working fine since a long time, and with multiples services, projects and databases. 

However, since yesterday morning, I saw a strange error message in Cloud Run log when attempting to connect to Cloud SQL :

Improper path /cloudsql/INSTANCE_NAME to connect to Postgres Cloud SQL instance INSTANCE_NAME.

where INSTANCE_NAME is the correct name of my cloud sql instance.

But everything is still working as usual. No problem. DB Connection is OK. 

This new log error happened first after a service update. No changes in my configuration, nothing in my dependencies (node.js), except if child dependencies have been updated during container build...

Cloud run environment variables are OK (INSTANCE_NAME).

Have you ever seen this kind of message ?

Thank you for your help and feedback !

Regards

Thierry

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/b9b3ea0a-532b-493c-a65d-3dc21737944fn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: why i got these queries on log every 1-2 seconds ?

Hi,

You can find your service account on the IAM & Admin > IAM page. Depending on if you create your own service account or used the default the may look like this[1]

[1]https://cloud.google.com/sql/docs/mysql/roles-and-permissions#serverless-roles-permissions

On Monday, May 17, 2021 at 9:17:47 AM UTC-4 ho...@interactivelabs.co wrote:

hi wushawn,

where can i find my service account ?

Thanks

hong

On Saturday, May 15, 2021 at 5:07:07 PM UTC-7 wushawn wrote:

Can you double confirm the service account you use  ? it seems something related to the IAM not set properly. 

On Friday, May 14, 2021 at 1:37:17 PM UTC-4 ho...@interactivelabs.co wrote:

no, i think because there are too many roles. 

Any suggestions for max roles for best practices?

Thanks

hong

On Thu, May 13, 2021 at 4:15 PM 'David (Cloud Platform Support)' via Google Cloud SQL discuss <google-cloud...@googlegroups.com> wrote:

Hello,

Do you have pg_stat_statements enabled? Seems like something it would be logged if you do. 

On Thursday, May 13, 2021 at 9:46:21 AM UTC-4 ho...@interactivelabs.co wrote:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/GDzR8uilu2c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-cloud-sql-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/da2919ec-6c99-46f7-a830-b8cd684fd17dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d3ddc4fe-965e-4915-a95f-71527feb711dn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: why i got these queries on log every 1-2 seconds ?

hi wushawn,

where can i find my service account ?

Thanks

hong

On Saturday, May 15, 2021 at 5:07:07 PM UTC-7 wushawn wrote:

Can you double confirm the service account you use  ? it seems something related to the IAM not set properly. 

On Friday, May 14, 2021 at 1:37:17 PM UTC-4 ho...@interactivelabs.co wrote:

no, i think because there are too many roles. 

Any suggestions for max roles for best practices?

Thanks

hong

On Thu, May 13, 2021 at 4:15 PM 'David (Cloud Platform Support)' via Google Cloud SQL discuss <google-cloud...@googlegroups.com> wrote:

Hello,

Do you have pg_stat_statements enabled? Seems like something it would be logged if you do. 

On Thursday, May 13, 2021 at 9:46:21 AM UTC-4 ho...@interactivelabs.co wrote:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/GDzR8uilu2c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-cloud-sql-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/da2919ec-6c99-46f7-a830-b8cd684fd17dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/508e71a2-abbc-413f-a4c0-49ba4e9ce93dn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: How do I work around MySQL Bug #69721 (Block Nested Loop making things slower)

Hi,

Please how were you able to resolve this eventually, I am also stuck in the same situation.

Thanks

On Thursday, 4 April 2019 at 08:53:38 UTC+1 krzysz...@gmail.com wrote:

Thanks, I issued https://issuetracker.google.com/issues/129897196

On Thu, Apr 4, 2019 at 1:57 AM 'Harmit Rishi (Cloud Platform Support)'
via Google Cloud SQL discuss
<google-cloud...@googlegroups.com> wrote:
>
> Thank you for using Google Groups.
>
>
> Considering that Cloud SQL offers MySQL instances, I believe this issue should be brought to the attention of the engineering team behind the Cloud SQL product. Due to this, I encourage you to follow the steps to file a Public Issue Tracker here[1].
>
>
> Also, doing so will ensure that this issue is getting the appropriate exposure to the engineering team as Google Groups is meant for discussion purposes. Let me know if you are able to file a Public Issue Tracker for this.
>
>
> [1] Public Issue Tracker: https://cloud.google.com/support/docs/issue-trackers
>
> On Wednesday, February 27, 2019 at 2:26:49 PM UTC-5, Krzysztof Drys wrote:
>>
>> Recently we experienced problems with our MySQL instance. One hour our db was running fine (CPU usage at about 20%), next hour CPU is at 100% and queries are stuck at "creating sort index" phase. After restarting DB everything was normal again.
>>
>> We suspect that it was MySQL Bug #69721 (https://bugs.mysql.com/bug.php?id=69721). As suggested here (https://community.sugarcrm.com/thread/33079-queries-getting-stuck-on-creating-sort-index-slowing-sugar-down), one option to mitigate this bug is to set optimizer-switch=block_nested_loop=off.
>>
>> However, as far as I can see, it is not possible to set this option in Cloud MySQL. Do you know any other way to work around this bug?
>>
>> Thanks,
>> Krzysztof Dryś
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/cPeyrQ4Rbz0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to google-cloud-sql-d...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/68475f91-8f40-4dbb-80ff-64e76e5f0dc0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/6d3ca1d4-e07a-426f-9f9a-1347914f4c8dn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: why i got these queries on log every 1-2 seconds ?

Can you double confirm the service account you use  ? it seems something related to the IAM not set properly. 

On Friday, May 14, 2021 at 1:37:17 PM UTC-4 ho...@interactivelabs.co wrote:

no, i think because there are too many roles. 

Any suggestions for max roles for best practices?

Thanks

hong

On Thu, May 13, 2021 at 4:15 PM 'David (Cloud Platform Support)' via Google Cloud SQL discuss <google-cloud...@googlegroups.com> wrote:

Hello,

Do you have pg_stat_statements enabled? Seems like something it would be logged if you do. 

On Thursday, May 13, 2021 at 9:46:21 AM UTC-4 ho...@interactivelabs.co wrote:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/GDzR8uilu2c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-cloud-sql-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/da2919ec-6c99-46f7-a830-b8cd684fd17dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c203ef94-b5a0-4b31-b7b6-de432a6d23d2n%40googlegroups.com.

[google-cloud-sql-discuss] Re: help for repeatedly executed sql for cloud sql postgres

Is this a query that starts again and again for no reason at all? Is this a single query that runs forever and you cannot stop? In both cases, best place to ask would be on programmer forums such as stackoverflow. This discussion group is oriented more towards general opinions, trends, and issues of general nature touching the App Engine. For MySQL intricacies, you may be better served in dedicated forums such as stackoverflow, where experienced programmers are within reach and ready to help.

On Thursday, 13 May 2021 at 09:46:21 UTC-4 ho...@interactivelabs.co wrote:

hi all,

hi i got this sql run over and over in google cloud sql postgres:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happened ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/696cff7f-27ab-4842-8b5f-9b8763fd1668n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: why i got these queries on log every 1-2 seconds ?

no, i think because there are too many roles. 

Any suggestions for max roles for best practices?

Thanks

hong

On Thu, May 13, 2021 at 4:15 PM 'David (Cloud Platform Support)' via Google Cloud SQL discuss <google-cloud-sql-discuss@googlegroups.com> wrote:

Hello,

Do you have pg_stat_statements enabled? Seems like something it would be logged if you do. 

On Thursday, May 13, 2021 at 9:46:21 AM UTC-4 ho...@interactivelabs.co wrote:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to a topic in the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-cloud-sql-discuss/GDzR8uilu2c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/da2919ec-6c99-46f7-a830-b8cd684fd17dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAFOwj3GJwgjWU0Bo-SgHB7wsw%3DehbwJz3%3DsHw9_Goz1mKoDzKQ%40mail.gmail.com.

[google-cloud-sql-discuss] Re: why i got these queries on log every 1-2 seconds ?

Hello,

Do you have pg_stat_statements enabled? Seems like something it would be logged if you do. 

On Thursday, May 13, 2021 at 9:46:21 AM UTC-4 ho...@interactivelabs.co wrote:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/da2919ec-6c99-46f7-a830-b8cd684fd17dn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Unable to connect to CloudSQL Instance with MYSQL

I see that you're using ODBC 5.1.

It is recommended to use Connector/ODBC 8.0 according to link [1]

[1] https://dev.mysql.com/doc/connector-odbc/en/connector-odbc-versions.html

On Tuesday, May 11, 2021 at 9:32:52 AM UTC-4 Ronaldo Fortunato Lopes wrote:

I am unable to connect to a Google Cloud SQL instance. I get the error: [MySQL] [ODBC 5.1 Driver] Can't connect to MySQL server on 'XX.XXX.XX.XXX' (10060)

The IP from which I try to connect is a fixed ip and is registered in the list of allowed IPs in the Cloud SQL instance.

The machine that needs to connect is a Windows server with Windows Server 2016 operating system and can connect to other instances of Google Cloud SQL. We only have problems connecting to this instance. Another detail of the error is that the ping performed on my Windows instance returns total loss in relation to the Cloud SQL instance IP. Finally, I can access the databases using my local internet or another server.

I will be very grateful if someone can give me a tip to identify the problem

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4f7abc17-6c91-4da2-83f5-7413cc9dd640n%40googlegroups.com.

[google-cloud-sql-discuss] why i got these queries on log every 1-2 seconds ?

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happen ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/05f9c0b9-90a3-4e10-bd29-6b4820e85eccn%40googlegroups.com.

[google-cloud-sql-discuss] help for repeatedly executed sql for cloud sql postgres

hi all,

hi i got this sql run over and over in google cloud sql postgres:

WITH non_cloudsql_users AS (
SELECT * FROM pg_catalog.pg_user WHERE usename NOT LIKE 'cloudsql%'),
iam_roles AS (
SELECT pg_catalog.pg_has_role(usename, 'cloudsqliamuser', 'member') AS is_iam_user,
pg_catalog.pg_has_role(usename, 'cloudsqliamserviceaccount', 'member') AS is_svc_acct
FROM non_cloudsql_users)
SELECT pg_catalog.SUM(is_iam_user::int),
pg_catalog.SUM(is_svc_acct::int),
pg_catalog.SUM((NOT(is_iam_user OR is_svc_acct))::int),
current_timestamp
FROM iam_roles

Why this happened ? how to stop it ?

Thanks

hong

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/512aabb1-f52b-44ea-828a-033c80196338n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Replication of Large dataset 7 TB from RDS Aurora to Google Cloud SQL

the cloud migrations may help to migrate the data between both services.

On Friday, May 7, 2021 at 12:58:11 PM UTC-4 Mubbashar Husain wrote:

Hi All,

I want to create a master/slave replication between Amazon RDS Aurora and Google Cloud SQL. 

The size of the RDS instance is 7 TB and the initial dump will take hell of a time. 

what are the approches I can take so I can create Initial dump with minimal time and then import it into Google cloud SQL using migration service and start replication.

NOTICE: The information contained in this e-mail is considered Tinuiti intellectual property and is subject to confidentiality agreements in place between Tinuiti and its business partners. If you have received this email in error, please reply to the sender, and delete this message, copies, and attachments.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d587ce38-3bd3-4f5d-887e-d322d31ca24en%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: Export from CloudSQL to CSV using select * into outfile

Heya, I'm guessing you didn't mean to reply to an existing conversation and meant to start a new one, nevertheless there is documentation for that:
Instructions on how to set up alerts can be found here; but I'm guessing your concern would be more to do with what to monitor specifically, and that depends on the error report given within the logs, which usually includes the metric that is used by the monitoring/alerting systems in the Google Operations Suite (which contains Cloud Monitoring and Cloud Alerts)

Is there any specific instruction that you require further clarification on?

On Friday, May 7, 2021 at 10:28:11 PM UTC+2 mr.bg...@gmail.com wrote:

Hi all 

How to set alerts when cloud sql exports to gcp every 4 hours  if failed how to get notifications set for email if exporting not successfully done to gcs 

How to set alerts to solve the problem 

On Thu, 6 May 2021 at 12:40 PM, 'Rajesh Golla' via Google Cloud SQL discuss <google-cloud...@googlegroups.com> wrote:

Hi Elliot,

Option3 is work around and I want to execute that select command using stored procedure and need to schedule it.

Anyways I got a approach with cloud function and cloud scheduler.

Thanks for the help

On Wednesday, May 5, 2021 at 8:56:49 PM UTC+5:30 Elliott (Google Cloud Platform Support) wrote:

Hello,

I understand that you would like to restore data from Cloud SQL to a Cloud Storage bucket but you are receiving Error 1227[1]. I was able to find an explanation indicating that this operation needs a Super User permission[2] that a managed Cloud SQL does not offer at this moment.

I was able to find this document[3], which describes how to generate a CSV from Cloud SQL as an alternative.

Would this be a sufficient workaround for you? I know this must be frustrating not to have technology not behaving as you wish. Please let us know your thoughts.

I would believe that the community may offer further advice or other ways to accomplish what you would like to do in another way in addition to the one I suggested.

[1] https://forums.mysql.com/read.php?10,674692,674692

[2] https://www.virtual-dba.com/blog/mysql-error-1227-access-denied/#:~:text=The%20reason%20MySQL%20throws%20error,the%20user%20importing%20the%20data.

[3] https://cloud.google.com/sql/docs/mysql/import-export/exporting

On Wednesday, May 5, 2021 at 9:11:43 AM UTC-4 rgo...@ctepl.com wrote:

Hi,

I am trying to generate csv file from cloudsql and place in cloud storage using below query:

select * INTO OUTFILE 'Cloudstorage_path/test.csv'         FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'         LINES TERMINATED BY '\n' from Table limit 10

But I am getting below error:

Error Code: 1227. Access denied; you need (at least one of) the FILE privilege(s) for this operation

Please confirm whether we can use this approach to push data from cloudsql to cloud storage.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2aa60a85-477d-49fa-88ae-7c34fe0d9becn%40googlegroups.com.

--

Regards

Biswanath Giri

Mob:9560016514

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9c7f0465-4adf-4be4-86f2-e91d1f14bcb6n%40googlegroups.com.

[google-cloud-sql-discuss] Unable to connect to CloudSQL Instance with MYSQL

I am unable to connect to a Google Cloud SQL instance. I get the error: [MySQL] [ODBC 5.1 Driver] Can't connect to MySQL server on 'XX.XXX.XX.XXX' (10060)

The IP from which I try to connect is a fixed ip and is registered in the list of allowed IPs in the Cloud SQL instance.

The machine that needs to connect is a Windows server with Windows Server 2016 operating system and can connect to other instances of Google Cloud SQL. We only have problems connecting to this instance. Another detail of the error is that the ping performed on my Windows instance returns total loss in relation to the Cloud SQL instance IP. Finally, I can access the databases using my local internet or another server.

I will be very grateful if someone can give me a tip to identify the problem

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2143e3a4-bccf-40dc-a2ae-ea3ec7fd5fa1n%40googlegroups.com.

[google-cloud-sql-discuss] Re: I/O performance (mostly write?) intermittently goes severely low

Thank you, David. I'm not sure if the aborted connection is the cause or the result of the slow down or they are unrelated. though I do see isolated connection aborts. As for transaction processing, after starting to see this problem I modified the programs to do inserts in a transaction and the number of data fsync was reduced by half . I expected more reduction but your note reminds me that the reduction is less significant when dealing with bulk inserts.

I got the problem this morning again and I terminated running connections one by one. I think I've tried that before and it did not work but this time the i/o performance has recovered. Now I'm suspecting that having two connections writing massively into a single table triggers this problem, which strangely results in slow down of inserts into other tables as well. I'll serialize the jobs to ensure no simultaneous inserts and see if it makes any difference.  

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e4446cf2-5794-4264-bfdb-cd328a0568a0n%40googlegroups.com.

[google-cloud-sql-discuss] Re: I/O performance (mostly write?) intermittently goes severely low

Hello,

Here you can find some information on how to resolve this error with the aborted connections. Using connection pools should definitely help as well. I understand that this issue seems to only show up after the aborted connections error gets triggered. However, generally speaking, if you want to achieve higher performance on inserts, you can try disabling the autocommit flag. Please note that autocommit is enabled by default to prevent users from accidentally forgetting to commit a transaction. You could also try enabling the performance schema flag although I'm not sure it'll make much difference for 1k batches of insert.

On Monday, May 10, 2021 at 9:08:32 AM UTC-4 shinji...@gmail.com wrote:

Thanks for your input. But the insertions are performed in bulk (like 1000 rows) and light in index. They read data from local files. Above all, they execute fast enough until the described problem gets triggered.

2021年5月10日月曜日 4:47:54 UTC+9 od...@eversql.com:

As it is mostly write, it can be from several reasons like: Having unnecessary indexes, not using a bulk insert, uploading the data from remote servers,  etc. 
you can use EverSQL to get a performance analysis and find missing MySQL indexes.

On Saturday, May 8, 2021 at 4:11:01 PM UTC+3 shinji...@gmail.com wrote:

Hello. I'm looking at an instance (MySQL 2.7 2nd gen + 4vcpu + 15GB RAM + 100GB SDD + no failover) that has been running happily for about a year but have started to show intermittent performance issue  since late Apr.

Insertions that get executed at several tens of times per second suddenly start  to take tens or hundreds of seconds to get just a row inserted. When this happens I see lots of messages in the log like 

  textPayload: "2021-05-08T11:16:29.861999Z 369333 [Note] Aborted connection 369333 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error writing communication packets)"

  textPayload: "2021-05-08T11:14:10.448301Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 9784ms. The settings might not be optimal. (flushed=521 and evicted=0, during the time.)" 

A restart seems to resolve the issue momentary but the problem resurfaces after some hours or days

Is there a solution or a workaround for the issue?

TIA for your help.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/47b729a5-dd19-42dc-ac8a-190193727a7bn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Migrating from AWS RDS Postgres to Cloud SQL Postgres

Hello,

You would need to follow  these instructions when exporting the data to make sure the dump will be importable into Cloud SQL. You would also need to follow the importing guidelines as well to make sure you encounter no issues at the moment of importing the dump. 

On Friday, May 7, 2021 at 12:58:11 PM UTC-4 ikonrao wrote:

Hi,

We are planning to migrate a postgres instance in AWS RDS to GCP Cloud SQL.

For now, we are not planning to use GCP database migration service.

Just wanted to know if we just take dump from AWS RDS and restore it in GCP, will it work? Do we need to create users before importing the database in GCP.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/93342db1-af4c-49a5-af26-5aed74be4e4cn%40googlegroups.com.

[google-cloud-sql-discuss] Re: PostgreSQL: checkpointer process was terminated by signal 9: killed

Hi,

The logs do not provide much information about the cause of the issue. More details may be needed in order to further investigate. That being said, similar issues are often caused by OOM. Have you checked the memory usage and other resource consumption for any abnormality before the instance went down?

On Friday, May 7, 2021 at 12:58:11 PM UTC-4 konrad...@gmail.com wrote:

Hello,

in the last couple weeks my Postgres instance went down a few times. I found the following messages in the logs, but can't put my finger on what the root cause may be.

Is this some known issue? What else can I do to troubleshoot?

[16-1] db=,user= LOG: checkpointer process (PID 645146) was terminated by signal 9: Killed

[17-1] db=,user= LOG: terminating any other active server processes

[2-1] db=,user= WARNING: terminating connection because of crash of another server process

[3-1] db=,user= DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

[4-1] db=,user= HINT: In a moment you should be able to reconnect to the database and repeat your command.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/25c1ed11-f8d4-4e12-94fc-9a0d81a75dd8n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Postgresql 11.11 segmentation fault executing update

I confirm that after disabling query insights, there was no more segmentation faults.

Thanks.

On Sunday, May 9, 2021 at 2:56:06 PM UTC+1 waz0wski wrote:

Try disabling query insights, it seems to be causing problems

previous discussion

https://groups.google.com/g/google-cloud-sql-discuss/c/HBtClOKtM-o

google issues reporting similar

179716520, 183108383, 184065841, 184283279

On May 6, 2021, at 1:19 PM, Pedro Caetano <pedro....@polarising.com> wrote:

Hi,

our cloud sql postgresql (11.11) instance is restarting about once per day because of a segmentation fault executing an update table statement. The performance metrics look all ok.

This is a managed instance; how can we procced to diagnose and solve the problem?

Thanks.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d22061b4-1cea-4aa7-ada0-cd82a3f2a65dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1859f8de-b389-4a60-97f1-6ac856957ee5n%40googlegroups.com.

[google-cloud-sql-discuss] Re: I/O performance (mostly write?) intermittently goes severely low

Thanks for your input. But the insertions are performed in bulk (like 1000 rows) and light in index. They read data from local files. Above all, they execute fast enough until the described problem gets triggered.

2021年5月10日月曜日 4:47:54 UTC+9 od...@eversql.com:

As it is mostly write, it can be from several reasons like: Having unnecessary indexes, not using a bulk insert, uploading the data from remote servers,  etc. 
you can use EverSQL to get a performance analysis and find missing MySQL indexes.

On Saturday, May 8, 2021 at 4:11:01 PM UTC+3 shinji...@gmail.com wrote:

Hello. I'm looking at an instance (MySQL 2.7 2nd gen + 4vcpu + 15GB RAM + 100GB SDD + no failover) that has been running happily for about a year but have started to show intermittent performance issue  since late Apr.

Insertions that get executed at several tens of times per second suddenly start  to take tens or hundreds of seconds to get just a row inserted. When this happens I see lots of messages in the log like 

  textPayload: "2021-05-08T11:16:29.861999Z 369333 [Note] Aborted connection 369333 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error writing communication packets)"

  textPayload: "2021-05-08T11:14:10.448301Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 9784ms. The settings might not be optimal. (flushed=521 and evicted=0, during the time.)" 

A restart seems to resolve the issue momentary but the problem resurfaces after some hours or days

Is there a solution or a workaround for the issue?

TIA for your help.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c1e487aa-94fd-4e03-8c2c-0d8b147ff6aan%40googlegroups.com.

[google-cloud-sql-discuss] Re: I/O performance (mostly write?) intermittently goes severely low

As it is mostly write, it can be from several reasons like: Having unnecessary indexes, not using a bulk insert, uploading the data from remote servers,  etc. 
you can use EverSQL to get a performance analysis and find missing MySQL indexes.

On Saturday, May 8, 2021 at 4:11:01 PM UTC+3 shinji...@gmail.com wrote:

Hello. I'm looking at an instance (MySQL 2.7 2nd gen + 4vcpu + 15GB RAM + 100GB SDD + no failover) that has been running happily for about a year but have started to show intermittent performance issue  since late Apr.

Insertions that get executed at several tens of times per second suddenly start  to take tens or hundreds of seconds to get just a row inserted. When this happens I see lots of messages in the log like 

  textPayload: "2021-05-08T11:16:29.861999Z 369333 [Note] Aborted connection 369333 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error writing communication packets)"

  textPayload: "2021-05-08T11:14:10.448301Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 9784ms. The settings might not be optimal. (flushed=521 and evicted=0, during the time.)" 

A restart seems to resolve the issue momentary but the problem resurfaces after some hours or days

Is there a solution or a workaround for the issue?

TIA for your help.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/66a40b2f-7dcf-4d94-94e1-fb60f1f2813cn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Postgresql 11.11 segmentation fault executing update

Try disabling query insights, it seems to be causing problems

previous discussion

https://groups.google.com/g/google-cloud-sql-discuss/c/HBtClOKtM-o

google issues reporting similar

179716520, 183108383, 184065841, 184283279

On May 6, 2021, at 1:19 PM, Pedro Caetano <pedro.caetano@polarising.com> wrote:

Hi,

our cloud sql postgresql (11.11) instance is restarting about once per day because of a segmentation fault executing an update table statement. The performance metrics look all ok.

This is a managed instance; how can we procced to diagnose and solve the problem?

Thanks.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d22061b4-1cea-4aa7-ada0-cd82a3f2a65dn%40googlegroups.com.