[google-cloud-sql-discuss] Re: Is anyone else noticing cloud sql for postgres 12 not working

Right now, there is an incident affecting europe-west2 Cloud SQL instances. The Software team is aware of it and is currently working on it to solve it ASAP.

You can get updates of this incident at:

• https://status.cloud.google.com/regional/europe

• https://status.cloud.google.com/incidents/fmEL9i2fArADKawkZAa2

On Thursday, July 14, 2022 at 5:06:19 PM UTC-5 jki...@northwell.edu wrote:

I see errors in the logs that appear to be a misplaced comma in the query used by cloudsqladmin

the query

reads SELECT wait_event current_timestamp, count(*) as num_queries, FROM pg_catalog.pg_stat_activity WHERE wait_event_type='IO' GROUP BY wait_event_type, wait_event

its should read

SELECT wait_event, current_timestamp, count(*) as num_queries FROM pg_catalog.pg_stat_activity WHERE wait_event_type='IO' GROUP BY wait_event_type, wait_event

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4ab14943-c810-4a5c-b48f-6245d8eaa024n%40googlegroups.com.

[google-cloud-sql-discuss] Cloud SQL Mysql - Failover operation in progress

Hi,

My Cloud SQL - MySQL 5.7.37 Highly Available instance is stuck in: "Failover operation in progress. This may take a few minutes. While this operation is running, you may continue to view information about the instance."

It is a fairly small database and it has been stuck like this for 5 hours and the failover is not available so all DB queries are failing. Since it is in an updating mode no command are available. No errors are found in the error log and the operations log only has this stuck operation and completed backups.

I hope anyone can help me as there is no way to get support from Google other than pay for it (which I think is pretty bad since this is a fully managed service they are selling).

Best,

Carl-Fredrik

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e9c5cb30-61b1-42b5-afdd-84693b3d9c80n%40googlegroups.com.

[google-cloud-sql-discuss] Is anyone else noticing cloud sql for postgres 12 not working

I see errors in the logs that appear to be a misplaced comma in the query used by cloudsqladmin

the query

reads SELECT wait_event current_timestamp, count(*) as num_queries, FROM pg_catalog.pg_stat_activity WHERE wait_event_type='IO' GROUP BY wait_event_type, wait_event

its should read

SELECT wait_event, current_timestamp, count(*) as num_queries FROM pg_catalog.pg_stat_activity WHERE wait_event_type='IO' GROUP BY wait_event_type, wait_event

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/ba00e577-bb10-49e7-8b0f-997890fa23b4n%40googlegroups.com.

[google-cloud-sql-discuss] Important Google Group update: Moving to the Google Cloud Community August 1st, 2022

Hi there, 

My name is Willie and I am a Community Manager here at Google. You're receiving this message because you've previously engaged with one of our Google Cloud Google Groups. This message is to inform you of an upcoming change to Google Cloud support via Google Groups, which will help streamline and accelerate the rate at which you receive answers to your questions.

Beginning on August 1st, 2022, you can no longer submit questions or receive answers in the Google Group in which you are receiving this message. You'll still be able to read historic messages within the Google Group, but you can no longer submit new messages or replies. 

All future questions and discussions around Google Cloud SQL will take place in the Google Cloud Community via the corresponding forum linked below.

Cloud SQL

Don't have a Google Cloud Community account yet? It's easy and free to sign up by following the steps here. 

Then you can ask questions, find answers, subscribe to receive the latest news, and more. Have questions about creating an account or publishing a post? You can reach out to the Google Cloud Community team at gc-customer-community@google.com 

Thank you! We look forward to seeing you in the Community. 

Willie, on behalf of The Google Cloud Community team

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/a39f1e9b-7b38-4b4e-9471-eab9d23a2261n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Shareing CloudSQL

Two ways:

1) Use Public IP. Safest way to do that is use Public IP but don't authorize any networks, and use the Cloud SQL Auth Proxy to connect to it from the other project (you'll need an IAM service account key from the Cloud SQL project to do this).

2) Set up VPC Sharing at the organization level and host the Cloud SQL's project with the project you want to access it from. This treats the VPCs as essentially the same between the two projects.

Gabe Weiss (he/him) |

Developer Advocate |

gweiss@google.com

On Fri, Jul 8, 2022 at 9:18 AM Kenika Pandey <pa.kenika_st@tni.ac.th> wrote:

Is it possible to share the ClouldSQL provisioned with one project with the other? If yes, could you please guide me on how to achieve it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/36e626f0-d645-4511-b9d4-a41b125f8bbdn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAM9LKEq%3DN%2B2xhmWGK8uJ48%2B8RVbNBvuN6n9PhbQUJ0Ug%3D-9RYw%40mail.gmail.com.

Re: [google-cloud-sql-discuss] Is it possible to connect through vpc peering?

Unfortunately no, not easily. Cloud SQL is a service deployed from Private Services (a Google-managed VPC) and when you instantiate an instance of it, that Cloud SQL is peered to your project's VPC where you deploy it. That means there's already one degree of VPC peering happening. VPC peering is not transitive sadly. So e.g. you have VPC A peered to VPC B, and VPC B is peered to VPC C. VPCs A and C cannot see each other.

Ways around this are to use VPC Sharing instead of peering, and use hosted projects to house your data (shared VPCs can see each other okay), or use Public IP on your Cloud SQL instance (but don't use authorized networks which becomes more of a security issue) and use the SQL Proxy like you are. You can also set up a bastion instance in the VPC with Cloud SQL. Use a VM with a public IP that you lock down to only be able to communicate with the application VPC, and use something like a Socks5 proxy on the bastion instance.

Gabe Weiss (he/him) |

Developer Advocate |

gweiss@google.com

On Fri, Jul 8, 2022 at 9:18 AM 변재천 <byunddong88@gmail.com> wrote:

I made two VPCs and connected them with peering.
Can I access Cloud SQL in B-VPC from VM in A-VPC?
SQL Proxy and Public IP are not being used.
Cloud SQL is using private IP.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/871246a5-5c07-413d-8a17-cbba2e0fb0a8n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAM9LKEr%3DtB2Wq%3DXEY94EGzXgDSeZ%2ByOmSgEkKCrDt-vMZ7paVQ%40mail.gmail.com.

[google-cloud-sql-discuss] Shareing CloudSQL

Is it possible to share the ClouldSQL provisioned with one project with the other? If yes, could you please guide me on how to achieve it?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/36e626f0-d645-4511-b9d4-a41b125f8bbdn%40googlegroups.com.

[google-cloud-sql-discuss] Is it possible to connect through vpc peering?

I made two VPCs and connected them with peering.
Can I access Cloud SQL in B-VPC from VM in A-VPC?
SQL Proxy and Public IP are not being used.
Cloud SQL is using private IP.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/871246a5-5c07-413d-8a17-cbba2e0fb0a8n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Data is garbled when importing CSV to SQL instance

SQL Server and Cloud SQL do not support CSV files. As already indicated, SQL Server presently supports importing databases using BAK and SQL files in Cloud SQL.

The Cloud SQL versions for MySQL and PostgreSQL somehow support it.

One of the following options is possible:

Alternatively, switch to MySQL or PostgreSQL as the database engine (where CSV files are supported).

You can utilize your CSV file to create a SQL file that you may use to import into Cloud SQL, SQL Server, if the data on it comes from an on-premises SQL Server DB table.

On Monday, July 4, 2022 at 3:04:10 PM UTC-5 airflo...@gmail.com wrote:

Hello,

    I have been trying to import a CSV file to GCP SQL Instance, PFA a screenshot of the part of the SQL File. Import is showing as successful, I tried both using the UI and gcp sql import command but same result for both. PFA SELECT * command from the table.

    Please let me know what I am doing wrong, and if you need any more information.

Thanks and Regards,

Shashanka

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/a2976065-3142-4c08-9e67-b5b92d0035a6n%40googlegroups.com.

[google-cloud-sql-discuss] Data is garbled when importing CSV to SQL instance

Hello,

    I have been trying to import a CSV file to GCP SQL Instance, PFA a screenshot of the part of the SQL File. Import is showing as successful, I tried both using the UI and gcp sql import command but same result for both. PFA SELECT * command from the table.

    Please let me know what I am doing wrong, and if you need any more information.

Thanks and Regards,

Shashanka

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/5db32f34-8c1a-49d7-b14d-0cde4c374ad9n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Why Cloud Postgres SQL instance updates randomly by itself?

These Update operations would be logged when the configurations of the Cloud SQL have been changed, for example the modifying of the database flags. 

On Wednesday, June 29, 2022 at 4:41:20 PM UTC-5 mappe...@gmail.com wrote:

Hi, 

I check the operation log for our Postgres SQL instance and found out it shows many times - "Update finished", every time it takes about a minute or two which causes a connection to drop if someone using it from the application.

I have no idea what it is, it is not part of the maintenance schedule we setup up.

Does anyone know what is that cause? how to avoid that?

Please see the attached screenshot to see how it looks in the SQL log entry.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2f035f78-3ed5-4e0a-a8b7-008b25ef9dd4n%40googlegroups.com.

[google-cloud-sql-discuss] Re: How to change root@localhost password or delete user

Unfortunately, it is not possible to delete 'root@localhost' or modify it, according to [1].

[1] https://cloud.google.com/sql/docs/mysql/users#system_users

On Tuesday, June 28, 2022 at 3:57:53 PM UTC-5 snee...@gmail.com wrote:

We have a mysql v5.7 instance. Which got compromised (Ransomware), Attacker delete databases and left one saying where to send money.

Anyway, The logins we found were using admin or root which were having default setting.

We deleted root and admin, including all older users, and created new users with different names. When I list users from GCP Console I seen only newly created users, while listing users through mysql shell I see an addition user (system user): root@localhost.

All GCP created users are regular user. So deleting or changing root@localhost fails.

Is it secure? Is there a way to change or remove it? Does GCP Console provide a way to create "system user"?

I have another instance which does not have root user!

  

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/6def2b4b-2525-4168-b491-e4c5db630398n%40googlegroups.com.

[google-cloud-sql-discuss] Why Cloud Postgres SQL instance updates randomly by itself?

Hi, 

I check the operation log for our Postgres SQL instance and found out it shows many times - "Update finished", every time it takes about a minute or two which causes a connection to drop if someone using it from the application.

I have no idea what it is, it is not part of the maintenance schedule we setup up.

Does anyone know what is that cause? how to avoid that?

Please see the attached screenshot to see how it looks in the SQL log entry.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/30596767-60a7-4fda-bc0f-88d2e02d9a95n%40googlegroups.com.

[google-cloud-sql-discuss] How to change root@localhost password or delete user

We have a mysql v5.7 instance. Which got compromised (Ransomware), Attacker delete databases and left one saying where to send money.

Anyway, The logins we found were using admin or root which were having default setting.

We deleted root and admin, including all older users, and created new users with different names. When I list users from GCP Console I seen only newly created users, while listing users through mysql shell I see an addition user (system user): root@localhost.

All GCP created users are regular user. So deleting or changing root@localhost fails.

Is it secure? Is there a way to change or remove it? Does GCP Console provide a way to create "system user"?

I have another instance which does not have root user!

  

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2b136d21-8083-484b-ad16-666fffed406cn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Re: Can't connect to instance because "Policy checks are unavailable"

The Cloud SQL team is working on fixing this right now.

I'll share updates here as they're available: https://github.com/GoogleCloudPlatform/cloudsql-proxy/issues/1248.

On Fri, Jun 24, 2022 at 12:10 PM 'perezsanchez' via Google Cloud SQL discuss <google-cloud-sql-discuss@googlegroups.com> wrote:

Reviewing other cases with the same issue, the suggestion so far is to use Cloudflare's WARP to get around the issue. I suspect a DNS blocking issue of some sort.

Basically, CloudFlare's VPN tunnel. According to what I most recently read in the following site, it's a Wireguard implementation:

https://developers.cloudflare.com/warp-client/warp-modes/#1111-with-warp

You may access the github repository in case there is another update of the issue related to.

On Thursday, June 23, 2022 at 1:30:49 PM UTC-5 haide...@tajir-app.com wrote:

Here's what `gcloud sql connect` says:

ERROR: (gcloud.sql.connect) UNAVAILABLE: Policy checks are unavailable.

Here's what cloud_sql_proxy says:

2022/06/23 17:22:11 errors parsing config:

    googleapi: Error 503: Policy checks are unavailable., backendError

What's wrong here?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/38c76976-5a52-4d3d-b18b-1aa617ce7caen%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CA%2BND%3DxcY0UygMUWRc1SQ3_mm8bNhB_w5OY5Z%3D6kaxA6aohOCYQ%40mail.gmail.com.

[google-cloud-sql-discuss] Re: Can't connect to instance because "Policy checks are unavailable"

Reviewing other cases with the same issue, the suggestion so far is to use Cloudflare's WARP to get around the issue. I suspect a DNS blocking issue of some sort.

Basically, CloudFlare's VPN tunnel. According to what I most recently read in the following site, it's a Wireguard implementation:

https://developers.cloudflare.com/warp-client/warp-modes/#1111-with-warp

You may access the github repository in case there is another update of the issue related to.

On Thursday, June 23, 2022 at 1:30:49 PM UTC-5 haide...@tajir-app.com wrote:

Here's what `gcloud sql connect` says:

ERROR: (gcloud.sql.connect) UNAVAILABLE: Policy checks are unavailable.

Here's what cloud_sql_proxy says:

2022/06/23 17:22:11 errors parsing config:

    googleapi: Error 503: Policy checks are unavailable., backendError

What's wrong here?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/38c76976-5a52-4d3d-b18b-1aa617ce7caen%40googlegroups.com.

Re: [google-cloud-sql-discuss] 403 Unauthorized issue when connecting to Cloud SQL using IAM

You also need the cloudsql.instances.login role to use IAM authentication.

https://cloud.google.com/sql/docs/mysql/authentication

Try that and see if it works?

Gabe Weiss (he/him) |

Developer Advocate |

gweiss@google.com

On Thu, Jun 23, 2022 at 11:30 AM Tony Ouyang <tony@terratrue.com> wrote:

Hi, I'm trying to use cloud IAM connection to Cloud SQL. I set up a service account and grant this account `Cloud SQL client` and `Cloud SQL instance user` role and configured it correctly in my Java code. It works fine in my local sample program.

However, when I write this as dataflow pipeline job and use apache Beam library while left all the configuration and settings the same. I encountered 403 notAuthorized issue when running in cloud. The dataflow runs perfectly fine in my local, just failing on cloud:

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
GET https://sqladmin.googleapis.com/v1/projects/sim-main-backend/instances/us-west1~sim-main-backend-mysql/connectSettings
{
  "code": 403,
  "errors": [
    {
      "domain": "global",
      "message": "The client is not authorized to make this request.",
      "reason": "notAuthorized"
    }
  ],
  "message": "The client is not authorized to make this request."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:439)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:525)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:466)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:576)
at com.google.cloud.sql.core.CloudSqlInstance.fetchMetadata(CloudSqlInstance.java:462)
... 9 more

Jun 22, 2022 3:31:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-06-22T22:31:34.777Z: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Could not create connection to database server.
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:596)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:582)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processSQL(GcpStorageSchemaClassifier.java:127)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processElement(GcpStorageSchemaClassifier.java:94)
Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:1001)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:448)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processSQL(GcpStorageSchemaClassifier.java:127)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processElement(GcpStorageSchemaClassifier.java:94)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: [sim-main-backend:us-west1:sim-main-backend-mysql] The Cloud SQL Instance does not exist or your account is not authorized to access it. Please verify the instance connection name and check the IAM permissions for project "sim-main-backend"
at com.google.cloud.sql.core.CloudSqlInstance.addExceptionContext(CloudSqlInstance.java:618)
at com.google.cloud.sql.core.CloudSqlInstance.fetchEphemeralCertificate(CloudSqlInstance.java:541)
at com.google.cloud.sql.core.CloudSqlInstance.lambda$performRefresh$0(CloudSqlInstance.java:332)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
... 3 more

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9fd06411-ef6f-428f-8d58-b19b7edcaacbn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAM9LKEpQNupFDcfmQG7hoXONu46BmKCWmqQ_mKwrJ_5LHYz8gw%40mail.gmail.com.

[google-cloud-sql-discuss] Can't connect to instance because "Policy checks are unavailable"

Here's what `gcloud sql connect` says:

ERROR: (gcloud.sql.connect) UNAVAILABLE: Policy checks are unavailable.

Here's what cloud_sql_proxy says:

2022/06/23 17:22:11 errors parsing config:

    googleapi: Error 503: Policy checks are unavailable., backendError

What's wrong here?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/a2fd53ac-a916-40c2-a5a4-a86e1c4a1e2bn%40googlegroups.com.

[google-cloud-sql-discuss] 403 Unauthorized issue when connecting to Cloud SQL using IAM

Hi, I'm trying to use cloud IAM connection to Cloud SQL. I set up a service account and grant this account `Cloud SQL client` and `Cloud SQL instance user` role and configured it correctly in my Java code. It works fine in my local sample program.

However, when I write this as dataflow pipeline job and use apache Beam library while left all the configuration and settings the same. I encountered 403 notAuthorized issue when running in cloud. The dataflow runs perfectly fine in my local, just failing on cloud:

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
GET https://sqladmin.googleapis.com/v1/projects/sim-main-backend/instances/us-west1~sim-main-backend-mysql/connectSettings
{
  "code": 403,
  "errors": [
    {
      "domain": "global",
      "message": "The client is not authorized to make this request.",
      "reason": "notAuthorized"
    }
  ],
  "message": "The client is not authorized to make this request."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:439)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:525)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:466)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:576)
at com.google.cloud.sql.core.CloudSqlInstance.fetchMetadata(CloudSqlInstance.java:462)
... 9 more

Jun 22, 2022 3:31:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-06-22T22:31:34.777Z: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Could not create connection to database server.
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:596)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:582)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processSQL(GcpStorageSchemaClassifier.java:127)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processElement(GcpStorageSchemaClassifier.java:94)
Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:1001)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:448)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processSQL(GcpStorageSchemaClassifier.java:127)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth.processElement(GcpStorageSchemaClassifier.java:94)
at com.terratrue.dataflow.gcp.pipelines.GcpStorageSchemaClassifier$CloudSqlIamAuth$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: [sim-main-backend:us-west1:sim-main-backend-mysql] The Cloud SQL Instance does not exist or your account is not authorized to access it. Please verify the instance connection name and check the IAM permissions for project "sim-main-backend"
at com.google.cloud.sql.core.CloudSqlInstance.addExceptionContext(CloudSqlInstance.java:618)
at com.google.cloud.sql.core.CloudSqlInstance.fetchEphemeralCertificate(CloudSqlInstance.java:541)
at com.google.cloud.sql.core.CloudSqlInstance.lambda$performRefresh$0(CloudSqlInstance.java:332)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
... 3 more

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9fd06411-ef6f-428f-8d58-b19b7edcaacbn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Differentiate between 2 parameters Total memory usage and Memory usage?

The Memory usage metric is measured at the VM level and does not include the OS cache. It's equivalent to "MemTotal - MemAvailable" from the VM's /proc/meminfo.

The Total memory usage metric is measured at the database container level. It includes memory used by the database container itself, and the OS cache.

The metric names might suggest that Total memory usage is always higher or equal to Memory usage. This is not true because those metrics are measured at different levels. (VM versus Docker). In other words, it's not Memory usage and Total memory usage for the same thing, but rather Memory usage for one thing and Total memory usage for another thing.

For example: 

• Memory usage can be higher than Total memory usage because it's measured at the entire VM level, so it includes memory usage by components other than the database container. If the container and cache usage is low (e.g. because the database is small, idle, and doesn't use much OS cache), VM usage can potentially be higher than container usage. There are also differences in how memory allocation is measured in Docker versus the OS, which can contribute to the discrepancy.

• Total memory usage can be, and often is, higher than Memory usage because it includes OS cache. Busy, warm databases can use large amounts of OS cache, which inflates Total memory usage.

On Tuesday, June 21, 2022 at 9:52:33 AM UTC-5 eucli...@gmail.com wrote:

I am using CloudSQL with MySQL 5.7

This is my configuration

• vCPUs: 1
• Memory: 614.4 MB
• SSD storage: 10 GB

Lately my application has been running pretty slow, and I'm guessing it's due to a bottleneck in MySQL.

But when I look at the following 2 parameters, I don't understand why Total memory usage is smaller than Memory usage.??? (Attachment)

 

As references, we known:

• Total memory usage: Total RAM usage in bytes including buffer cache. Sampled every 60 seconds. After sampling, data is not visible for up to 210 seconds.
• Memory usage: RAM usage in bytes. Sampled every 60 seconds. After sampling, data is not visible for up to 210 seconds.

Can anyone help me explain in more detail about these 2 metrics?

Thanks very much

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8a6a0867-94e0-41f8-9163-58103292b384n%40googlegroups.com.

[google-cloud-sql-discuss] Differentiate between 2 parameters Total memory usage and Memory usage?

I am using CloudSQL with MySQL 5.7

This is my configuration

• vCPUs: 1
• Memory: 614.4 MB
• SSD storage: 10 GB

Lately my application has been running pretty slow, and I'm guessing it's due to a bottleneck in MySQL.

But when I look at the following 2 parameters, I don't understand why Total memory usage is smaller than Memory usage.??? (Attachment)

 

As references, we known:

• Total memory usage: Total RAM usage in bytes including buffer cache. Sampled every 60 seconds. After sampling, data is not visible for up to 210 seconds.
• Memory usage: RAM usage in bytes. Sampled every 60 seconds. After sampling, data is not visible for up to 210 seconds.

Can anyone help me explain in more detail about these 2 metrics?

Thanks very much

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f315d5ff-6016-42e2-906b-15adffd0f4bbn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Performance issues moving data between Cloud SQL instances (PostgreSQL)

In the Cloud SQL for PostgreSQL documentation are some articles related to importing/exporting data.

Regarding the Best practices for importing and exporting data, here are some of them:

• Don't use Cloud Storage Requester Pays buckets.

• Use serverless export to offload the export operation from the primary instance.

• Use the correct flags when you create a SQL dump file.

• Compress data to reduce cost.

• Reduce long-running import and export processes.

• Verify the imported database.

Also known issues are referred to in this documentation.

As shown in this Stack Overflow answer, you can use pg_dump to extract the table. This page describes exporting and importing data into Cloud SQL instances using pg_dump and pg_restore.

The postgres_fdw is an extension allows tables from other ("foreign") PostgreSQL databases to be exposed as "foreign" tables in the current database. Currently, this extension works for two Cloud SQL private IP instances within the same VPC network, or for cross databases within the same instance.

To connect to databases within the same instance, you cannot set host to localhost or to 127.0.0.1. Instead, you must use the IP address shown for your instance in the Google Cloud console.

Additionally, you cannot choose, in the Google Cloud console, the Allow only SSL connections button for an instance where foreign data is stored. Only cloudsqlsuperuser can be the owner of a postgres_fdw foreign data wrapper.

On Thursday, June 16, 2022 at 4:26:43 PM UTC-5 aba...@gmail.com wrote:

Hi. I'm having performance issues trying to move a large amount of data between two Cloud SQL instances using Postgre.

Are there any guidelines or best practices to move data between two Cloud SQL instances? Not the entire database, just some tables, directly between instances.

I'm using the Foreign Data Wrapper extention in Postgre to archive the migration.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/3e0da8a2-4e5b-49fb-9741-dfe0f9b33e80n%40googlegroups.com.

[google-cloud-sql-discuss] Performance issues moving data between Cloud SQL instances (PostgreSQL)

Hi. I'm having performance issues trying to move a large amount of data between two Cloud SQL instances using Postgre.

Are there any guidelines or best practices to move data between two Cloud SQL instances? Not the entire database, just some tables, directly between instances.

I'm using the Foreign Data Wrapper extention in Postgre to archive the migration.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9b337e10-d547-4272-920c-92d8f03bc6d1n%40googlegroups.com.

[google-cloud-sql-discuss] Re: CloudIAP with CloudSQL/Redis and Private Connect

As said before, 

    "there is no ETA's on Feature requests, however, keep track of this issue for updates" 

To keep track on this feature request, you can follow updates at the following link: https://issuetracker.google.com/159364421

Regarding your question to configure Cloud IAP with Cloud SQL, the following tutorial could help you.

On Monday, May 30, 2022 at 11:39:19 AM UTC-5 hs...@ischoolconnect.com wrote:

is there any update by IAP team? can we configure IAP with Cloud SQL with private IP.

On Thursday, April 30, 2020 at 7:23:32 PM UTC+5:30 Olu wrote:

As indicated on this IAP Overview documentation[1], IAP may be used with applications running on App Engine standard environment, App Engine flexible environment, Compute Engine, and GKE. The Cloud IAP cannot be configured with CloudSQL or Redis at the moment. A feature request was submitted with the IAP team for this implementation but there is no ETA for such implementations at this time. 

[1]https://cloud.google.com/iap/docs

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1e71b86c-6ecf-45ae-8f12-73e72d253727n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Import of data to GCP Database is failing with error

You will need to provide more information. If you could provide more details such as the process you are following or the documentation or reference used, it would be very helpful for the community to provide a better answer.

I will leave this other Groups Post that can be used as a good example on how to post a more informational question and could have some hints for your issue. 

I would also recommend trying to follow the official documentation for Export and import using BAK files, including the Best practices for importing and exporting data

On Monday, May 30, 2022 at 11:39:19 AM UTC-5 pri...@signicat.com wrote:

Getting this error  when trying to Import data to GCP Database is failing with error - 

Multiple databases detected in BAK file. Only importing a single database is supported

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8113c1d4-36dd-4c3f-955f-dfb72bf47541n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Postgres storage 16.5TB with DB size under 1TB

Hi, Miroslav, 

Can you please share  the output of the queries below?

1) select pg_size_pretty(sum(size)) as "Total WAL disk usage" from pg_ls_waldir();

2) select pg_size_pretty(sum(size)) as "Total WAL disk usage" from pg_ls_waldir() where name not like '%.backup';

3) select * from pg_ls_waldir() order by modification Asc;

On Monday, May 30, 2022 at 11:39:19 AM UTC-5 miro...@carted.com wrote:

Hi,

Got a disk usage on Postgres indicating we are utilising 16.5 TB of data, however running various table size queries (https://wiki.postgresql.org/wiki/Disk_Usage) does not show any significant disk usage amounting to that amount? 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/6d4d7266-f2eb-4f27-9600-fb8b462d718cn%40googlegroups.com.

[google-cloud-sql-discuss] Re: wpdatatable and Google Cloud SQL connection

Hi Meryl,

Can you provide us with more information on how you are connecting (an example or code snippet might help)?

Are you using public IP? Are you using the Cloud SQL proxy? Are you using SSL/TLS certificates?

Does the IP address of your new host have an IPV4 address? Or is is possibly coming from an IPV6 ip? You should be able to verify the public ip of the host with `curl icanhaszip.com`. 

On Tuesday, May 31, 2022 at 12:14:47 PM UTC-6 me...@theteam.net.au wrote:

Hi, 

I have using Google Cloud SQL to connect to our wpdatatables. Over the weekend, our website had to change its host provider and now the SQL connection is being refused despite us whitelisting the new IP address + also having the 0.0.0.0/0 IP address in the white list. 

The issue that appears is basically this: 

wpDataTables could not connect to mysql server. mysql said: There was a problem with your SQL connection - Connection refused

I have entered the correct credentials, etc. Not sure what I'm doing wrong. 

Can anybody help?

Thanks

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/3cae3b91-d32e-49a7-9342-1472579e95efn%40googlegroups.com.

Re: [google-cloud-sql-discuss] Postgres storage 16.5TB with DB size under 1TB

Do you have point in time recovery on? That will do it...

Peter 

On Mon, May 30, 2022, 10:39 AM Miroslav Kosteckij <miroslav@carted.com> wrote:

Hi,

Got a disk usage on Postgres indicating we are utilising 16.5 TB of data, however running various table size queries (https://wiki.postgresql.org/wiki/Disk_Usage) does not show any significant disk usage amounting to that amount? 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/3a241566-002d-4352-9ff7-84e0cd73e28cn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAC%2B%3DFfb3vWRTLw3wPUvm3Vxck%2BvW%3DvDXO_Q5U63AtareY3D7Qw%40mail.gmail.com.

[google-cloud-sql-discuss] Re: Share Cloud SQL DB between 2 projects with app engine

Ok

 solution found. 

Just need to add service account of ProjectB to IAM of ProjectA

Le lundi 30 mai 2022 à 18:39:19 UTC+2, - - a écrit :

Hi, 

I have a postgres Cloud SQL DB instance in my project A, it works perfectly with my django project on App Engine of project A. 

Now I have developed a project B and I would like to share the DB instance with project B. 

I put the same host as project A in the settings file of project B like:

/cloudsql/project-285409:europe-west4:project-A-DB

but when I deploy project B on app engine, django keeps having connection error 

could not connect to server: Connection refused   

Is the server running locally and accepting connections on Unix domain socket "/cloudsql/project-285409:europe-west4:project-A-DB.s.PGSQL.5432"?

I have enabled Cloud SQL API

Did I miss some settings or it is not possible to share DB between 2 projects?

Thank you

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1cdf81e1-6635-4378-a342-95bb8b4f27d0n%40googlegroups.com.

[google-cloud-sql-discuss] wpdatatable and Google Cloud SQL connection

Hi, 

I have using Google Cloud SQL to connect to our wpdatatables. Over the weekend, our website had to change its host provider and now the SQL connection is being refused despite us whitelisting the new IP address + also having the 0.0.0.0/0 IP address in the white list. 

The issue that appears is basically this: 

wpDataTables could not connect to mysql server. mysql said: There was a problem with your SQL connection - Connection refused

I have entered the correct credentials, etc. Not sure what I'm doing wrong. 

Can anybody help?

Thanks

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4481d413-a2a7-4c7b-8e06-67ad70d94cban%40googlegroups.com.

[google-cloud-sql-discuss] Import of data to GCP Database is failing with error

Getting this error  when trying to Import data to GCP Database is failing with error - 

Multiple databases detected in BAK file. Only importing a single database is supported

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/fddf1f07-bb23-45d3-a9e7-717a4a320217n%40googlegroups.com.

[google-cloud-sql-discuss] Postgres storage 16.5TB with DB size under 1TB

Hi,

Got a disk usage on Postgres indicating we are utilising 16.5 TB of data, however running various table size queries (https://wiki.postgresql.org/wiki/Disk_Usage) does not show any significant disk usage amounting to that amount? 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/3a241566-002d-4352-9ff7-84e0cd73e28cn%40googlegroups.com.

[google-cloud-sql-discuss] Share Cloud SQL DB between 2 projects with app engine

Hi, 

I have a postgres Cloud SQL DB instance in my project A, it works perfectly with my django project on App Engine of project A. 

Now I have developed a project B and I would like to share the DB instance with project B. 

I put the same host as project A in the settings file of project B like:

/cloudsql/project-285409:europe-west4:project-A-DB

but when I deploy project B on app engine, django keeps having connection error 

could not connect to server: Connection refused   

Is the server running locally and accepting connections on Unix domain socket "/cloudsql/project-285409:europe-west4:project-A-DB.s.PGSQL.5432"?

I have enabled Cloud SQL API

Did I miss some settings or it is not possible to share DB between 2 projects?

Thank you

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/676ac7d5-6004-4f67-96bf-3cff8a419fc8n%40googlegroups.com.

[google-cloud-sql-discuss] Re: CloudIAP with CloudSQL/Redis and Private Connect

is there any update by IAP team? can we configure IAP with Cloud SQL with private IP.

On Thursday, April 30, 2020 at 7:23:32 PM UTC+5:30 Olu wrote:

As indicated on this IAP Overview documentation[1], IAP may be used with applications running on App Engine standard environment, App Engine flexible environment, Compute Engine, and GKE. The Cloud IAP cannot be configured with CloudSQL or Redis at the moment. A feature request was submitted with the IAP team for this implementation but there is no ETA for such implementations at this time. 

[1]https://cloud.google.com/iap/docs

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d7fb4986-dd87-43f2-8548-61451b890b4dn%40googlegroups.com.

[google-cloud-sql-discuss] Re: No fuss strategy to migrate a CloudSQL PostgreSQL database to another GCP account, using replication and zero loss and downtime

Hi, alexhguerra,

Users sometimes want to migrate their (normal) relational database with "zero" downtime. While downtime can be reduced, migration cannot be done without any impact on applications (that is, zero downtime). Replication causes replication lag.

The instant the decision is made to "migrate" all applications from one replica to another, applications (and therefore customers) have to wait (that is, downtime) at least as long as the "replication lag" before using the new database. In practice, the downtime is a few orders of magnitude higher (minutes to hours) because:

* Database queries can take multiple seconds to complete and in flight queries must be completed or aborted at the time of migration.

* The database has to be "warmed up" if it has substantial buffer memory - common in large databases.

* If database shards have duplicate tables, some writes may need to be paused while the shards are being migrated.

* Applications must be stopped at source and restarted in GCP and connection to the GCP database instance must be established.

* Network routes to the applications must be rerouted. Based on how DNS entries are set up, this can take some time.

All of these can be reduced with some planning and "cost" (some operations not permitted for some time before/after migration).

More about: https://cloud.google.com/architecture/database-migration-concepts-principles-part-1?hl=en

On Wednesday, May 25, 2022 at 11:02:07 AM UTC-5 alexh...@gmail.com wrote:

Hello

We need to migrate a CloudSQL PostgreSQL 12 database from one GCP account to another, and currently there's no automatic / 'cloudy' solution offered by Google to perform this.

Its a critical online database that cant stop for a multitude of reasons

Im looking to implement the following process.

0- Setup primary as replica source, and increase WAL files retention time

1- take a PITR backup of primary

2- restore the PITR backup as a new database server

3- Make the necessary changes on the primary and standby to activate stream replication

4- Use standby as read replica to test the application for a short period of time to test if its allright.

5- switch off the primary and activate the app on the newly promoted standby to primary

May i ask if someone have a step by step procedure as reference for this?

Also, if choosing logical replication , is it possible to keep the PITR in the same way?

Thanks

Alexandre

A CloudSQL PostgreSQL database replicated to another GCP account, initially thru PITR restore, then using stream replication and zero downtime for switchover/takeover manually by the second one. 

Once the standby is up to date with the primary (LSN, and so on), it will take over as primary (or even standalone) , then to shut down the application, and point it to the standby now as primary 

I could see that i could use pglog

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/36030f3e-33e8-4ea2-a0cd-14ec1bdb6accn%40googlegroups.com.