Monday, December 31, 2018

[google-cloud-sql-discuss] Re: How to restore multiple PostgreSQL databases on a single instance to the same point in time?

The Postgres file system level backup technical details can be verified here. However, please note that the internal technical details, for instance, how the Cloud SQL Postgres managed database service backups are performed, are only available to the Cloud SQL engineering team. That way, they can guarantee that the service will remain reliable and secure.

On Friday, December 14, 2018 at 5:44:12 PM UTC-5, Alon Herbst wrote:
Thank you for the answer Aref.
Can you give more technical details on how the backup and restore are done? Which PostgreSQL files? And how is the snapshot taken without service interruption?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f6f1cc7f-afec-4eaa-a008-7dc3545022bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Friday, December 28, 2018

Re: [google-cloud-sql-discuss] Re: CloudSQL proxy & corp firewall question

That doesn't help. I need to know what ip's the sql proxy server uses to whitelist. My security team will not open the port to all ip's

On Fri, Dec 7, 2018, 10:42 AM 'Katayoon (Cloud Platform Support)' via Google Cloud SQL discuss <google-cloud-sql-discuss@googlegroups.com wrote:
Hi Scott,

When you connect using TCP, you only need to ensure outgoing TCP (3306) on remote port 3307 is allowed, however you can specify another port for it to use as well.


--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9b1bc87c-1b99-4e68-80dd-52cea363c110%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAOTThP%3DShxJVSbvQ%3DZYpw1ZcTQAFKz2wamHbbFQm5dMQrAZx_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Wednesday, December 19, 2018

[google-cloud-sql-discuss] "SQLSTATE[HY000] [2006] MySQL server has gone away" error

Hi,

My SQL Cloud Instance had a problem for ~20 hours (13:27 GMT yesterday to 09:47 today), it randomly dropped connections with the following error, without any reason :
SQLSTATE[HY000] [2006] MySQL server has gone away

The graphs were also dead for this period :

Screenshot_20181219_111405.png



I've found these errors in the logs, occurring only at the beginning and the end of this abnormal situation :

E 2018-12-18T13:27:08.099086Z 583928 [Note] Aborted connection 583928 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-18T13:27:09.334641Z 583904 [Note] Aborted connection 583904 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-18T13:27:09.335742Z 24 [Note] Aborted connection 24 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-18T13:27:11.526683Z 9 [Note] Aborted connection 9 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-19T09:47:58.863151Z 594976 [Note] Aborted connection 594976 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-19T09:47:58.864093Z 594975 [Note] Aborted connection 594975 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-19T09:47:58.914624Z 594974 [Note] Aborted connection 594974 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
E 2018-12-19T09:48:06.935322Z 594973 [Note] Aborted connection 594973 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)

My project only uses the cloudsql/gce-proxy docker image to connect to the sql server.

Any idea what could have caused this situation ?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/54e7390b-d864-425f-b04a-2d404b77406e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tuesday, December 18, 2018

[google-cloud-sql-discuss] (gcloud.sql.instances.patch) HTTPError 409: The instance or operation is not in an appropriate state to handle the request.

(gcloud.sql.instances.patch) HTTPError 409: The instance or operation is not in an appropriate state to handle the request.

We have been down for about an hour. All UI options are grayed out. Can we get a restart?

Project is:
 jcps-backpack

Cloud sql name:
jcps-backpack


Thanks

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/89272117-bd20-4ad9-8212-85838e46c3c8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Friday, December 14, 2018

[google-cloud-sql-discuss] Re: How to restore multiple PostgreSQL databases on a single instance to the same point in time?

Thank you for the answer Aref.
Can you give more technical details on how the backup and restore are done? Which PostgreSQL files? And how is the snapshot taken without service interruption?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/de380d20-ebaf-4d77-af60-f244922497b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: How to Monitor DML GoogleCloud postgres

Hi Jonathan,


At the moment this feature is only available for Cloud SQL MySQL instances and not available for Cloud PostgreSQL. There is however a feature request for this on issue tracker that you can follow for updates.


Hope this helps.



On Friday, December 14, 2018 at 12:11:39 PM UTC-5, Jonathan Delgado wrote:
Good morning, someone can help me with the following:
How to monitor DML queries in GoogleCloud postgres?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4a75e584-c626-4f9c-b856-e4c9cdb51ba1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] How to Monitor DML GoogleCloud postgres

Good morning, someone can help me with the following:
How to monitor DML queries in GoogleCloud postgres?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4878c94f-b51e-4976-a929-72332b468feb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Wednesday, December 12, 2018

[google-cloud-sql-discuss] Re: How to restore multiple PostgreSQL databases on a single instance to the same point in time?

To avoid operation disruption, the backup and recovery is done using file-system backup.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7163505c-30b7-499c-aa9e-9f32014b23f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Friday, December 7, 2018

[google-cloud-sql-discuss] Re: How to restore multiple PostgreSQL databases on a single instance to the same point in time?

Can you explain which kind of backup is used? Is it pg_dump like / file system backup / something else?
This will help to understand how the promise to restore all databases in the instance to the exact same point in time works.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/465c3193-2ad5-423b-8be9-5133d3eff485%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: How to restore multiple PostgreSQL databases on a single instance to the same point in time?

If you restore a Cloud SQL instance backup, it will restore all the databases in that instance to the point when backup was taken. One thing to note here is that if you restore the backup in the same instance it will overwrite the current data with the one in backup [1].


You can see the General tips about performing a restore at [2].


[1]https://cloud.google.com/sql/docs/postgres/backup-recovery/restoring#restorebackups-same-instance

[2] https://cloud.google.com/sql/docs/postgres/backup-recovery/restore#tips-restore



On Friday, December 7, 2018 at 10:25:09 AM UTC-5, Alon Herbst wrote:
I have 2 separate PostgreSQL databases in Google Cloud SQL. The databases should be separate due to security concerns, but they can reside in the same instance. When restoring from a backup, it is important for me to have the databases restored to the same point in time, due to data relations.
I tried to understand how the backup works. I tried to research the backup documentation and the restore one, but found no clue.
Can I be sure that restoring an instance would bring all its databases to the same point in time?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/a4f1796e-09d2-4723-8aa4-142f7efb858a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: CloudSQL proxy & corp firewall question

Hi Scott,

When you connect using TCP, you only need to ensure outgoing TCP (3306) on remote port 3307 is allowed, however you can specify another port for it to use as well.


--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/9b1bc87c-1b99-4e68-80dd-52cea363c110%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] How to restore multiple PostgreSQL databases on a single instance to the same point in time?

I have 2 separate PostgreSQL databases in Google Cloud SQL. The databases should be separate due to security concerns, but they can reside in the same instance. When restoring from a backup, it is important for me to have the databases restored to the same point in time, due to data relations.
I tried to understand how the backup works. I tried to research the backup documentation and the restore one, but found no clue.
Can I be sure that restoring an instance would bring all its databases to the same point in time?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/6f4a805c-a034-429a-803a-6b786079b3a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thursday, December 6, 2018

[google-cloud-sql-discuss] Re: Cloud SQL Proxy host

Apparently you have created a duplicate thread on the same question. In order to avoid duplicating efforts I recommend to work with the duplicate one.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c1cb1771-3702-4c87-9f3c-5f1e6903fe66%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Very high CPU Usage for few hrs every day

Hi,

Diagnosing issues with your Cloud SQL instance, you may inspect the Cloud SQL instance operation logs or the instance error logs. There is a likelihood that you are running a workload with intensive queries at the mentioned interval. 

If you couldn't find the root cause in the logs, I recommend to raise a PRIVATE report in the Issue Tracker and provide us with your project number, instance id and the timestamp you get these spikes along with your zone to investigate the issue efficiently.


--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/dac86c72-b53d-41de-9637-21c7794f2107%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] Re: Backup restore to different instance, in different project

Found a way, you can do:
gcloud sql export sql db-production gs://someproject-development-bucket/db-production_dump.sql --project someproject-production && \
gcloud sql import sql db-development gs://someproject-development-bucket/db-production_dump.sql --project someproject-development

As long as permissions are set, you can import/export to/from any project to/from any project.

---

Our setup:
  
project-development:
- Cloud SQL: db-development
- Cloud Storage: bucket-development
- Cloud Build: clone-prod-to-dev

project-production:
- Cloud SQL: db-production

// YAML from Cloud Build trigger:
steps:
  
  # export SQL databases
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'sql', 'export', 'sql',
      'db-production',
      'gs://someproject-development-bucket/db-production_dump-$BUILD_ID.sql',
      '--project', 'someproject-production'
    ]
    waitFor: ['-']
    id: 'sql-export'
    
  # import SQL databases
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'sql', 'import', 'sql',
      'db-development',
      'gs://someproject-development-bucket/db-production_dump-$BUILD_ID.sql',
      '--project', 'someproject-development'
    ]
    waitFor: ['sql-export']


On Tuesday, January 24, 2017 at 6:25:52 PM UTC+1, Dan Gardner wrote:
I'm failing to restore a backup from a source instance to a target instance when each instance is in a different project.  Is this simply not supported ?

I get ambiguous permissions error from the target instance restore cmd**.   Also if I look in the console no option is offered to target instances in diff projects,  resonable I suppose for thr console.  

I guess unless you can pass a project in the restoreBackupContext json, as below, this isn't supported yet ?
  
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     
--header 'Content-Type: application/json' \
     
--data '{ "restoreBackupContext": {"backupRunId": "[BACKUP_ID]", "instanceId": "[SOURCE_INSTANCE_NAME]", "kind": "sql#restoreBackupContext"}}' \
     
-X POST \
     https
://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[TARGET_INSTANCE_NAME]/restoreBackup

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e42b0387-9395-417a-a98b-44c849061ae2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Wednesday, December 5, 2018

[google-cloud-sql-discuss] Cloud SQL Proxy host

I am behind a corp firewall. My security team is willing to open the port but they want to allow it to only certain hosts. Does anyone know if there is a range or lists of hosts that needs to be allowed?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/47bf97ad-82e4-4b45-99a6-d83dcb4b3c21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-cloud-sql-discuss] CloudSQL proxy & corp firewall question

Our corp firewall is blocking outbound connections to port 3307. The security team is willing to work with me on opening it up, but they want to open it up to only specific IP's or ranges. 

Does anyone know of the hosts that would need to be allowed? im assuming its by zone. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1501b197-45cd-4f60-a523-c29c14517f27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tuesday, December 4, 2018

[google-cloud-sql-discuss] Very high CPU Usage for few hrs every day

Hi

Our instance configuration is this

6 core with 18 GB RAM

We dont get much traffic to our site, but for last 1 week we have been noticing that the mysql server CPU usage is every high, something like 500%. After 2-3 hrs it goes back to normal. This usually happens between around 11 AM  to 2 PM. I tried process list but there is nothing abnormal. Not sure where to look for as i am mostly a dev and dont have much db experience 

Any pointers ??? 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/16296360-a322-4bc0-9db1-93b88cdf540e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Saturday, December 1, 2018

[google-cloud-sql-discuss] Flag "lower_case_table_names" dont change after server restart

The specific flag "lower_case_table_names" don't change after server restart. Its disabled by Google SQL or need additional config ?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/5a3885ca-636c-466c-a81e-00d6af04ef3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.