[google-cloud-sql-discuss] Re: Consistent "Segmentation fault" error in logs when running a query

After prolonged testing, I have an update: to completely stop segfaults, Query Insights must be disabled after all. Merely disabling application tags makes them less frequent, but doesn't prevent them entirely.

On Thursday, April 15, 2021 at 2:39:51 AM UTC+12 e...@contractbook.dk wrote:

Hi

> Would you be able to confirm if you are still able to see the segmentation fault errors? 

Yes, I confirm this. The last one per our logs happened today at 2021-04-13 08:11:32.469 UTC. The segmentation errors are not extremely common (due to low traffic on staging environment), but happen every day, once or twice. This may become a deal breaker should be plan to move our production DB to Google Cloud though.

> does the query involve a LEFT JOIN?

The query definition itself has two JOINs, although not LEFT JOINs. However, one of the joined relations is a view that is defined to have LEFT JOINs in it.

Noted on the SO question. I am not sure what to take from it though...

Noted on the issue tracker too. I'll gather up the info I have so far and will publish there. Thank you!

On Monday, April 12, 2021 at 12:24:55 PM UTC+3 jli...@google.com wrote:

Hello, 

Would you be able to confirm if you are still able to see the segmentation fault errors? If so, would you be able to please provide the most recent timestamp of the occurrence. 

Also, would you be able to provide a little more details regarding the query? As per Andrew's question, does the query involve a LEFT JOIN? According to the post presented here [1], this may be due to a PostgreSQL bug. Would you be able to verify this and get back to use with this information? 

Better yet, I would suggest you bring this issue up via public issue tracker [2], as this will allow for a more in depth investigation into the segmentation fault; at the same time bring the issue closer to the Cloud SQL team for further inspection.

[1] https://stackoverflow.com/questions/55585276/postgres-crashes-with-segmentation-fault-when-using-left-join

[2] issuetracker.google.com

On Tuesday, March 30, 2021 at 1:10:32 PM UTC-4 Andrew K. wrote:

Does this query involve either a LEFT JOIN or aggregate functions? We have a very similar issue with Cloud SQL Postgres 12.5, with four different queries causing segmentation faults, and these are the only similarities between then. So far we were not able to reproduce the segfaults on a local PG instance running the same DB and queries, and unfortunately it's not possible to attach a debugger to Cloud SQL (as far as we know).

On Tuesday, March 30, 2021 at 3:23:04 AM UTC+13 e...@contractbook.dk wrote:

We're seeing a "Segmentation fault" error in logs for our staging environment, caused by a single specific DB query.

I am unable to replicate the problem consistently by running the same query manually, but we see the issue in logs basically every day now.

Our environment is: PostgreSQL 13.1, the DB tier is "db-custom-1-3840".

In the 1st half of February, we were seeing "Segmentation fault" in relation to another DB query & different DB instance. That issue somehow got self-resolved around Feb 15.

I'd love to provide more info to diagnose this – is there any other info I could provide that would help solve this case? Otherwise, would it be possible for a Google Cloud SQL engineer look into this for us?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/8450f075-7305-4669-9b0b-bc1e920db737n%40googlegroups.com.