[google-cloud-sql-discuss] Re: I/O performance (mostly write?) intermittently goes severely low

One month after implementing serialization of the ingesting processes no performance degradation has been observed.

That suggests inserting into a table concurrently by multiple processes degrades insertion performance by order of magnitude beyond my expectation.

Strangely the degradation occurs not only on the table but also on others.

Anyway the problem seems to be successfully worked around. 

2021年5月11日火曜日 16:33:52 UTC+9 Shinji Suzuki:

Thank you, David. I'm not sure if the aborted connection is the cause or the result of the slow down or they are unrelated. though I do see isolated connection aborts. As for transaction processing, after starting to see this problem I modified the programs to do inserts in a transaction and the number of data fsync was reduced by half . I expected more reduction but your note reminds me that the reduction is less significant when dealing with bulk inserts.

I got the problem this morning again and I terminated running connections one by one. I think I've tried that before and it did not work but this time the i/o performance has recovered. Now I'm suspecting that having two connections writing massively into a single table triggers this problem, which strangely results in slow down of inserts into other tables as well. I'll serialize the jobs to ensure no simultaneous inserts and see if it makes any difference.  

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/78d795fb-32ee-4ba7-bdf4-ebd0b8f938a1n%40googlegroups.com.