Wednesday, August 22, 2018

[google-cloud-sql-discuss] Output DataPrep job to PostGres

I'm running jobs using DataPrep and DataFlow. I write out to BigQuery at the moment, my teams plans are to move from our postgres DB to BigQuery once we outgrow the former.

However, DataPrep and DataFlow is far superior to producing our desired output. We are currently writing the output to CSV in a storage bucket rather then running code that chunks the rows into Postgres from our local machines.

My question, feature request is, when can I write the DataPrep and DataFlow outputs directly into my Postgres DB? Would be ideal. Even when we move to BigQuery, our postgres implementation still Satisfies business cases on the cheap. So we won't move off it completely.

Thanks for reading


--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/41b16a04-3a9f-4975-aff1-b04270eee114%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment