[google-cloud-sql-discuss] Re: KeyError: 'SECRET_KEY' on Secret Manager

Hello, 

The documentation you followed guides towards setting up authentication in a Flexible environment. You need to run the app locally, so you should follow Run the app on your local computer. 

This discussion group is oriented more towards general opinions, trends, and issues of general nature touching the App Engine. For coding and program architecture, as well as setting up local authentication, you may be better served in dedicated forums such as stackoverflow, where experienced programmers are within reach and ready to help. 

On Sunday, 27 February 2022 at 13:02:55 UTC-5 Valdinia - Office wrote:

I want to run a Django application on the App Engine flexible environment.

I follow the documentation step by step (https://cloud.google.com/python/django/flexible-environment).

For the Django sample app, on Cloud SQL I'm using PostgreSQL

I store the secret values (Django settings) in Secret Manager, (https://cloud.google.com/python/django/flexible-environment#store-secret-values-in-secret-manager)

Now with the backing services configured, I try to run the sample app on my computer (I mean the sample app provided by the documentation https://cloud.google.com/python/django/flexible-environment#clone_a_sample_app)

At step 4, "Run the Django migrations to set up your models and assets" when I run the  "python manage.py makemigrations" command I get a series of errors:

File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\site-packages\environ\environ.py", line 367, in get_value
    value = self.ENVIRON[var]
  File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: 'SECRET_KEY'

During handling of the above exception, another exception occurred:

....

File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\site-packages\environ\environ.py", line 371, in get_value
    raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

What could be the cause? I followed the documentation step by step.

Of course, on the code samples I edited the db name, the db password, project ID, region, instance name, bucket name etc

Thank you!

Valdinia

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/e4afe0ce-0737-4374-ac42-985ff78cc35an%40googlegroups.com.

[google-cloud-sql-discuss] Full table scan perform by spanner database

We are using a query  which has multiple left outer joins which perform full table scan while running in google cloud spanner database. In which scenarios usually it performs full table scan and how to avoid it in query level?

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c18583f4-1e38-4f13-8358-0116a85e40a8n%40googlegroups.com.

[google-cloud-sql-discuss] KeyError: 'SECRET_KEY' on Secret Manager

I want to run a Django application on the App Engine flexible environment.

I follow the documentation step by step (https://cloud.google.com/python/django/flexible-environment).

For the Django sample app, on Cloud SQL I'm using PostgreSQL

I store the secret values (Django settings) in Secret Manager, (https://cloud.google.com/python/django/flexible-environment#store-secret-values-in-secret-manager)

Now with the backing services configured, I try to run the sample app on my computer (I mean the sample app provided by the documentation https://cloud.google.com/python/django/flexible-environment#clone_a_sample_app)

At step 4, "Run the Django migrations to set up your models and assets" when I run the  "python manage.py makemigrations" command I get a series of errors:

File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\site-packages\environ\environ.py", line 367, in get_value
    value = self.ENVIRON[var]
  File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: 'SECRET_KEY'

During handling of the above exception, another exception occurred:

....

File "C:\Users\zzz\AppData\Local\Programs\Python\Python39\lib\site-packages\environ\environ.py", line 371, in get_value
    raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

What could be the cause? I followed the documentation step by step.

Of course, on the code samples I edited the db name, the db password, project ID, region, instance name, bucket name etc

Thank you!

Valdinia

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/6b667fbf-d405-4e98-b95e-c07b59da9557n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Token-based server access validation failed with an infrastructure error. Login lacks connect endpoint

Your contribution is greatly appreciated. Your solution is very useful for Techno blade clothes.

On Friday, 4 February 2022 at 03:16:44 UTC+5 David (Cloud Platform Support) wrote:

Hello,

Did you follow our quickstart about how to use SSMS with Cloud SQL and were not able to get it working properly? If you have made sure that steps were followed properly, and you are still not able to connect to your instance, I would recommend you to contact GCP support as they would be able to work with you to find out what's the issue.

In general, Cloud SQL is designed for stability over performance. If you are using MySQL, you can actually enable a flag called performance_schema. I would also make sure the instance is not being overwhelmed and in the same region as the VM. This is also a good subject to discuss with GCP support as they may advise you on things you can try to gain some performance on your instance depending on their analysis of your specific instance. However, a local server, on average, will execute queries faster.

On Wednesday, February 2, 2022 at 10:00:14 AM UTC-5 daria.a...@fitogram.pro wrote:

 I have Cloud SQL instance(16 vCPUs, Memory 128 GB, SSD storage 300 GB). Db size 110GB.

When I execute a complex query from the cloud windows VM using SQL Server Management Studio(this vm is in the same VPC as SQL Server), it always takes on average 35s. When I execute the same query from the same machine using DBeaver it takes 1.5s on average. Tried connecting with SQL Proxy, nothing changed.

Every time i try to connect the database using SSMS, I see the following error in the sql server logs in SSMS:

Login failed for user '3aec90b46f01d28\3aec90b46f01d28$'. Reason: Token-based server access validation failed with an infrastructure error. Login lacks connect endpoint permission. [CLIENT: 127.0.0.1]

Is there a solution to this? We are close to giving up migrating to Cloud SQL :(

Thanks for all the help in advance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/60d56c94-1b2f-4af2-9b6f-47f458d786a4n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Cloud SQL update operations issue

Hello

Thank you for reporting this issue. Are you getting a similar error message such as [1, 2]?

Can you kindly include the stack trace that you are getting and furthermore a sample query and call that resulted in this error. However, I would first look at the parameters that you are passing as it could be an incorrect type.

Regards

[1] TypeError: val.slice is not a function

[2] Truncated incorrect INTEGER value

On Tuesday, February 22, 2022 at 5:58:32 PM UTC-5 burn...@gmail.com wrote:

I am also facing the same problem. My API of lilpeep merch did not get data properly. I need a solution for the latest API changes.

On Wednesday, 2 February 2022 at 15:49:52 UTC+5 danidar...@flype.org wrote:

Hi! My API has worked without any issues in weeks now and suddenly as of yesterday, many of the update operations are failing due to SQL errors that I have never seen before (notice that I haven't changed the code for affected places). This is only affecting update operations and everything else seems to be working well. Is this happening to anybody else? 

Example error: val.slice is not a function

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/972c6892-5866-4590-a9d8-3bf2c9098712n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Cloud SQL update operations issue

I am also facing the same problem. My API of lilpeep merch did not get data properly. I need a solution for the latest API changes.

On Wednesday, 2 February 2022 at 15:49:52 UTC+5 danidar...@flype.org wrote:

Hi! My API has worked without any issues in weeks now and suddenly as of yesterday, many of the update operations are failing due to SQL errors that I have never seen before (notice that I haven't changed the code for affected places). This is only affecting update operations and everything else seems to be working well. Is this happening to anybody else? 

Example error: val.slice is not a function

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/21b588bf-d999-4dbb-88ad-ad664e2713bbn%40googlegroups.com.

[google-cloud-sql-discuss] Re: maintenance + cloudsqladmin

Not sure of the kind of transactions you are seeing and how are you displyaing them but according to the documentation the last step of a maintenance includes:

"The updated database engine is started on the data disk. Using a common data disk ensures that all transactions written to the original instance prior to maintenance are still present on the updated database after maintenance. If any incomplete transactions didn't finish rolling back during database shutdown, the database automatically goes through crash recovery to ensure that the database is restored to a usable state." 

Which might explain those transactions that you see. 

On Thursday, February 17, 2022 at 11:30:38 AM UTC+1 pedroagus...@gmail.com wrote:

Hi,

Does anyone know if the cloudsqladmin user performs some type of task after a scheduled maintenance, cloudsql postgres instance.
Once the maintenance is finished, the cloudsqladmin transactions per second increased considerably. I couldn't find anything in the documentation.

Any hints are much appreciated! 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1c7e6495-0c3e-405f-b890-af258b9a2453n%40googlegroups.com.

[google-cloud-sql-discuss] Re: MAXDOP gets reset back to default

There is an open feature request to set the MAXDOP on an instance level, you can check it in this link [1] and star it to give it more visibility. 

As a workaround, you could set the MAXDOP at a query level instead of database, so if the database setting changes the queries won't be impacted as they are now. 

[1]: https://issuetracker.google.com/218883128 

[2]: https://docs.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-query?view=sql-server-ver15#maxdop-integer_value 

On Thursday, February 10, 2022 at 11:32:13 AM UTC+1 daria.a...@fitogram.pro wrote:

Since Cloud SQL Server is a managed database I can't set maximum degree of parallelism on an instance level, but am able override it by altering MAXDOP on a database level in DATABASE SCOPED CONFIGURATION. But for some reason, this MAXDOP gets very randomly reset back to default 0. As it affects the overall performance, I would like to understand how can this be fixed. Thank you in advance for all the suggestions

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/b3325bd0-c73d-4835-8d3c-32357723591an%40googlegroups.com.

[google-cloud-sql-discuss] Re: Problem with importContext always import csv header

Hello Jamir, 

The easiest way to avoid importing the header row were to simply edit it out from the CSV file. There are other options; you may check, for instance, When importing a CSV from Google Storage into Google SQL is there a way to skip the first row?. 

On Saturday, 12 February 2022 at 12:52:21 UTC-5 jamir.hua...@gmail.com wrote:

I have a csv file with a header, whenever I import the csv, the header is imported as one more record in the table

uri = "gs://mybucket/backup/mysql-000000000000.csv"          

            body = {

                "importContext": {

                    "kind": "sql#importContext",

                    "fileType": "CSV",

                    "uri": uri,

                    "database": 'mydbname',

                    "csvImportOptions": {

                        "table": 'mytable'

                    }

                }

            }

            request = self.service_sql.instances().import_(project='myproject', instance='myinstance', body=body)

            response = request.execute()

Is there a way to import the csv without header using any option? or I should pass the csv without header?

Thank

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d16c21c5-b2ba-46dd-b167-fc4f95537928n%40googlegroups.com.

[google-cloud-sql-discuss] Re: How to connect to postgre using cloud auth proxy , and service account

Hello,

Please review this document that explains how to add a service account to your Cloud SQL instance so it can be used to access it. Connecting to your instance will depend on where you are connecting from. You are able to review the following document about the Cloud SQL auth proxy which will give you more understanding on how to properly connect from different environments.

If you are not able to connect to your instance after following this documentation, I would recommend you to contact GCP support where they will work with you to make sure you are successfully able to connect to your instance.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/499b35b1-b98e-4660-b272-8aa7fcd325e4n%40googlegroups.com.

[google-cloud-sql-discuss] maintenance + cloudsqladmin

Hi,

Does anyone know if the cloudsqladmin user performs some type of task after a scheduled maintenance, cloudsql postgres instance.
Once the maintenance is finished, the cloudsqladmin transactions per second increased considerably. I couldn't find anything in the documentation.

Any hints are much appreciated! 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7b7e5fb5-a275-4221-a6b1-9be2925a94f4n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Provide SQL Database Restoration Process

How did you export data, and how exactly did you restore it from backup? 

This issue might be strictly related to your project, and SQL instance. One needs to have project number and other private data to investigate. This information is not supposed to be shared in a public thread as this one. This is why it's recommendable to rather open a support case from your Cloud Console, or an issue in the Public Issue Tracker. 

On Monday, 14 February 2022 at 13:15:02 UTC-5 kprudhv...@gmail.com wrote:

we have SQL Backup retention for 7 days, Here we have the backup of 7 days.

we need to restore all data as we followed the process of restoring from  https://cloud.google.com/sql/docs/mysql/backup-recovery/restoring 

I have selected the latest backup and restored it into a new SQL instance.

But some of the data is missing in the new SQL instance. please provide steps on how to recover full data from backups

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7e994dc2-2978-4cc2-84a5-4434f3f30275n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Extension request: pqrouting

Hello,

Thank you for the feedback. I believe that there is already a feature request [1] that was submitted for this extension. Do note that feature requests are not subject to any SLA nor guaranteed to be implemented. However I would invite you to "star" or post an update on this public Feature Request to increase its visibility with our engineering team, and to enable you to receive public reports about it.

Regards,

[1] https://issuetracker.google.com/63537525

On Monday, February 14, 2022 at 1:13:13 PM UTC-5 co...@goskyway.com wrote:

Really need Google Cloud to add pgrouting ASAP. Pgrouting contains A*, Dijkstra, Traveling Salesperson and other algorithms that are useful not just for surface traffic navigation, but other network analysis problems including Urban Air Mobility / Advanced Air Mobility eVTOL electric flight.

On Monday, April 24, 2017 at 1:35:26 PM UTC-5 Patrick Brown wrote:

This would be useful for me as well

On Friday, March 24, 2017 at 7:30:39 AM UTC-6, Vladimir Rusinov wrote:

Thank you for the feedback!

We will consider doing adding this extension.

--
Vladimir Rusinov
PostgreSQL SRE, Google Ireland

Google Ireland Ltd.,Gordon House, Barrow Street, Dublin 4, Ireland
Registered in Dublin, Ireland
Registration Number: 368047

On Fri, Mar 24, 2017 at 10:12 AM, Luis Román Gutiérrez <luisr...@gmail.com> wrote:

It would be very useful for us to have support in the new Postgres CloudSQL offering for pgrouting.

Many thanks,

Luis

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/edd20c40-2479-453d-bedc-8c1947f5d894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f7206cc0-9ee8-4428-b789-b028057b4821n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Extension request: pqrouting

Really need Google Cloud to add pgrouting ASAP. Pgrouting contains A*, Dijkstra, Traveling Salesperson and other algorithms that are useful not just for surface traffic navigation, but other network analysis problems including Urban Air Mobility / Advanced Air Mobility eVTOL electric flight.

On Monday, April 24, 2017 at 1:35:26 PM UTC-5 Patrick Brown wrote:

This would be useful for me as well

On Friday, March 24, 2017 at 7:30:39 AM UTC-6, Vladimir Rusinov wrote:

Thank you for the feedback!

We will consider doing adding this extension.

--
Vladimir Rusinov
PostgreSQL SRE, Google Ireland

Google Ireland Ltd.,Gordon House, Barrow Street, Dublin 4, Ireland
Registered in Dublin, Ireland
Registration Number: 368047

On Fri, Mar 24, 2017 at 10:12 AM, Luis Román Gutiérrez <luisr...@gmail.com> wrote:

It would be very useful for us to have support in the new Postgres CloudSQL offering for pgrouting.

Many thanks,

Luis

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/edd20c40-2479-453d-bedc-8c1947f5d894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/466076c2-e2bb-4fa2-bf32-9801b5e12d38n%40googlegroups.com.

[google-cloud-sql-discuss] Provide SQL Database Restoration Process

we have SQL Backup retention for 7 days, Here we have the backup of 7 days.

we need to restore all data as we followed the process of restoring from  https://cloud.google.com/sql/docs/mysql/backup-recovery/restoring 

I have selected the latest backup and restored it into a new SQL instance.

But some of the data is missing in the new SQL instance. please provide steps on how to recover full data from backups

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/84e03192-d475-4031-8b63-d56f1c123fbfn%40googlegroups.com.

[google-cloud-sql-discuss] Re: API problem SQLAdminService.Users.List

Thanks for sharing. It is really helpful for ranboomerch data getting.

On Tuesday, 8 February 2022 at 00:17:25 UTC+5 Eyal Maoz wrote:

Hi.

First thank you very much for your comment. It is very clear.

Second I'm trying to get this information without connecting or interacting at all with the database itself.

I did open a thread in StackOverflow where a comment has mentioned that there is a feature request for this property to return somehow (hashed or a boolean) from the API it self like mentioned in the documentation.

https://stackoverflow.com/questions/70952667/gcp-api-for-cloud-sql-user-doesnt-fetch-password

Hope will happen soon.

Thanks

On Monday, February 7, 2022 at 12:53:33 PM UTC+2 mariavi...@google.com wrote:

Hello,

Regarding documentation you have shared it is true that the HTTP call doesn't show the password property and it also doesn't appear if you access through the GCP console to the CLoud SQL User's section, where you can only remove or change the password.

As all you want to know is if some user has or not a password I found a possible way that you can use to retrieve this data but instead of using the GCP console, is using the shell/terminal and going into the mySQL Database following the next steps:

1. From the shell pass the next gcloud command to enter to the mySQL Database:

$ gcloud sql connect INSTANCE_ID \ --user=root

2. Once you are inside the mySQL instance and after enter your password you must enter the following query that will retrieve a list of the user names and their password hashes, if they don't have password the space will be empty:

mysql> select user, authentication_string from mysql.user;

I hope this message helps you.

On Wednesday, February 2, 2022 at 11:50:07 AM UTC+1 Eyal Maoz wrote:

Hi,
I am trying to get information about the users of a specific SQL Database Instance by using the Google.Apis.SQLAdmin api.
You can check it here where the "Try this API" part of the list documentation page.

The return value is a List of User.
This class contain some properties that remain null, no matter what I change through the GCP console.
The properties are: Password, PasswordPolicy.
Is there a reason for that? Am I missing a request parameter that will fetch this data?
Tried MySQL 5.7 and MySQL 8.0
This is the place in console where I add\change the password:

The thing that I am trying to check is if the user has a password that he set for himself (it is important for Cloud SQL where you don't have to set password but it is very recommended). I don't mind what the password is so if you have any other direction that might help it would be great.
Thank you.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7f9b2503-ef67-4b32-b2f5-5300e9bd0566n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Beginner needing help

I appreciate you sharing such valuable information. I am facing the same problem with georgenotfound merchandise. But this solution is really helpful.

On Thursday, 27 January 2022 at 22:28:12 UTC+5 larryk...@gmail.com wrote:

  I appreciate you sharing this. It was very helpful for my first project on Karl Jacobs shirts.

On Friday, January 7, 2022 at 3:06:43 AM UTC+5 wokmou wrote:

Aggregate Functions[1] will be a good start to help you summarizes the rows of a group into a single value. They return a single result row based on groups of rows, rather than on single rows.

[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions

On Thursday, January 6, 2022 at 11:04:53 AM UTC-5 Michael Turnbull wrote:

Hi all, I'm new to this forum and just starting out on my SQL learnings in Google's Big Query. 

I have an issue where I'm trying to show data in a particular way based on multiple events. I'll try my best to explain my problem, and hopefully someone can help or offer advice. Thanks in advance. 

Problem statement - 

I'm trying to show when a customer (who has a unique identifier) has a series of job references booked over a period of 3 weeks, in order to show volume of repeat jobs for that customer. 

Example of what I have (trying to layout how I have the data in rows and columns)

Row 1 - (Column 1) = Week 1, (Column 2) = Customer 123, (Column 3) = Job ref 123

Row 2 - (Column 1) = Week 3, (Column 2) = Customer 123, (Column 3) = Job ref 345

I'd like to be able to simplify this view and show that over a 3 week period, that Customer 123 has had a repeat job ref against their unique customer number. 

I'd like it to almost show something like the below 

Row 1 - (Column 1) = Week of initial job, (Column 2) = Customer 123, Column 3 = count of first job in week 1, (Column 4) = Count of any repeat jobs in the weeks after week 1 occurred. 

Hopefully that makes sense, and this is my first time posting, so any further info needed then please ask. 

Thanks in advance for any help relating to my issue. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/f78d5cd0-b59d-40f0-bcf8-4afa2700a2c8n%40googlegroups.com.

[google-cloud-sql-discuss] Problem with importContext always import csv header

I have a csv file with a header, whenever I import the csv, the header is imported as one more record in the table

uri = "gs://mybucket/backup/mysql-000000000000.csv"          

            body = {

                "importContext": {

                    "kind": "sql#importContext",

                    "fileType": "CSV",

                    "uri": uri,

                    "database": 'mydbname',

                    "csvImportOptions": {

                        "table": 'mytable'

                    }

                }

            }

            request = self.service_sql.instances().import_(project='myproject', instance='myinstance', body=body)

            response = request.execute()

Is there a way to import the csv without header using any option? or I should pass the csv without header?

Thank

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1dcd15cc-6531-45aa-9290-c5edf704f8d2n%40googlegroups.com.

[google-cloud-sql-discuss] How to connect to postgre using cloud auth proxy , and service account

Hi,

I have a Database in postgre hosted as a service,

It is connected with cloud auth proxy 

i need to sonnect with service account and retrive values of database

Please help 

Sindhu

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/16fd8ba1-ec46-4b61-b62b-ec7221e83f23n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Is is possible to run postgREST on google cloud sql

You have the option to Use the Cloud SQL Admin API as a REST API. 

Your implementation details are better helped with by forums such as stackoverflow, where expert programmers are active and ready to help. This is a forum oriented towards opinions and general discussion on Cloud SQL matters.  

On Friday, 11 February 2022 at 04:18:53 UTC-5 bk...@atb.com wrote:

Yes, I'm sorry I should have been more specific in this regard however at the time of opening the ticket I wasnt properly informed and thus wasn't specific.

The issue is in relation to role impersonation. Given:

```
-- This file contains the definition of the applications specific roles
-- the roles defined here should not be made owners of database entities (tables/views/...)

\echo # Loading roles

-- the role used by postgrest to connect to the database
-- notice how this role does not have any privileges attached specifically to it
-- it can only switch to other roles
drop role if exists authenticator;
create role authenticator with
login
noinherit
password 'password1234blahblah';

-- this is an application level role
-- requests that are not authenticated will be executed with this role's privileges
drop role if exists anonymous;
create role anonymous;
grant anonymous to authenticator;
```

I'm not able to get the authenticator role to successfully switch/impersonate an anonymous role despite the above and given that the db uri uses the authenticator/password1234blahblah combo
In logs I get `ERROR: permission denied to set role "anonymous"`
Everything however works as expected if for example I use postgres/postgresPassword uri and with `grant anonymous to postgres` instead of the last line as shown above;

Is there some specific configuration settings that needs to be enables or adjusted to enable the above to function as expected and without using the postgres admin role specifically?

On Friday, February 11, 2022 at 3:28:07 AM UTC+6 darrel...@google.com wrote:

Hi,

Can you be more specific on what the issue is? Are you getting any error messages? 

I propose to submit your post to StackOverflow to have more expert visibility on your issue. Do note that Google Groups is geared towards general questions about GCP products.

On Wednesday, February 9, 2022 at 6:39:09 PM UTC-5 bk...@atb.com wrote:

Hi, I was wondering about possibility of running https://postgrest.org/en/stable/index.html on google cloud sql
It seems that altho everything seems to be working, I'm not able to specify one key env variable for the whole things to work, i.e. `DB_ANON_ROLE: anonymous`
Please advice!

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/581383b0-51ed-4cd9-bdc2-f97cec392c45n%40googlegroups.com.

Re: [google-cloud-sql-discuss] Cloud SQL provisioning time

Will have to check configuration code along with network latency 

After finding root cause we can support in this and also need to understand your requirements which you are trying to provision postgresql configuration as well .

On Fri, 11 Feb 2022 at 2:48 PM, 'Romeel Khan' via Google Cloud SQL discuss <google-cloud-sql-discuss@googlegroups.com> wrote:

Hello,

I have been using Terraform to provision Cloud SQL with Postgres. 

Is there any underlying reason it takes approximately 14  minutes to provision

in us-central1?

Romeel

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4f2d6036-ccff-432e-a87b-233a7c92f808n%40googlegroups.com.

--

Regards

Biswanath Giri

Google Cloud Architect 

Mob:9560016514

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/CAEJ8qt-r1xXQs37QvS612%2BKZsqtT1rVc9DLD%3DvLvK7PJU0rg0w%40mail.gmail.com.

[google-cloud-sql-discuss] Re: Is is possible to run postgREST on google cloud sql

Yes, I'm sorry I should have been more specific in this regard however at the time of opening the ticket I wasnt properly informed and thus wasn't specific.

The issue is in relation to role impersonation. Given:

```
-- This file contains the definition of the applications specific roles
-- the roles defined here should not be made owners of database entities (tables/views/...)

\echo # Loading roles

-- the role used by postgrest to connect to the database
-- notice how this role does not have any privileges attached specifically to it
-- it can only switch to other roles
drop role if exists authenticator;
create role authenticator with
login
noinherit
password 'password1234blahblah';

-- this is an application level role
-- requests that are not authenticated will be executed with this role's privileges
drop role if exists anonymous;
create role anonymous;
grant anonymous to authenticator;
```

I'm not able to get the authenticator role to successfully switch/impersonate an anonymous role despite the above and given that the db uri uses the authenticator/password1234blahblah combo
In logs I get `ERROR: permission denied to set role "anonymous"`
Everything however works as expected if for example I use postgres/postgresPassword uri and with `grant anonymous to postgres` instead of the last line as shown above;

Is there some specific configuration settings that needs to be enables or adjusted to enable the above to function as expected and without using the postgres admin role specifically?

On Friday, February 11, 2022 at 3:28:07 AM UTC+6 darrelldavid@google.com wrote:

Hi,

Can you be more specific on what the issue is? Are you getting any error messages? 

I propose to submit your post to StackOverflow to have more expert visibility on your issue. Do note that Google Groups is geared towards general questions about GCP products.

On Wednesday, February 9, 2022 at 6:39:09 PM UTC-5 bk...@atb.com wrote:

Hi, I was wondering about possibility of running https://postgrest.org/en/stable/index.html on google cloud sql
It seems that altho everything seems to be working, I'm not able to specify one key env variable for the whole things to work, i.e. `DB_ANON_ROLE: anonymous`
Please advice!

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/2a0148b5-3174-4f5b-897b-afdc1247cb8dn%40googlegroups.com.

[google-cloud-sql-discuss] Cloud SQL provisioning time

Hello,

I have been using Terraform to provision Cloud SQL with Postgres. 

Is there any underlying reason it takes approximately 14  minutes to provision

in us-central1?

Romeel

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/4f2d6036-ccff-432e-a87b-233a7c92f808n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Is is possible to run postgREST on google cloud sql

Hi,

Can you be more specific on what the issue is? Are you getting any error messages? 

I propose to submit your post to StackOverflow to have more expert visibility on your issue. Do note that Google Groups is geared towards general questions about GCP products.

On Wednesday, February 9, 2022 at 6:39:09 PM UTC-5 bk...@atb.com wrote:

Hi, I was wondering about possibility of running https://postgrest.org/en/stable/index.html on google cloud sql
It seems that altho everything seems to be working, I'm not able to specify one key env variable for the whole things to work, i.e. `DB_ANON_ROLE: anonymous`
Please advice!

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/12e24a43-4505-4d74-afee-3efbb34b3ecan%40googlegroups.com.

[google-cloud-sql-discuss] CloudSQL Postgres

Hello,

Does anyone know if I can have a problem if I downgrade my instance with the flag max_connections activated?
And also the max_connections for a 16vCPU instance?
DB version: PostgreSQL 11

Thanks!

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/d909bad7-be92-4958-9d04-2ac061607e98n%40googlegroups.com.

[google-cloud-sql-discuss] MAXDOP gets reset back to default

Since Cloud SQL Server is a managed database I can't set maximum degree of parallelism on an instance level, but am able override it by altering MAXDOP on a database level in DATABASE SCOPED CONFIGURATION. But for some reason, this MAXDOP gets very randomly reset back to default 0. As it affects the overall performance, I would like to understand how can this be fixed. Thank you in advance for all the suggestions

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/03466e12-1428-4a14-a18a-882f89a2199dn%40googlegroups.com.

[google-cloud-sql-discuss] Re: Sql Logs filled with Logon Login failed for user 'a-...-\a-...-$'.

I was able to find this Issue Tracker thread designated to keep track of updates for these logs [1], from it the Cloud SQL team is still checking this. 

As these logs can be ignored, you can filter the logs from your side in the meantime 

[1]: https://issuetracker.google.com/168698592

On Wednesday, February 2, 2022 at 11:50:14 AM UTC+1 daria.a...@fitogram.pro wrote:

Hello,

Im facing the same issue. Has this not been resolved?

Thank you for help in advance!

On Tuesday, April 14, 2020 at 4:16:39 PM UTC+2 marble...@gmail.com wrote:

Thanks for your clear and prompt response. That helps assuage my concerns that I had broken something.

Travis

On Tuesday, April 14, 2020 at 7:30:53 AM UTC-6, Katayoon (Cloud Platform Support) wrote:

Hi Travis,

This error can be ignored for the moment until the Cloud SQL product team roll out a fix from Microsoft, for which there is currently no timeline. Microsoft seems have released an update in which the Customer Experience Improvement Program (CEIP) has been removed from the product (additional more info link). The workaround to make the issue disappear is to grant CONNECT permission on your Cloud SQL Endpoint (localhost) endpoint to public which I wouldn't recommend for a production instance. Disabling CEIP only disables the upload to Microsoft and not the collection of data which is what results in the error.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/61012c78-046a-4473-b5bb-4f3d8dc021a8n%40googlegroups.com.

[google-cloud-sql-discuss] Is is possible to run postgREST on google cloud sql

Hi, I was wondering about possibility of running https://postgrest.org/en/stable/index.html on google cloud sql
It seems that altho everything seems to be working, I'm not able to specify one key env variable for the whole things to work, i.e. `DB_ANON_ROLE: anonymous`
Please advice!

If you have received this email in error, please let me know by return email so I can make sure it doesn't happen again. Because emails can contain confidential and privileged material, I'd ask for your help by deleting it and any attachments. Thanks!

We like to keep people up to date with information about new products and services at ATB or changes that could affect you. You can check out more about ATB and CASL at http://www.atb.com/important-information/privacy-security/Pages/ATB-and-CASL.aspx 

If you would like to unsubscribe from our updates, please use this URL - http://www.atb.com/important-information/privacy-security/Pages/unsubscribe.aspx

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/1fa20d30-5f7b-4744-b7d2-b7bfdd131059n%40googlegroups.com.

[google-cloud-sql-discuss] Re: Contract after using CloudSQL commitment discount

I recommend you to contact Cloud Billing support. As mentioned in that documentation page, all Google Cloud accounts get free billing and payments support. So you could ask this question there and get a response from the Billing team.

On Wednesday, February 9, 2022 at 1:33:00 AM UTC+1 hirok...@gmail.com wrote:

I'm thinking of using CloudSQL commitment discounts. What happens after one year with the one-year commitment discount? Will the confirmed discount for the next year start again one year later?

I would like to ask someone for support.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/fd1bf353-3525-43d1-89a9-5b15de81ef5en%40googlegroups.com.

[google-cloud-sql-discuss] Contract after using CloudSQL commitment discount

I'm thinking of using CloudSQL commitment discounts. What happens after one year with the one-year commitment discount? Will the confirmed discount for the next year start again one year later?

I would like to ask someone for support.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/b2b41015-d406-4359-910f-bf62d4adbad7n%40googlegroups.com.

[google-cloud-sql-discuss] Re: API problem SQLAdminService.Users.List

Hi.

First thank you very much for your comment. It is very clear.

Second I'm trying to get this information without connecting or interacting at all with the database itself.

I did open a thread in StackOverflow where a comment has mentioned that there is a feature request for this property to return somehow (hashed or a boolean) from the API it self like mentioned in the documentation.

https://stackoverflow.com/questions/70952667/gcp-api-for-cloud-sql-user-doesnt-fetch-password

Hope will happen soon.

Thanks

On Monday, February 7, 2022 at 12:53:33 PM UTC+2 mariavi...@google.com wrote:

Hello,

Regarding documentation you have shared it is true that the HTTP call doesn't show the password property and it also doesn't appear if you access through the GCP console to the CLoud SQL User's section, where you can only remove or change the password.

As all you want to know is if some user has or not a password I found a possible way that you can use to retrieve this data but instead of using the GCP console, is using the shell/terminal and going into the mySQL Database following the next steps:

1. From the shell pass the next gcloud command to enter to the mySQL Database:

$ gcloud sql connect INSTANCE_ID \ --user=root

2. Once you are inside the mySQL instance and after enter your password you must enter the following query that will retrieve a list of the user names and their password hashes, if they don't have password the space will be empty:

mysql> select user, authentication_string from mysql.user;

I hope this message helps you.

On Wednesday, February 2, 2022 at 11:50:07 AM UTC+1 Eyal Maoz wrote:

Hi,
I am trying to get information about the users of a specific SQL Database Instance by using the Google.Apis.SQLAdmin api.
You can check it here where the "Try this API" part of the list documentation page.

The return value is a List of User.
This class contain some properties that remain null, no matter what I change through the GCP console.
The properties are: Password, PasswordPolicy.
Is there a reason for that? Am I missing a request parameter that will fetch this data?
Tried MySQL 5.7 and MySQL 8.0
This is the place in console where I add\change the password:

The thing that I am trying to check is if the user has a password that he set for himself (it is important for Cloud SQL where you don't have to set password but it is very recommended). I don't mind what the password is so if you have any other direction that might help it would be great.
Thank you.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/c69bc1f4-ac6b-46da-a190-0402b3cd4eb2n%40googlegroups.com.

[google-cloud-sql-discuss] Re: API problem SQLAdminService.Users.List

Hello,

Regarding documentation you have shared it is true that the HTTP call doesn't show the password property and it also doesn't appear if you access through the GCP console to the CLoud SQL User's section, where you can only remove or change the password.

As all you want to know is if some user has or not a password I found a possible way that you can use to retrieve this data but instead of using the GCP console, is using the shell/terminal and going into the mySQL Database following the next steps:

1. From the shell pass the next gcloud command to enter to the mySQL Database:

$ gcloud sql connect INSTANCE_ID \ --user=root

2. Once you are inside the mySQL instance and after enter your password you must enter the following query that will retrieve a list of the user names and their password hashes, if they don't have password the space will be empty:

mysql> select user, authentication_string from mysql.user;

I hope this message helps you.

On Wednesday, February 2, 2022 at 11:50:07 AM UTC+1 Eyal Maoz wrote:

Hi,
I am trying to get information about the users of a specific SQL Database Instance by using the Google.Apis.SQLAdmin api.
You can check it here where the "Try this API" part of the list documentation page.

The return value is a List of User.
This class contain some properties that remain null, no matter what I change through the GCP console.
The properties are: Password, PasswordPolicy.
Is there a reason for that? Am I missing a request parameter that will fetch this data?
Tried MySQL 5.7 and MySQL 8.0
This is the place in console where I add\change the password:

The thing that I am trying to check is if the user has a password that he set for himself (it is important for Cloud SQL where you don't have to set password but it is very recommended). I don't mind what the password is so if you have any other direction that might help it would be great.
Thank you.

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/df0147df-2be0-4095-a699-2a82a23af9b7n%40googlegroups.com.