i missed "fileType": "CSV" in "importContext"
On Wednesday, April 4, 2018 at 3:35:11 PM UTC+2, Johannes Valbjorn wrote:
HiI am trying to get a script working to import a CSV from Cloud Storage to Cloud SQL but i get 'kind': 'sql#operationError' , 'code': 'INTERNAL_ERROR'.
I finally got a script to work calling the Import api found athttps://developers.google.com/apis-explorer/?hl=en_US#p/ sqladmin/v1beta4/sql. instances.import and afterwards checking the job withBut when i get response back an Internal error has occured, here is the full response:
INFO:root:{'kind': 'sql#operation', 'selfLink': 'https://www.googleapis.com/sql/v1beta4/projects/REDACTED ', 'targetProject': 'statistics-198910', 'targetId': 'invitationsdata-staging', 'targetLink': 'https://www.googleapis.com/sql/v1beta4/projects/REDACTED ', 'name': 'f63f4b33-0e67-4a11-9b8e-156f5c13c0b0' , 'operationType': 'IMPORT', 'status': 'DONE', 'user': 'REDACTED', 'insertTime': '2018-04-04T08:52:24.182Z', 'startTime': '2018-04-04T08:52:24.277Z', 'endTime': '2018-04-04T08:52:34.467Z', 'error': {'kind': 'sql#operationErrors', 'errors': [{'kind': 'sql#operationError', 'code': 'INTERNAL_ERROR'}]}, 'importContext': {'kind': 'sql#importContext', 'uri': 'gs://REDACTED.csv', 'database': 'trustpilot'}}
I had the GCP console opened, in a browser tab at the same time, showing the Cloud SQL instance that i wanted to import to and at the same time as my script yielded INTERNAL ERROR i saw a popup in the browser window saying "Unknown Error".
I have tried to do the same import using the console and it works fine.
here is the script i used:
from google.oauth2 import service_account
import googleapiclient.discovery
from time import sleep
import logging
logging.basicConfig(level=logg
ing .INFO)logger = logging.getLogger()
SCOPES = ['https://www.googleapis.com/
auth/sqlservice.admin ']SERVICE_ACCOUNT_FILE = 'SOMEPLACE'
credentials = service_account.Credentials.fr
om_service_account_file (SERVICE_ACCOUNT_FILE)
service = googleapiclient.discovery.buil
d ('sqladmin', 'v1beta4', credentials=credentials)
instance_id = 'REDACTED'
project_id = "REDACTED"
def upload():
processed = False
while processed == False:
try:
instances_import_request_body = {
"importContext": {
"kind": "sql#importContext",
"uri": "REDACTED",
"database": "trustpilot",
"csvImportOptions": {
"table": 'invitations'
}
}
}
request = service.instances().import_(pr
oject =project_id, instance=instance_id, body=instances_import_request_body )response = request.execute()
logger.info("Request made ...")
process_status = False
while not process_status:
sleep(1)
logger.info("Checking with GS")
resp = service.operations().get(proje
ct =project_id, operation=response['name']).execute ()logger.info(resp)
if resp['status'] == "DONE":
logger.info("Request completed.")
process_status = True
processed=True
except Exception as err:
logger.info("Error: {}".format(err.args))
logger.info("Probably an operation is in progress. Retrying after 5 seconds.")
sleep(5)
upload()
redacted details are available upon request
I need to get a script working that imports a csv to Cloud SQL usiing the import API, but it seems the errors are in GCP Cloud SQL itself
Can anybody help we figure this out?
Kind regards
Johannes Valbjørn
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/7b6c2794-f708-4011-a2ee-d844d1bd99eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
No comments:
Post a Comment