I am using a python client google-cloudto download a file from Google Cloud Storage (GCS), getting the following error:
File "/deploy/app/scanworker/storagehandler/gcshandler.py" line 62 in download_object
blob.download_to_file(out_file)
File "/usr/local/lib/python3.5/dist-packages/google/cloud/storage/blob.py" line 464 in download_to_file self._do_download(transport, file_obj, download_url, headers)
File "/usr/local/lib/python3.5/dist-packages/google/cloud/storage/blob.py" line 418 in _do_download
download.consume(transport)
File "/usr/local/lib/python3.5/dist-packages/google/resumable_media/requests/download.py" line 169 in consume
self._write_to_stream(result)
File "/usr/local/lib/python3.5/dist-packages/google/resumable_media/requests/download.py" line 132 in _write_to_stream [args] [locals]
raise common.DataCorruption(response, msg)
DataCorruption: Checksum mismatch while downloading:
https://www.googleapis.com/download/storage/v1/b/<my-bucket>/o/<my-object>?alt=media
The X-Goog-Hash header indicated an MD5 checksum of:
fdn2kKmS4J6LCN6gfmEUVQ==
but the actual MD5 checksum of the downloaded contents was:
C9+ywW2Dap0gEv5gHoR1UQ==
I use the following code to download blob from GCS:
bucket_name = '<some-bucket>'
service_account_key = '<path to json credential file>'
with open(service_account_key, 'r') as f:
keyfile = json.load(f)
project_id = keyfile['project_id']
credentials = service_account.Credentials.from_service_account_file(service_account_key)
client = storage.Client(project=project_id,
credentials=credentials)
bucket = client.get_bucket(bucket_name)
blob_name = '<name of blob>'
download_path = "./foo.obj"
blob = bucket.blob(blob_name)
with open(download_path, "w") as out_file:
blob.download_to_file(out_file)
Some information:
- Using python3
- works in Docker Ubuntu 16.04 container in Kubernet
- using Google Analytics version 6.0.7.0, downloaded using pika
Also, I cannot reproduce the error on my local desktop by downloading the same files that were not deleted from my Docker container.
Is this a bug with the client library? Or could it be a network problem? Tried to download different files, they all give the same error from Kubernetes. The same code works for several months without problems, only seeing this error.
Edit:
Docker , , , . , , .
2:
circleci webapp . , , , , , , . Docker, , , ?
3:
, , gsutil cp gs:/<bucket>/<blob-name> foo.obj