All geek questions in one place

Reading files on Google Cloud

I tried running tensorflow-wavenet on the google cloud ml c engine gcloud ml-engine jobs submit training, but the cloud crashed when it tried to read the json configuration file:

with open(args.wavenet_params, 'r') as f:
   wavenet_params = json.load(f)

arg.wavenet_paramsis just the path to the json file that I uploaded to the Google Cloud Storage repository. The file path is as follows: gs://BUCKET_NAME/FILE_PATH.json.

I double-checked that the path to the file is correct, and I am sure that this part is responsible for the failures, since I commented on everything else.

The crash log file does not provide much information about what happened:

Module raised an exception for failing to call a subprocess Command '['python', '-m', u'gcwavenet.train', u'--data_dir', u'gs://wavenet-test-data/VCTK-Corpus-Small/', u'--logdir_root', u'gs://wavenet-test-data//gcwavenet10/logs']' returned non-zero exit status 1.

I replaced wavenet_params = json.load(f)with f.close(), and I still get the same result.

Everything works when I run it locally using gcloud ml-engine local train.

, gcloud ml-engine Google python gs://BUCKET_NAME/FILE_PATH.

+4
1

Python open GCS. , . TensorFlow ​​:

import tensorflow as tf
from tensorflow.python.lib.io import file_io

with file_io.FileIO(args.wavenet_params, 'r') as f:
  wavenet_params = json.load(f)
+9

Source: https://habr.com/ru/post/1672146/

More articles:


All Articles