Gcsfuse I / O Error

Question

I get an I / O error when trying to create a directory or file in the Google Cloud storage bucket installed in the linux directory (Ubuntu 15.10).

The steps I took:

• Created a user named transfer
• Created the /mnt/backups and ran chown -R transfer /mnt/backups
• As a user transfer, run gcsfuse --implicit-dir backup01-bucket /mnt/backups . File system completed successfully
• Run mkdir test and get mkdir: cannot create directory test: Input/output error

Is there something I missed? What I'm trying to do is to have FTP files on the server and store them in the google storage web, and not in local storage.

Update I changed the command to get debugging information:

 gcsfuse --implicit-dirs --foreground --debug_gcs --debug_fuse backup01-bucket /mnt/backups 

Then ran mkdir /mnt/backups/test as user transfer .

The following error information came out:

 fuse_debug: Op 0x00000060 connection.go:395] <- GetInodeAttributes (inode 1) fuse_debug: Op 0x00000060 connection.go:474] -> OK fuse_debug: Op 0x00000061 connection.go:395] <- LookUpInode (parent 1, name "test") gcs: Req 0x3a: <- StatObject("test/") gcs: Req 0x3b: <- ListObjects() gcs: Req 0x3c: <- StatObject("test") gcs: Req 0x3c: -> StatObject("test") (53.375107ms): gcs.NotFoundError: googleapi: Error 404: Not Found, notFound gcs: Req 0x3b: -> ListObjects() (59.061271ms): OK gcs: Req 0x3a: -> StatObject("test/") (71.666112ms): gcs.NotFoundError: googleapi: Error 404: Not Found, notFound fuse_debug: Op 0x00000061 connection.go:476] -> Error: "no such file or directory" fuse_debug: Op 0x00000062 connection.go:395] <- MkDir gcs: Req 0x3d: <- CreateObject("test/") gcs: Req 0x3d: -> CreateObject("test/") (22.090155ms): googleapi: Error 403: Insufficient Permission, insufficientPermissions fuse_debug: Op 0x00000062 connection.go:476] -> Error: "CreateChildDir: googleapi: Error 403: Insufficient Permission, insufficientPermissions" fuse: 2016/04/04 06:51:02.922866 *fuseops.MkDirOp error: CreateChildDir: googleapi: Error 403: Insufficient Permission, insufficientPermissions 2016/04/04 06:51:08.378100 Starting a garbage collection run. gcs: Req 0x3e: <- ListObjects() gcs: Req 0x3e: -> ListObjects() (54.901164ms): OK 2016/04/04 06:51:08.433405 Garbage collection succeeded after deleted 0 objects in 55.248203ms. 

Note. . If I create a directory in the web console, I can see the directory in order.

Answer 1

Of the Insufficient Permission errors in your debug output, a message appears stating that gcsfuse does not have sufficient permissions for your bucket. It probably has read-only access.

Be sure to read the credentials documentation for gcsfuse. In particular, if you use a service account in the GCE VM, be sure to configure the virtual machine using the storage-full access area.

Answer 2

The problem arises due to insufficient permissions, but you should not destroy and recreate a virtual machine with a different realm to solve this problem. Here is another approach that is more suitable for production systems:

• Create Service Account
• Create a key for the service account and upload the JSON file
• Provide the appropriate service account role
• Grant appropriate permissions to the service account in the bucket
• Upload JSON credentials for service account in VM

Finally, define an environment variable that contains the path to the credentials of the service account when calling gcsfuse from the command line:

 GOOGLE_APPLICATION_CREDENTIALS=/root/credentials/service_credential_file.json gcsfuse bucket_name /my/mount/point 

Use the key_file parameter to accomplish the same thing in fstab . Both of these options are described in the gcsfuse credential documentation . (EDIT: This parameter is documented, but will not work for me.)

Interestingly, you need to use an environment variable or key_file , even if you set up a service account in a virtual machine using:

 gcloud auth activate-service-account --key-file /root/credentials/service_credential_file.json 

For some reason, gcsfuse is ignoring the active account.

Using the storage-full when creating a virtual machine has security and stability implications as it allows the virtual machine to have full access to each bucket belonging to the same project. Should your file storage server really rewrite logs to logging or read database backups in another bucket?