FTP for Google Storage

Question

Some files are uploaded daily to the FTP server, and I need these files in Google Cloud Storage. I don’t want users to upload files to install any additional software, and just let them continue to use their FTP client. Is there a way to use GCS as an FTP server? If not, how can I create a task that periodically collects files from an FTP address and puts them in GCS? In other words: what is the best and easiest way to do this?

Answer 1

I have successfully configured FTP proxy for GCS using gcsfs in VM in Google Compute (mentioned by jkff in the comment to my question), with these instructions: http://ilyapimenov.com/blog/2015/01/19/ftp-proxy- to-gcs.html

Some changes are required:

• In /etc/vsftpd.conf change # write_enable = YES
  to write_enable = YES
• Add firewall rules to your GC project to allow access to ports 21 and passive ports 15393 through 15592 ( https://console.cloud.google.com/networking/firewalls/list )

Some possible problems:

• If you can access the FTP server using a local IP address but not a remote ip, it is possible because you have not configured firewall rules
• ftp-, , , , write_enable = YES
• , /mnt, -, , , , gcsfs_config .

, ftp- , "".

Answer 2

FTP-, GCS, , pyftpdlib

, GCS

import os
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
from google.cloud import storage

class MyHandler:
    def on_file_received(self, file):
        storage_client = storage.Client()
        bucket = storage_client.get_bucket('your_gcs_bucket')
        blob = bucket.blob(file[5:]) # strip leading /tmp/
        blob.upload_from_filename(file)
        os.remove(file)
    def on_... # implement other events

def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', 'password', homedir='/tmp', perm='elradfmw')

    handler = MyHandler
    handler.authorizer = authorizer
    handler.masquerade_address = add.your.public.ip
    handler.passive_ports = range(60000, 60999)

    server = FTPServer(("127.0.0.1", 21), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Google Container Engine ( FTP), Compute Engine. 21 60000 - 60999 .

, python my_ftp_server.py - 21, root.

Answer 3

cron rsync FTP- Google, gsutil rsync rclone.

FTP-, FTP- (Linux, Windows)