How to get notified when an object is loaded into my GCS bucket?

Question

I have an application that regularly uploads photos to a GCS bucket. When these photos load, I need to add thumbnails and do some analysis. How to set up notifications for a bucket?

Answer 1

The way to do this is to create a Cloud Bub / Sub for new objects and configure the GCS bucket to publish messages in this section when creating new objects.

First create a PHOTOBUCKET bucket:

$ gsutil mb gs://PHOTOBUCKET

Now be sure to activate the Cloud Pub / Sub API .

Cloud Pub/Sub GCS gsutil:

$ gsutil notification create \
    -t uploadedphotos -f json \
    -e OBJECT_FINALIZE gs://PHOTOBUCKET

-t Pub/Sub. , gsutil .

-e , OBJECT_FINALIZE ( ). .

-f , , API JSON.

, gsutil, gcloud gsutil update, gsutil.

, . Pub/Sub:

$gcloud beta pubsub subscriptions processphotos --topic = uploadedphotos

. Python. :

def poll_notifications(subscription_id):
    client = pubsub.Client()
    subscription = pubsub.subscription.Subscription(
        subscription_id, client=client)
    while True:
        pulled = subscription.pull(max_messages=100)
        for ack_id, message in pulled:
            print('Received message {0}:\n{1}'.format(
                message.message_id, summarize(message)))
            subscription.acknowledge([ack_id])

def summarize(message):
    # [START parse_message]
    data = message.data
    attributes = message.attributes

    event_type = attributes['eventType']
    bucket_id = attributes['bucketId']
    object_id = attributes['objectId']
    return "A user uploaded %s, we should do something here." % object_id

, :

https://cloud.google.com/storage/docs/reporting-changes https://cloud.google.com/storage/docs/pubsub-notifications

Answer 2

GCP Pub/Sub, . POST (), . Google Pub/Sub.

https://cloud.google.com/storage/docs/object-change-notification

Answer 3

AttributeError: "google.cloud.pubsub" ""

Answer 4

using this example ! keep two things in mind: 1) they upgraded the code to python 3.6 pub_v1, which may not work on python 2.7 2) when calling poll_notifications (projectid, subscriptionname), pass the identifier of your GCP project: for example, bold-idad and the name subscrition, for example asrtopic