Google Cloud Run as message queue for AX Webhooks

Andreas

2020-12-22 12:00

Introduction

In the evaluation phase of a new hobby project I built a Google Cloud Run service that accepts and validates webhooks from AX Semantics.
The Code is on Github: https://github.com/mfa/ax-webhook-service.

The current version uses Google Cloud Datastore to store the texts. The advantage of saving to Datastore is speed and low costs. Processing the texts can be done later by a cronjob as described in the next section.

An example for a cronjob processing

First: Authentication!

If the cron is run in the Google Cloud everything is set. When not in the Google Cloud (locally or elsewhere), see Google Auth Api Documentation.

For this example I chose the service account with json keyfile option.

The snippet gets all texts for the collection 12345 and prints them:

from google.cloud import datastore
client = datastore.Client()

query = client.query(kind="AX-NLG-Text")
query.add_filter("collection_id", "=", 12345)

for item in query.fetch():
    print(item.get("uid"), " -- ", item.get("data").get("text"))

A cronjob would probably delete the item after processing it, to prohibit another processing on the next cronjob run.
To delete an item use client.delete(item).
The full example is on Github in https://github.com/mfa/ax-webhook-service/tree/main/example.