FastApi with PGMQ on Fly.io
Out of curiosity I want to explore alternatives to RabbitMQ. Todays alternative is a message queue as extension to PostgreSQL: PGMQ. To test this I deployed PGMQ to fly.io and connected a FastAPI Python app with psycopg3.
First the deployment of PGMQ. Because we want to store the data on a volume, we need a custom Dockerfile:
and of course a fly app with a volume, the fly.toml
for me looks like this:
app = "empty-dream-2474" primary_region = "ams" [[services]] internal_port = 5432 protocol = "tcp" auto_stop_machines = false auto_start_machines = true min_machines_running = 1 [[mounts]] source = "pgmq_data" destination = "/data" processes = ["app"]
The app is on purpose not stopped when idle, because it wasn't waking up when connecting. Something so solve another day.
Finally the app needs a secret for the password, i.e.
When deployed this app will provide a PostgreSQL service on port 5432 to be used from another app in the same account.
So I deployed a second fly.io app with FastAPI connecting to the PostgreSQL instance.
All apps in the same account can connect to each other internally by their region and name, i.e. the address of my PostgreSQL instance is ams.empty-dream-2474.internal
.
The full code of the FastAPI + psycopg3 on fly.io example: https://gist.github.com/mfa/1977381ef7f57c7884098c7dee233295.
Some remarks to the code:
Using parameters from a rest api, plugging them together to json and adding them to the psycopg3-SQL is messy.
The init of the extension and creation of queues for PGMQ needs to run once. This should probably happen when initializing the PostgreSQL app.
The two rest function are
send
andpop
to add a message to the queue and pop the oldest one. For more possibities see the PGMQ documentation
The send call
returns the current length of the queue:
The pop call
returns the oldest message from the queue (I called already twice before, to get to this message):
As a result of this PGMQ exploration, I will probably not use this for an actual (hobby) project. I probably don't need the performance that is possible with this. And running a custom PostgreSQL with its own Dockerfile feels like too much future maintenance.
Both, database and FastAPI app, are not deployed anymore.