Symfony and Heroku employee: can I use the database as a mechanism for communication between the front-end and the background employee?

Question

I need Heroku Worker to complete some background tasks.

Now, after reading the “Reference Jobs” article with PHP workers, I found that the architecture developed in the article uses RabbitMQ as a system messaging between the web dyno and the working speaker.

But I do not want to use RabbitMQ, because at this stage it is too complicated.

So, as the communication mechanism between web dyno and worker dyno , I would like to use one of these two alternatives:

• Just a database (basically my best bet: P)
• AWS SQS (I can use it, but the database will be better for my needs)

Now in the example presented using RabbitMQ , a callback is used to make the script live and constantly receive new messages in the queue:

 $callback = function($msg) use($app) { $app['monolog']->debug('New task received for censoring message: ' . $msg->body); try { // call the "censor" API and pass it the text to clean up $result = $app['guzzle']->get('censor', ['query' => ['corpus' => $msg->body]]); $result = json_decode($result->getBody()); if($result) { $app['monolog']->debug('Censored message result is: ' . $result->censored_text); // store in Redis $app['predis']->lpush('opinions', $result->censored_text); // mark as delivered in RabbitMQ $msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']); } else { $app['monolog']->warning('Failed to decode JSON, will retry later'); } } catch(Exception $e) { $app['monolog']->warning('Failed to call API, will retry later'); } }; $channel->basic_qos(null, 1, null); $channel->basic_consume('task_queue', '', false, false, false, false, $callback); // loop over incoming messages while(count($channel->callbacks)) { $channel->wait(); } 

My question is: how can I "emulate" the command $channel->wait() without using RabbitMQ?

In other words, how can I make worker dyno able to read a queue from a database or from AWS SQS, continuously starting to process messages as they appear in the database or in the AWS SQS queue

Should I use a scheduled task using the Heroku Scheduler that runs dyno? (Not applicable: see here why).

Or is there another thread that I am not considering?

Or maybe creating a symfony command-line application based on an external application is the ultimate solution? Will it work without stopping?

Answer 1

If you use Heroku PostgreSQL (for example), you can force your workflow to listen to asynchronous notifications that you could come from the database by creating the appropriate triggers.

See this nice post on Postgres Pub-Sub features for more details.

So, your web dino can insert or update database entries that can automatically trigger notifications for your working dino.

Your working dyno works forever and just listens to the notification channel, processing any messages it receives.

If you use a different database, it may or may not have similar capabilities. But this is definitely doable (and actually very simple) using Postgres.