Worker versus process versus client and tasks and tasks with explanation in Celery Python module

Question

Recently, I started working on distributed computing to increase the speed of computing. After a rigorous Google search and many SO forums, I chose celery for my purpose, but I have to deal with some problems in understanding concepts and terminology.

What is the main difference between an employee and a process .. ?? are they the same or are called different in different contexts .. ?? Please explain

from celery docs it says:

Celery communicates through messages, usually using an intermediary to mediate between customers and employees. To initiate a task, the client adds a message to the queue, which the broker then delivers to the employee.

if so, what are the customers here .. ?? (I currently use only one octacore machine)

if messages are transmitted through a broker .. ?? why use the backend and message queues for interprocess communication ... ?? Please explain

Here I have a big question (for me) that I cannot understand. !!! when I start the Celery console:

celery worker - tasks --loglevel = info - concurrency 5

Does this mean that the celery console is a workflow that is responsible for 5 different processes and keeps track of the task queue. When a new task enters the task queue, this worker assigns the task / task to any of the 5 processes.

This question may be very simple, but please provide some links or links for a better understanding. Thanks at Advance

I am starting SO, Please feel free to edit my question for better understanding by others (this may help me more)

Answer 1

Last question:

celery worker -A tasks --loglevel=info --concurrency 5 

You are right - the worker controls 5 processes. The employee distributes tasks among 5 processes.

A “client” is any code that asynchronously performs celery tasks.

There are two different types of communication - when you run apply_async you send a request for execution by the browser (most often rabbitmq) - this is basically a set of message queues.

When employees finish work, they contribute their results to the result.

The broker and the backend of the results are quite separate and require that various types of software function optimally.

You can use RabbitMQ for both, but once you reach a certain message speed, it will not work properly. The most common combination is RabbitMQ for a broker and Redis for getting results.