Understanding python twisted asynchrony in terms of operating system

Question

I am new to twisted library and I am trying to understand how this is done, that operations in python / twisted are asynchronous. So far, I thought that only GUI platforms (Qt or javascript) make extensive use of event driven architecture.

data:

• Twisted programs run in a single thread = no multithreading
• reactors and deferred patterns are used: callbacks / errors are declared and the execution of everything is controlled by the main loop of the reactor
• a single processor can never do anything truly parallel, because it shares its resources between processes, etc. In parallel code execution, I mean that the programming platform (python, javascript, whatever) performs more than one sequence of operations (which can be performed, for example, using multithreading)

question 1

Python can be thought of as a high-level shell for the operating system. What are the OS functions (or C functions) that provide asynchronous operation? Whether there is a?

question 2

Q1 leads me to the idea that distorted asynchrony is not true asynchrony, as in Javascript. For example, in JavaScript, if we provide 3 different buttons, attach callback functions to them, and we click all three buttons - then 3 callbacks will be executed in parallel. Indeed, in parallel.

In Twisted - as I understand it - this is not true asynchrony - it is, say, approximated asynchrony, since no operations would be performed in parallel (from the point of view of the code, as I mentioned actually 3). In Twisted, the first n line of code (defining protocols, factories, connections, etc.) is an announcement of what will happen when the whole system starts. Nothing is working yet. The actual execution is executed, then reactor.run() started. I understand that reactor runtimes are based on a single while True that repeats through events. The reactor checks all pending tasks, processes them, sends their result to the queue (either for callbacks or for errors). In the next run of the cycle, they will be processed one more step. Thus, deferred execution is actually linear (although, from the outside, it seems to be executed in parallel). Is my interpretation correct?

I would appreciate it if someone could answer my questions and / or explain how asynchrony works on the twisted / python platform and how it relates to the operating system. Thanks in advance for the good explanations!

edit: asynchronous article links are very welcome!

Answer 1

It is difficult to talk about this without specifying many terms and without affecting your facts, but here is my attempt:

Question 1:

Try man select , which roughly corresponds to the Twisted implementation - this is a way to ask the operating system to track several things at once and tell the application when one of them is triggered (block several things).

Question 2:

Yes, pretty much, but you're wrong in Javascript, it's just like Twisted.

Answer 2

Thomas has already answered your first question, but I would like to add something additional to question 2. From the way you formulated the (second) question, it seems that you misunderstand asynchrony a lot. Asynchronous should not be confused with multiprocessing. Here is an example of asynchrony. Let's say you have two tasks. 1) Reading a file from disk 2) Sum a couple of integers in memory

In a synchronous system, these tasks are performed one at a time, and we wait for the result of the operation before moving on to the next. In an asynchronous system, we start each operation, and then periodically check its completion. (This is where this excellent selection operation comes in. See this wiki page for more information: http://en.wikipedia.org/wiki/Asynchronous_I/O

Thus, even in one basic system, where in fact only one thread is running at a certain point in time, we can still have an asynchronous system so that tasks that take a lot of time (reading a file in the above example) do not throw a wrench to work for tasks that can quickly complete a job (by summing some integers in memory)

(As a note, Twisted has support for creating new threads. Http://twistedmatrix.com/documents/11.0.0/core/howto/threading.html )

Answer 3

I gave a half hour talk on this exact topic in PyCon 2012. You can watch it here.