Python Interactive Client / Server with Twisted

Question

I am trying to draw attention to how to get Twisted to execute, due to the lack of a better word, the "interactive" behavior of the client / server.

I managed to collect a couple of Protocol and ClientFactory classes that connect to the service and perform an immediate request / response (see: connectionMade → self.queryStatus). This runs as expected and prints a server response from the Factory class.

Now my problem is that I will have external events that should trigger data transfer, always listening to potential incoming data. But once the reactor.run () loop is started, I'm not sure how the rest of my application is designed to send data.

Since then, I have tried several different approaches, but this is the simplest approach that handled the recv part as described:

class myListenerProtocol(LineReceiver): delimiter = '\n' def connectionMade(self): print("Connected to: %s" % self.transport.getPeer()) self.queryStatus(1) def dataReceived(self, data): print("Receiving Data from %s" % self.transport.getPeer()) ... self.commandReceived(self.myData) def commandReceived(self, myData): self.factory.commandReceived(myData) def connectionLost(self, reason): print("Disconnected.") def queryStatus(self, CommandValue): ... strSend = CommandValue # or some such self.transport.write(strSend) class mySocketFactory(ClientFactory): protocol = myListenerProtocol def __init__(self): pass def buildProtocol(self, address): proto = ClientFactory.buildProtocol(self, address) return proto def commandReceived(self, myData): print myData reactor.stop() # It won't normally stop after recv def clientConnectionFailed(self, connector, reason): print("Connection failed.") reactor.stop() def main(): f = mySocketFactory() reactor.connectTCP("10.10.10.1", 1234, f) reactor.run() 

I suppose this is pretty straight forward, but the countless hours in the numerous examples and documentation left me without a good understanding of how I should deal with this scenario.

Answer 1

Now my problem is that I will have external events that should trigger data transfer, always listening to potential incoming data. But once the reactor.run () loop is started, I'm not sure how the rest of my application is designed to send data.

"External events"? Like what? Data coming in through the connection? Well, if you use a reactor, you can really process this data.

Or maybe someone clicks a button in a graphical interface? Try one of the GUI integration reactors - again, you will not be able to cope with these events until the reactor runs.

You are probably stuck because you think your main function is reactor.run() , and then continue to do other things. This is not how it works. When you write an event-driven program, you define all the sources of events, and then let the event loop invoke handlers when events arrive at these sources.

Answer 2

Well, there are many approaches to this, and the best one really depends on the context of your application, so I won’t stop you in detail on one of the ways to do it here, but rather connect you with the reading that I recently published on Hacker News:

• http://www.devmusings.com/blog/2013/05/23/python-concurrency/

and an example with a good use case, although it may not apply to what you are working on (or you may have read it):

• http://eflorenzano.com/blog/2008/11/17/writing-markov-chain-irc-bot-twisted-and-python/

By the way, you can also take a look at gevent or tornadoes that do such things well.

If your other “events” from the GUI toolkit (like GTK or QT) are really careful with the GIL , and even if you just need command line events, you will need threads and be careful anyway.

Finally, if you want to interact more, you can write different types of peers for your server, which interacts with different cases you use (one client that connects to the GUI, another with the CLI, another with the database, another with the SAAS API etc.).

In other words, if your project is not working, try changing your perspective!