How to make SMTPHandler not block

Question

I installed a local SMTP server and used logging.handlers.SMTPHandler to register an exception using this code:

 import logging import logging.handlers import time gm = logging.handlers.SMTPHandler(("localhost", 25), ' [e-mail] ', [' [e-mail] '], 'Hello Exception!',) gm.setLevel(logging.ERROR) logger.addHandler(gm) t0 = time.clock() try: 1/0 except: logger.exception('testest') print time.clock()-t0 

Over 1sec completed for blocking the python script in all this time. How so? How can I make it not block the script?

Answer 1

Here is the implementation that I use, which is based on this Gmail adapted SMTPHandler .
I took part in SMTP and put it in another thread.

 import logging.handlers import smtplib from threading import Thread def smtp_at_your_own_leasure(mailhost, port, username, password, fromaddr, toaddrs, msg): smtp = smtplib.SMTP(mailhost, port) if username: smtp.ehlo() # for tls add this line smtp.starttls() # for tls add this line smtp.ehlo() # for tls add this line smtp.login(username, password) smtp.sendmail(fromaddr, toaddrs, msg) smtp.quit() class ThreadedTlsSMTPHandler(logging.handlers.SMTPHandler): def emit(self, record): try: import string # for tls add this line try: from email.utils import formatdate except ImportError: formatdate = self.date_time port = self.mailport if not port: port = smtplib.SMTP_PORT msg = self.format(record) msg = "From: %s\r\nTo: %s\r\nSubject: %s\r\nDate: %s\r\n\r\n%s" % ( self.fromaddr, string.join(self.toaddrs, ","), self.getSubject(record), formatdate(), msg) thread = Thread(target=smtp_at_your_own_leasure, args=(self.mailhost, port, self.username, self.password, self.fromaddr, self.toaddrs, msg)) thread.start() except (KeyboardInterrupt, SystemExit): raise except: self.handleError(record) 

Usage example:

 logger = logging.getLogger() gm = ThreadedTlsSMTPHandler(("smtp.gmail.com", 587), ' bugs@my _company.com', [' admin@my _company.com'], 'Error found!', (' [e-mail] ', 'top_secret_gmail_password')) gm.setLevel(logging.ERROR) logger.addHandler(gm) try: 1/0 except: logger.exception('FFFFFFFFFFFFFFFFFFFFFFFUUUUUUUUUUUUUUUUUUUUUU-') 

Answer 2

You can use QueueHandler and QueueListener . Adapted from the documents:

Along with the QueueListener class, QueueHandler can be used to allow handlers to do their work in a separate thread from the one that does the logging. This is important in web applications, as well as in other service applications, in which threads serving clients should respond as quickly as possible, while any potentially slow operations (such as sending emails via SMTPHandler) are performed in a separate thread.

Alas, they are only available with Python 3.2.

Answer 3

The simplest form of the smtp asynchronous handler for me is simply to override the emit method and use the original method in the new thread. GIL is not a problem in this case, because there is an I / O call to the SMTP server that issues the GIL. The code is as follows

 class ThreadedSMTPHandler(SMTPHandler): def emit(self, record): thread = Thread(target=SMTPHandler.emit, args=(self, record)) thread.start() 

Answer 4

Most likely, you need to write your own logging handler that will send email in the background.

Answer 5

Keep in mind when coding in Python is GIL (Global Interpreter Lock). This lock prevents multiple processes from running simultaneously. There are many things that are “locking” in Python. They will stop everything until they run out.

Currently, the only way related to GIL is to either undo the action you are trying to use for an external source, such as aix and MattH, or implement your code using the multiprocessing module (http://docs.python.org/library /multiprocessing.html), so one process handles the sending of messages, and the rest is handled by another process.