Python computing the state of calculations slows down the calculation itself

Question

This is a basic example of what I am saying:

Amount from 0 to 10,000,000

import time k = 0 beginning = time.time() while k < 10000000: k = k+1 elapsed = round(time.time()-beginning, 5) print (elapsed) 

Now with a kind of “status” calculation (displays the percentage every 1 second):

 import time k = 0 beginning = time.time() interval = time.time() while k < 10000000: if time.time() - interval > 1: print (round((k/100000), 5)) interval = time.time() k = k+1 elapsed = round(time.time()-beginning, 5) print (elapsed) 

The first example takes 3.67188 seconds, the second example - 12.62541 seconds. I assume that since scripts should constantly check if 1 second has passed. Is there any way to solve this problem? I found something about threads and multiprocesses, but I can't figure out how to implement it. Thanks

Answer 1

Benchmarking

I wrote different solutions and compared them. I needed to multiply my value by 10 to get measurable results. First, without any measure of progress, to find out how fast it works on my machine.

 def process_without_measuring(self): self._start = time.time() self._k = 0 while self._k < 100000000: self._k = self._k+1 print (time.time() - self._start) 

I get a duration of 13.8 seconds .

Start with your implementation:

 def process_with_original_measuring(self): self._start = time.time() next = self._start self._k = 0 while self._k < 100000000: self._k = self._k+1 if time.time() - next > 1: print (round((self._k / 1000000), 5)) next = time.time() print("duration:") print (time.time() - self._start) 

If I run this, I get a duration of 30.31 seconds and about 3 percent per second. The problem is that it must compare the time of each cycle and perform an arithmetic operation. You can reduce the time by changing the loop to:

 def process_with_manual_measuring(self): self._start = time.time() next = self._start + 1 self._k = 0 while self._k < 100000000: self._k = self._k+1 if time.time() > next: print (round((self._k / 1000000), 5)) next = time.time() + 1 print("duration:") print (time.time() - self._start) 

Instead of subtracting the timestamps of each cycle, I calculate the next timestamp only once and compare it. This, of course, is not very fast, but faster than before. This gives me 22.0 seconds , therefore saving 8 seconds with just deleting this one operation.

With a timer object as a stream, you get a much better result, and this is the preferred way:

 def show_progress(self): print (round((self._k / 1000000), 5)) self._timer = Timer(1, self.show_progress) self._timer.start() def process_with_timer(self): self._start = time.time() self._timer = Timer(1, self.show_progress) self._timer.start() self._k = 0 while self._k < 100000000: self._k = self._k+1 self._timer.cancel() print("duration:") print (time.time() - self._start) 

By running this, I get 7% more output every second, and this is done after 13.8 seconds . As you can see, there is no difference. There are not so many requests, and this is done almost immediately.

How to use a timer class

The Timer constructor expects the length of time in seconds and the method to call after the time has passed. You can use a class method, function, or lambda expression. After building you need to start the timer using start() .

The first timer is started by the process itself. After that, each time the timer is called, a new timer starts to get an interval of one second. When the process ends, remember to call cancel() on the timer. Otherwise, it will work endlessly because it will restart every time.

How to run examples

Note that the above methods are class methods, so see the indentation.

 import time from threading import Timer class ProgressMeter: def __init__(self): self._k = 0 self._timer = 0 # insert above methods with indentation as same level as __init__(self): 

To run them, you need to create an instance of ProgressMeter and call the method you want.

 meter = ProgressMeter() meter.process_with_timer() 

Answer 2

If you are using a Unix-like system, you can use signal.alarm from the signal module.

The following code is a bit messy, but you can make it tidier by encapsulating things in a class.

 import time import signal # Alarm delay in seconds. Must be an integer. alarm_delay = 1 def handler(signum, frame): report() # Request a new SIGALRM signal signal.alarm(alarm_delay) # Set a handler for the SIGALRM signal signal.signal(signal.SIGALRM, handler) # Request the initial SIGALRM signal signal.alarm(alarm_delay) # Report current progress def report(): print(round(k / 100000, 5)) k = 0 beginning = time.time() while k < 10000000: k = k + 1 elapsed = round(time.time() - beginning, 5) print (elapsed) # Cancel the pending alarm signal.alarm(0) 

typical output on my computer at 2 GHz.

 18.90911 35.98427 50.17902 64.53358 83.42723 5.94397