Measuring elapsed time in python

Question

Is there an easy way / module to correctly measure elapsed time in python? I know that I can just call time.time() twice and account for the difference, but this will lead to incorrect results if the system time is changed. Of course, this does not happen very often, but it indicates that I am measuring the wrong thing.

Using time.time() to measure duration is incredibly circular when you think about it. You accept the difference between two measurements of absolute time, which, in turn, are built according to measurements of duration (performed by timers) and known absolute times (set manually or via ntp), which does not interest you at all.

So, is there a way to request this "timer time" directly? I would suggest that it can be represented as a millisecond or microsecond, which does not have a meaningful absolute representation (and therefore, it does not need to be configured with system time). Looking back a bit, it seems that this is exactly what System.nanoTime() does in Java, but I did not find the corresponding Python function, although it should (technically and technically) be simpler than time.time() .

Edit: To avoid confusion and answer the answers below: this is not about DST changes, and I also do not want CPU time - I want the elapsed physical time. It does not have to be very fine-grained or even not particularly accurate. It just should not give me a negative duration, or a duration that is disabled several orders of magnitude (higher than granularity), just because someone decided to set the system clock to a different value. Here is what Python docs says about "time.time ()":

"Although this function usually returns non-decreasing values, it can return a lower value than the previous call if the system clock was set back between the two calls"

This is exactly what I want to avoid, as it can lead to strange things, such as negative values ​​in time calculations. I can get around this for now, but I think it's a good idea to learn how to use the right solutions where possible, as kludges will return to bite you one day.

Edit2: Some research shows that you can get the system-independent dimension that I want on Windows with GetTickCount64 (), on Linux you can get it in the return value of times (). However, I still cannot find a module that provides this functionality in Python.

Answer 1

What you are looking for is a monotonous timer . A monotonous countdown does not jump or go back.

Several attempts have been made to implement a cross-platform monotome clock for Python based on a link to the OS. (Windows, POSIX and BSD are completely different) See Discussions and some attempts at monotonous time in this SO post .

Basically, you can just use os.times ():

os.times ()

Returns a 5-bit floating point number indicating the accumulated (processor or other) time, in seconds. Elements: user time, system time, time for children, time in the system for children, and elapsed real time from a fixed point in the past, in that order. See Temporary Unix Help Pages (2) or the corresponding Windows Windows API Platform documentation. On Windows, only the first two elements are full, the rest are zero.

Availability: Unix, Windows

But this does not fill in the necessary elapsed real time (fifth tuple) in Windows.

If you need Windows support, consider ctypes , and you can directly call GetTickCount64 (), as was done in this recipe .

Answer 2

To measure elapsed processor time, see time.clock () . This is the equivalent of Linux times () user time fields.

For benchmarking use timeit .

The datetime module , which is part of Python 2.3+ , also has microsecond time, if supported by the platform.

Example:

 >>> import datetime as dt >>> n1=dt.datetime.now() >>> n2=dt.datetime.now() >>> (n2-n1).microseconds 678521 >>> (n2.microsecond-n1.microsecond)/1e6 0.678521 ie, it took me .678521 seconds to type the second n2= line -- slow >>> n1.resolution datetime.timedelta(0, 0, 1) 1/1e6 resolution is claimed. 

If you are concerned about changing the system time (from DS → ST), just check the object returned by datetime.Presumable, the system time may have a slight adjustment from NTP . This should be reduced and corrections applied gradually , but ntp sync beats can have an effect with very small (milliseconds or microseconds) references to time.

You can also reference the Alex Martelli C function if you want any of this permission. I would not go too far to reinvent the wheel. The exact time is basic, and most modern OSs do a pretty good job.

Edit

Based on your explanation, it looks like you need a simple side check if the system clock has changed. Just compare with a friendly local ntp server:

 import socket import struct import time ntp="pool.ntp.org" # or whatever ntp server you have handy client = socket.socket( socket.AF_INET, socket.SOCK_DGRAM ) data = '\x1b' + 47 * '\0' client.sendto( data, ( ntp, 123 )) data, address = client.recvfrom( 1024 ) if data: print 'Response received from:', address t = struct.unpack( '!12I', data )[10] t -= 2208988800L #seconds since Epoch print '\tTime=%s' % time.ctime(t) 

NTP is accurate to milliseconds over the Internet and has a resolution of 2–32 seconds (233 picoseconds). Should it be good enough?

Remember that the 64-bit NTP data structure will overflow in 2036 and every 136 years after that - if you really want a reliable solution, it is better to check overflow ...

Answer 3

Python 3.3 added a monotonous timer to the standard library, which does exactly what I was looking for. Thanks Paddy3118 for pointing this out in "How to get monotonous time durations in python?" .

Answer 4

 >>> import datetime >>> t1=datetime.datetime.utcnow() >>> t2=datetime.datetime.utcnow() >>> t2-t1 datetime.timedelta(0, 8, 600000) 

Using UTC avoids these periods of embarrassment when the clock shifts due to daylight saving time.

Regarding the use of an alternative method, rather than subtracting two hours, keep in mind that the OS actually contains a clock that is initialized from the hardware clock on the PC. Modern OS implementations will also support the synchronization of these watches with some official source so that it does not drift. This is much more accurate than any timer that a PC can run.

Answer 5

The examples of functions that you specify in your editing are two completely different things:

• Linux times () returns the process time in milliseconds of the CPU. Python's equivalent is time.clock () or os.times() .
• Windows GetTickCount64 () returns system uptime.

Although there are two different functions, both (potentially) can be used to identify system clocks that have burped using these methods:

Firstly:

You can take system time with time.time() and processor time with time.clock() . Since the wall clock time will ALWAYS be greater than or equal to the processor time, discard any measurements where the interval between the two time.time() readings is less than the time.clock() paired test readings.

Example:

 t1=time.time() t1check=time.clock() # your timed event... t2=time.time() t2check=time.clock() if t2-t1 < t2check - t1check: print "Things are rotten in Denmark" # discard that sample else: # do what you do with t2 - t1... 

Second:

Ensuring system uptime is also promising if you are concerned about the system clock, since the user reset in most cases does not consider reset the number of uptime counts. (what i know ...)

Now the more complex question is: getting the system uptime regardless of the platform - especially without the appearance of a new shell - in the second second. Hmmm ...

Probably the best option is psutil . View source , they use uptime = GetTickCount() / 1000.00f; for Windows and sysctl "kern.boottime" for BSD / OS X, etc. Unfortunately, they all have a resolution of 1 second,

Answer 6

 from datetime import datetime start = datetime.now() print 'hello' end = datetime.now() delta = end-start print type(delta) <type 'datetime.timedelta'> import datetime help(datetime.timedelta) ...elapsed seconds and microseconds...