The best way to access a value from the last iteration in a loop

Question

What is the best and fastest way to access a value from a previous iteration in a for loop, assuming the object will be very large (for example, a cursor object that contains 100,000 entries)

Using a simple example:

tmp = [
         ['xyz', 335], ['zzz', 338], ['yyy', 339], ['yyy', 442], 
         ['abc', 443], ['efg', 444], ['ttt', 446], ['fff', 447]
      ]

for x in tmp:
   if not prev:
     prev = x[1]
   print 'seq: ', x[1], 'prev seq:', prev, 'variance: ', x[1]-prev
   prev = x[1]

Is this the best way to handle this?

Based on the answers below, I did some testing: tmp was created with 500 lists, the average value to run it 20 times is shown below.

results:

Mines: 0.623
Dave snippet1: 0.605
Dave snippet2: 0.586
Catchmeifyoutry (edited code): 0.707

Answer 1

" " , . - .

, , :

tmp_iter = iter(tmp)
prev = tmp_iter.next()

for x in tmp_iter: 
   print 'seq: ', x[1], 'prev seq:', prev[1], 'variance: ', x[1]-prev[1]
   prev = x

, :

tmp_iter = iter(tmp)
[_, prev] = tmp_iter.next()

for [_, x] in tmp_iter: 
   print 'seq: ', x, 'prev seq:', prev, 'variance: ', x-prev
   prev = x

, _, .

Answer 2

, zip(), .

UPDATE: python 2.x, itertools.izip , !

from itertools import izip
for prev, next in izip(tmp, tmp[1:]):
    print 'seq: ', next[1], 'prev seq:', prev[1], 'variance: ', next[1]-prev[1]

, :

for (_, prev), (_, next) in izip(tmp, tmp[1:]):
    print 'seq: ', next, 'prev seq:', prev, 'variance: ', next-prev

,

for prev, next in izip(tmp, tmp[:1] + tmp):
    print 'seq: ', next[1], 'prev seq:', prev[1], 'variance: ', next[1]-prev[1]

, :

itr = iter(tmp)
itr.next() # here I assume tmp is not empty, otherwise an exception will be thrown
for prev, next in izip(tmp, itr):
    print 'seq: ', next[1], 'prev seq:', prev[1], 'variance: ', next[1]-prev[1]

. zip . , :

xs = range(9)
triplets = zip(xs[::3], xs[1::3], xs[2::3]) # python 2.x, zip returns a list

print xs       # [0, 1, 2, 3, 4, 5, 6, 7, 8]
print triplets # [(0, 1, 2), (3, 4, 5), (6, 7, 8)]

, python 3 zip , itertools.izip.

Answer 3

itertools:

from itertools import izip, islice
for prev, cur in izip(l, islice(l, 1, None)):
    print 'seq:', cur[1], 'prev seq:', prev[1], 'delta:', cur[1]-prev[1]

, , , 32- ints, , numpy:

import numpy
a = numpy.array([x[1] for x in tmp])
delta = numpy.diff(a)

Answer 4

Guido !

itertools recipes :

import itertools
def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = itertools.tee(iterable)
    next(b, None)
    return itertools.izip(a, b)

(, iterable (random.randint(100) for x in xrange(1000)), iter(iterable); next(iterable), .

:

for prev_item, item in pairwise(iterable):
    …

Answer 5

NameError, if not prev prev . False None. :

for i in xrange(1, len(tmp)):
    print 'seq: {0}, prev seq: {1}, variance: {2}'.format(tmp[i][1], tmp[i - 1][1], tmp[i] - tmp[i - 1][1])

100 000 , , , . : () 100 . , :

( , )

def reader(filename):
    with open(filename) as f:
        prev = f.next()
        for l in f:
            l = l.split('\t')
            yield (prev, l)
            prev = l

for (prev, curr) in reader(myfile):
    print 'seq: {0}, prev seq: {1}, variance: {2}'.format(curr[1], prev[1], curr[1] - prev[1])

readeris a generator , it repeatedly returns values ​​from a sequence. Thus, only 2 rows of data will be stored in memory at any time, and your application will support even millions of rows.

To make the code readable, I set it aside, so in the body of the program we looked at the sequence of data without worrying about how it is composed.

Answer 6

it = imap(operator.itemgetter(1), tmp) # get all 2nd items
prev = next(it, None) # get 1st element (doesn't throw exception for empty `tmp`)
for x in it:
    print 'seq: %s prev seq: %s variance: %s' % (x, prev, x-prev)
    prev = x