Efficiently iterate through python nested lists

Question

I am working on a network traffic monitoring project in Python. Not that familiar with Python, so I'm looking for help here.

In short, I check both inbound and outbound traffic, I wrote it like this:

for iter in ('in','out'): netdata = myhttp() print data 

netdata is a list consisting of nested lists, its format is as follows:

 [ [t1,f1], [t2,f2], ...] 

Here t represents the moment and f is the flow. However, I just want to save these fs at this point for both input and output, I wonder what way to get efficient code.

After some searching, I think I need to use create a list of traffic (2 elements), and then use the zip function to iterate both lists at the same time, but it's hard for me to write the correct one. Since my netdata is a very long list, efficiency is also very important.

If there is something confusing, let me know, I will try to clarify. thanks for the help

Answer 1

Besides minor corrections in your code (questions raised by @Zero Piraeus), your question probably answered here . Possible code to iterate over a list of lists in N-degree (tree):

 def traverse(item): try: for i in iter(item): for j in traverse(i): yield j except TypeError: yield item 

Example:

 l = [1, [2, 3], [4, 5, [[6, 7], 8], 9], 10] print [i for i in traverse(l)] [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] 

The key to getting it to work is recursion, and the key to make it work is to use a generator (the yield keyword gives a hint). The generator will iterate over your list of lists, returning you item by item, without having to copy data or create a completely new list (unless you use the entire generator that assigns the result to the list, as in my example)

Using iterators and generators can be a strange concept to understand (the yield keyword is mostly). Complete this great answer to fully understand them.

Answer 2

The code you showed does not make much sense. Here is what he does:

• Let's go through the sequence of 'in', 'out' , assigning each of these two lines in turn to the iter variable (masking the built-in iter() function in this process) on its two passes through the loop.

• Completely ignore the iter value inside the loop.

• Assign the result of myhttp() to the netdata variable on each pass through the loop.

• netdata value netdata and instead try to print out the undefined data variable for each pass through the loop.

Perhaps, given the nested list you described, you want something like this:

 for t, f in myhttp(): print t print f # ... or whatever you want to do with those values. 

Answer 3

When trying to answer another, the function was not able to recursively, so I changed it so as not to overwrite. It still works pretty fast and can handle large nested lists (at least as far as I can tell from my testing). This is only a feature of Python 3.

 # Originally by Bruno Polaco def traverse(item, reverse=False): its = [item] #stack of items to-be-processed out = [] # Output (no longer generator) ite = False while len(its) > 0: it = its.pop() try: # Check if item is iterable iter(it) ite = not isinstance(it, str) except TypeError: ite = False if ite: # Do something with it for i in it: its.append(i) else: out.append(it) if not reverse: out.reverse() return out