Dumping queue to list / array in python

Question

I start several threads and collect the result in a queue. I would like to dump it into an array or list so that I can index and retrieve these results. Each of the elements in the queue is an array of dimension n. I would like to access these arrays. Could you tell me how I would do this?

def dump_queue(model_queue): queue_list = [] for i in iter(model_queue.get,'STOP'): queue_list.append(i) return queue_list aux_model=train_svm(np.array(trainExample),np.array(trainLabel)) model_queue.put(aux_model.coef_) 

Thus, arrays are the studied parameters of the svm model. model_queue is shared between threads. I want to access each of the model parameter vectors, and not each of the model parameter records.

Answer 1

You are done with the parallel part and just want to get the results in the list, right? Then try:

 list(my_queue.queue) 

Example:

 from queue import Queue q = Queue() for i in range(5): q.put(i) l = list(q.queue) print(l) 

Output:

 [0, 1, 2, 3, 4] 

Answer 2

As Gabriel said, you should indicate your questions.

When you talk about arrays, I think you are referencing lists. Python arrays are used to store numeric values.

However, what you could do to convert your turn into a nested list: # output list:

 collectedData = list() # # # # for every thread, call: threadBuffer = [item for item in q.queue] collectedData.append(threadBuffer) 

This will collect your data in nested lists, which you can easily access, for example, in this example:

 l = list() l.append([1,2,3,4,5]) l.append([6,7,8,9,0,11,22,33]) 

Now you can freely contact: l [2] [2] → 9

Does this help you?

Best regards, Christian

Answer 3

Not sure if there was the same problem, but I needed to do the same, and in the end I wrote this. I am using threading.Thread and a Queue object in python 2.7. and just wanted to drop the queue into the list.

 def queue_to_list(q): """ Dump a Queue to a list """ # A new list l = [] while q.qsize() > 0: l.append(q.get()) return l 

Answer 4

I would go with the following solution:

 from collections import deque q = deque() for i in range(5): q.append([i,i+10]) listed_q = list(q) 

Now you can easily access values ​​by indexing or slicing:

 print listed_q[1] # [1, 11] print listed_q[1][1] # 11 

Since list () creates a copy of q, you must keep track of the dimension of your queue. The time required to complete this operation is longer than your turn. An alternative approach can be found using itertools.islice, where you first cut the queue before storing the result in a list. Check out the following links (performance indicators are also listed here):

Use fragment notation with collection.deque

How to cut deque? [Duplicate]