How to decode a csv file with long lines in a tensor stream using tf.decode_csv?

Question

How to decode a file csvwith long lines (for example, with many elements per line, so that it is not realistic to display them one by one for output) with tf.TextLineReader () and tf.decode_csv?

Typical Usage:

reader = tf.TextLineReader()    
key, value = reader.read(filename_queue)    
record_defaults = [1,1,1,1,1]    
a,b,c,d,e =  tf.decode_csv(records=value,record_defaults=record_defaults, field_delim=" ")

When we have thousands of elements in a row, it is impossible to assign them one by one as (a, b, c, d, e) above, can all elements be decoded into a list or something like that?

Answer 1

Let's say you have 1800 data columns. You can use this as a default entry:

record_defaults=[[1]]*1800

and then use

all_columns = tf.decode_csv(value, record_defaults=record_defaults)

to read them.

Answer 2

, tf.decode_csv , :

record_defaults = [[1], [1], [1], [1], [1]]
all_columns = tf.decode_csv(value, record_defaults=record_defaults)
all_columns
Out: [<tf.Tensor 'DecodeCSV:0' shape=() dtype=int32>,
 <tf.Tensor 'DecodeCSV:1' shape=() dtype=int32>,
 <tf.Tensor 'DecodeCSV:2' shape=() dtype=int32>,
 <tf.Tensor 'DecodeCSV:3' shape=() dtype=int32>,
 <tf.Tensor 'DecodeCSV:4' shape=() dtype=int32>
]

:

sess = tf.Session() 
sess.run(all_columns)
Out: [1, 1, 1, 1, 1]

, 1 record_defaults. .