Import a large text file into matlab (including comments, textscan)

Question

Im trying to import large txt files (> 1gb) into matlab.

this is a data structure:

667.55535 -0.00 0.000 0.0158 667.5554 -0.01 -0.000 0.0158 667.55545 -0.01 4.037 10.0000 667.5555 -0.00 4.000 10.0000 #1 Trigger Camera 10 Hz #2 Trigger Camera 10 Hz 667.55555 -0.00 4.000 10.0000 667.5556 -0.01 4.000 10.0000 

I am using the textscan function:

 segarray = textscan(file_id, '%f %f %f %f', blocksize, 'delimiter','\n', 'commentStyle', '#'); 

works very well, but I need comments marked with a "#" if I change the format string to "% f% f% f% f% s" and delete the parameter "commentStyle", "#" every second line is read as one line : /

any ideas?

Answer 1

you can use

 segarray = textscan(fid, '%f %f %f %f %[^\n]'); 

to achieve what you want (therefore without any textscan() options). The last character of the format means that textscan will match any trailing character that is not a newline.

This leads to:

test.txt :

 667.55535 -0.00 0.000 0.0158 667.5554 -0.01 -0.000 0.0158 667.55545 -0.01 4.037 10.0000 667.5555 -0.00 4.000 10.0000 #1 Trigger Camera 10 Hz #2 Trigger Camera 10 Hz 667.55555 -0.00 4.000 10.0000 667.5556 -0.01 4.000 10.0000 667.5555 -0.11 4.000 12.0000 #1 Trigger Camera 11 Hz #2 Trigger Camera 12 Hz 667.5557 -0.00 4.000 10.0000 667.556 -0.01 4.000 10.0000 667.55855 -0.00 4.000 10.0000 667.5596 -0.01 4.000 10.0000 667.55105 -0.00 4.000 10.0000 667.5511 -0.01 4.000 10.0000 

segarray{:} :

 [first three columns omitted for brevity] ans = 0.0158 0.0158 10.0000 ... % fourth column abbreviated 10.0000 10.0000 ans = '' '' '' '#1 Trigger Camera 10 Hz #2 Trigger Camera 10 Hz ' '' '' '#1 Trigger Camera 11 Hz #2 Trigger Camera 12 Hz ' '' '' '' '' ''