Kdb / q how to remove entries from lists

Question

I am trying to combine multiple csv with the same columns.

getDataFromCsv :{[fn]; if[not () ~key hsym fn; data: ("zSzzSISSIIIIIffffff"; enlist "\t") 0:fn; ... do stuff... :data];} getFiles:{[dates;strat];root:"/home/me/data_";:{x: `$x, ssr[string y; "."; ""], ".csv"}[root] each dates;} getData:{[dates;strat];`tasks set ([]c:());files:getFiles[dates;strat];:getDataFromCsv each files;} 

doing this, I get a list of tables with some records where there was no file

 [0] = ([] c1;c2;c2 ... [1] = ([] c1;c2;c2 ... [2] = ([] c1;c2;c2 ... [3] = ([] c1;c2;c2 ... [4] = ([] c1;c2;c2 ... [5] = :: [6] = ([] c1;c2;c2 ... 

With these records, I cannot demolish the list to get a table that includes all the records. How can I delete these blank entries?

Answer 1

You can remove from the list where the type is not 98h for a quick fix if there are no other data types in the list:

 q)r :: +`a`b`c!(41 48 29;2 8 6;5 8 5) +`a`b`c!(41 48 29;2 8 6;5 8 5) +`a`b`c!(41 48 29;2 8 6;5 8 5) q)raze @[r;where 98h=type each r] abc ------ 41 2 5 48 8 8 29 6 5 41 2 5 48 8 8 29 6 5 41 2 5 48 8 8 29 6 5 

This also assumes that all columns are the same for each output. If not, you can use uj to join the columns:

 q)t:r,enlist ([] d:1 2 3; e:3 4 5) q)(uj/)@[t;where 98h=type each t] abcde ---------- 41 2 5 48 8 8 29 6 5 41 2 5 48 8 8 29 6 5 41 2 5 48 8 8 29 6 5 1 3 2 4 3 5 

Answer 2

Personally...

If you have "... do stuff ..." or ": data", I would just check the amount of data or add a similar check. If count = 0, return an empty list '()', and not a generic null '(: :)', which you will return in your function currently.

Generic null is the problem here and what you want to fix.

Example below ...

 // example returning generic null q){if[x~0;:(::)];([]2?10)}each 1 0 3 +(,`x)!,4 4 :: +(,`x)!,7 9 q)raze {if[x~0;:(::)];([]2?10)}each 1 0 3 (,`x)!,6 (,`x)!,2 :: (,`x)!,9 (,`x)!,2 // put a check in against 'data' to return an empty list if count=0 or similar q){if[x~0;:()];([]2?10)}each 1 0 3 +(,`x)!,3 2 () +(,`x)!,1 8 // your raze works now q)raze {if[x~0;:()];([]2?10)}each 1 0 3 x - 3 1 7 2