I'm new to Pig, so there might be a simple solution, but I can't figure it out.
Problem: I have a flat file with !as a separator, so the file structure looks something like this:
!id!value!value2
!1!100!200
When I try to load this file with pigstorage('\t'), everything is loaded into the first variable.
foo=load'bar.txt' using Pigstorage('\t') as
Id:chararray,value:chararray,value3:chararray;
This does not work as the entire file is read in a column ID. I want to split the load so that each value is read in the corresponding column.
So far I have tried customloader, which I found on how to load files with different delimiters in piglatin every time
However, this also does not work with !.
Thanks in advance