I am trying to read a parquet file and perform some operations on it and save the result as parquet on HDFS. I do this using Spark. In doing so, I get the following exception.
java.io.EOFException: Premature EOF: no length prefix available at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2203) at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:176) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:867)
Any help as to what might be the cause and solution for the same.
Using CDH 5.4.1
source share