All geek questions in one place

Failed to check serde: org.openx.data.jsonserde.jsonserde

I wrote this query to create a table on a hive. My data was originally in json format, so I downloaded and built serde and added all the jar needed to run it. But I get the following error:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe

QUERY

 create table tip(type string, text string, business_id string, user_id string, date date, likes int) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES("date.mapping"="date") STORED AS TEXTFILE;
+6
source share
2 answers

I also ran into this problem. In my case, I managed to fix this problem by adding json-serde-1.3.7-SNAPSHOT-jar-with-dependencies.jar in the hive command line, as shown below:

 hive> ADD JAR /usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.7-SNAPSHOT-jar-with-dependencies.jar;

Below are the steps that I followed in Ubuntu 14.04:

 1. Fire up Linux terminal and cd /usr/local 2. sudo git clone https://github.com/rcongiu/Hive-JSON-Serde.git 3. sudo mvn -Pcdh5 clean package 4. The serde file will be in /usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.7-SNAPSHOT-jar-with-dependencies.jar 5. Go to hive prompt and ADD JAR file as shown in Step 6. 6. hive> ADD JAR /usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.7- SNAPSHOT-jar-with-dependencies.jar; 7. Now create hive table from hive> prompt. At this stage, Hive table should be created successfully without any error.

hive Version: 1.2.1

Hadoop Version: 2.7.1

Link: Hive-JSON-Serde

+5
source

You need to create a project cloned using maven!

mvn install in the directory / path / directory / Hive -JSON-Serd here we are in / usr / local

0
source

Source: https://habr.com/ru/post/977451/

More articles:


All Articles