Hive ParseException - cannot recognize input near 'end' 'string'

I get the following error when trying to create a Hive table from an existing DynamoDB table:

NoViableAltException( 88@ []) at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:9123) at org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:30750) ...more stack trace... FAILED: ParseException line 1:77 cannot recognize input near 'end' 'string' ',' in column specification 

The query looks like this (simplified to protect the innocent):

 CREATE EXTERNAL TABLE moveProjects (cid string, end string, category string) STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' TBLPROPERTIES ("dynamodb.table.name" = "Projects", "dynamodb.column.mapping" = "cid:cid,end:end,category:category"); 

Basically, I am trying to create a Hive table containing the contents of the Projects DynamoDB table, but the create statement throws a parsing error from Hive / Hadoop.

+4
source share
2 answers

The problem is not really a syntax error, Hive ParseException is simply caused by a reserved keyword in Hive (in this case end ).

Solution: use backlinks around the name of the calling column:

 CREATE EXTERNAL TABLE moveProjects (cid string, `end` string, category string) STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' TBLPROPERTIES ("dynamodb.table.name" = "Projects", "dynamodb.column.mapping" = "cid:cid,end:end,category:category"); 

With the added reverse windows around end query works as expected.

Reserved words in the Amazon gorge (as of February 2013):

IF, HAVING, WHERE, SELECT, UNIQUEJOIN, JOIN, ON, TRANSFORM, MAP, REDUCE, TABLESAMPLE, CAST, FUNCTION, EXTENDED, CASE, WHEN, THEN, ELSE, END, DATABASE, CROSS

Source: This beehive ticket from the Facebook Phabricator tracker

+20
source

I used / Date = 20161003 in the folder path when performing rewriting of the insert, and it failed. I changed it to / Dt = 20161003 and it worked

+1
source

Source: https://habr.com/ru/post/1500762/


All Articles