Dynamic shredding

I am trying to create a partitioned table using dynamic partitioning, but I ran into a problem. I am running Hive 0.12 on Hortonworks Sandbox 2.0.

set hive.exec.dynamic.partition=true;
INSERT OVERWRITE TABLE demo_tab PARTITION (land)
SELECT stadt, geograph_breite, id, t.country
FROM demo_stg t;

however this does not work. I get an error message.

Here is the query to create the demo_stg table :

create table demo_stg
(
    country STRING,
    stadt STRING,
    geograph_breite FLOAT,
    id INT
    )
ROW FORMAT DELIMITED FIELDS TERMINATED BY "\073";

And demo_tab :

CREATE TABLE demo_tab 
(
    stadt STRING,
    geograph_breite FLOAT,
    id INT
)
PARTITIONED BY (land STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY "\073";
  • The demo_stg table is also populated with data, so it is not empty.

Thanks for the help:)

+4
source share
2 answers

You need to change your choice:

set hive.exec.dynamic.partition=true;
INSERT OVERWRITE TABLE demo_tab PARTITION (land)
SELECT stadt, geograph_breite, id, t.country
FROM demo_stg t;

, . , select, , - id, :

INSERT OVERWRITE TABLE demo_tab PARTITION (land)
SELECT stadt, geograph_breite, id, t.country,t.id as land
FROM demo_stg t;

, .

+9

.

, true, :

set hive.exec.dynamic.partition.mode=nonstrict
+2

Source: https://habr.com/ru/post/1544712/


All Articles