Is it possible to convert specific columns using Hive streaming?

The following is the syntax below:

SELECT TRANSFORM(col1,col2,col3) USING 'python script.py' AS col_1,col_2,col_3 FROM... 

The python script actually only converts col3, and col1, col2 just go through the python script without any changes. I want to reuse this python script so that I can replace col1, col2 with any arbitrary columns while col1 has passed. But the following two codes do not work:

 SELECT col1, col2, TRANSFORM(col3) USING 'python script.py' AS col_1 FROM... SELECT TRANSFORM(col3) USING 'python script.py' AS col_3, col1, col2 FROM... 

If there is a way to transfer only a subset of the selected columns to a streaming script, and the other selected columns to leave the streaming process?

early.

+4
source share
1 answer

This will not work because the conversion can return multiple rows. Imagine if script.py for some col3 returns 2 lines (col31 and col32). What would the syntax mean? Return col1, col2, col31 and col1, col2, col32?

0
source

Source: https://habr.com/ru/post/1495394/


All Articles