How to find the last section in the HIVE table

I have a partitioned table - with 201 partition. I need to find the last section in this table and use it to publish my data. A query to search for a list of all sections:

use db; show partitions table_name; 

I need a query to find the last of these sections. Sections are in format

 ingest_date=2016-03-09 

I tried using max (), which gave me the wrong result. I do not want to go through the whole table doing

 select max(ingest_date) from db.table_name; 

This will give me the expected result ... but it will kill the whole point with a split in 1st place.

Is there a more efficient query to get the last partition for a HIve table?

+5
source share
1 answer

You can use the "show sections":

 hive -e "set hive.cli.print.header=false;show partitions table_name;" | tail -1 | cut -d'=' -f2 

As a result, you will get "2016-03-09".

+4
source

Source: https://habr.com/ru/post/1245353/


All Articles