All geek questions in one place

PySpark: Get Artist ID

How can I get the executor id when running pySpark code? I know that with scala I can use SparkEnv.get().executorId(), but I cannot find the equivalent when using pySpark.

+4
source share
2 answers

The Spark user interface will give you access to the identifier of the artist, as well as their individual performance indicators.

+1
source

You can use the REST API to request artists, I used it in pySparkUtils to find the IP addresses of the artist

Boaz

0
source

Source: https://habr.com/ru/post/1679991/

More articles:


All Articles