I have spark.cores.max set to 24 [3 work nodes], but if I get into my work node and see that there is only one process [command = Java] that consumes memory and processor. I suspect that it does not use all 8 cores (on m2.4x large ).
How to find out the number?
source share