All geek questions in one place

Apache Spark: SparkPi example

I am trying to run Spark Examples, and I just do not understand what is going on. I used

MASTER=spark://Illidan:7077 ./bin/run-example SparkPi 10

which starts this process, but all I get are INFO messages .

So what is "10" for?

Can I exchange INFO messages?

Where is the day off? Where is Pi calculated?

Can I run the example from the shell? Do I have to run it from the spark shell to see the prints, or is it stored in some file that I don’t know about?

I swear to God, I processed the documentation a hundred times. I need help.

Hier is a small fragment of my terminal output. Thanks in advance .: D

14/12/31 00:02:25 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.2.5:44913 (size: 1295.0 B, free: 267.3 MB)
14/12/31 00:02:26 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 25231 ms on 192.168.2.7 (8/10)
14/12/31 00:02:26 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 25358 ms on 192.168.2.5 (9/10)
14/12/31 00:02:26 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.2.4:36505 (size: 1295.0 B, free: 267.3 MB)
14/12/31 00:02:27 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 25877 ms on 192.168.2.4 (10/10)
14/12/31 00:02:27 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
+4
source share
2 answers

, 10 - ( ), . PI 10 (PI ).

, . .

PI .

+3

OrangePi One SBC

  • : 1,6 H3 Cortex-A7 H.265/HEVC 4K

  • : Mali400MP2 @600 , OpenGL ES 2.0

  • : 512 DDR3 ( GPU)

  • Armbian OS Debian GNU/Linux 8 (jessie) 3.4.112-sun8i

, OrangePi . 4 , . . . , , GPU (~ 7 GigaFLOPS).

root @orangepione: ~/spark/spark-2.0.0-bin-hadoop2.7 #./bin/run-example SparkPi 10

  • 10 - ///

1 4-

  • 19.0 . 3.145951459514595
  • 19.0 . 3.1346713467134673

2 4- .

  • 19,3 . 3.1420757103785517
  • 19,4 . 3.13639568197841.

4 4-

  • 21,2 . 3.141427853569634
  • 21,5 . 3.1445478613696536

10 4-

  • 40,8 . 3.143983143983144
  • 40,4 . 3.141019141019141.

50 4-

  • 156,5 . 3.1399118279823655
+1

Source: https://habr.com/ru/post/1569490/

More articles:


All Articles