How to distinguish operation from spark - is it a transformation or action?

I swim recently and am confused about training and action. I read the documentation about sparks and the book about sparks, and I know that the action will cause the spark to sound in the cluster, but the transformation will not. But the rdd operations listed in the spark api doc do not indicate whether this is a transform or an action.

For example, abbreviation is an action; on the other hand, abbreviation ByKey is a transformation! Why could it be.

+5
source share
1 answer

You can tell by looking at the return type. The action will return a non-RDD type (usually your stored value types), while the conversion will return an RDD[Type] , because it still just represents your calculation.

+9
source

Source: https://habr.com/ru/post/1238717/


All Articles