Fine-grained transformation compared to coarse-grained transformations

Question

Can someone explain the difference between fine-grained and coarse-grained transformations in the context of Spark? I read an article about RDD ( https://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf ) and it is not very clear how crudely obtained transformation provides fault tolerance in an efficient way.

Answer 1

A small grainy update will update a single record in the database, while coarse grainy, as a rule, functional operators (for example, used in sparks), for example, map, reduction, flatMap, join. The Spark model uses this because when it saves your small DAG operations (small compared to the data you process), it can use this to recalculate while the original data still exists. With fine-grained updates you cannot count, because saving updates can cost as much as saving the data itself, basically, if you update each record from billions separately, you need to save this information to calculate each update, whereas with coarse grain you can save one feature that updates a billion records. Clearly, this is due to the fact that it is not as flexible as the fine-grained model.