Define the term ‘Lazy Evaluation’ with reference to Apache Spark.

bookmark

Lazy evaluation is a strategy used by Apache Spark to optimize the execution of transformations on distributed datasets (RDDs). Instead of immediately executing each transformation when called, Spark delays the execution until an action is triggered. This allows Spark to optimize the execution plan by combining multiple transformations into efficient stages and minimizing data shuffling. Lazy evaluation helps Spark to avoid unnecessary computations and improve overall performance.