databricks · Certified-Data-Engineer-Professional · Q427 · multiple_choice · topic_1

Which indicators would you look for in the Spark UI’s Storage tab to signal that a cached table is not performing optim…

Which indicators would you look for in the Spark UI’s Storage tab to signal that a cached table is not performing optimally? Assume you are using Spark’s MEMORY_ONLY storage level.

A.Size on Disk is < Size in Memory
B.The RDD Block Name includes the “*” annotation signaling a failure to cache
C.Size on Disk is > 0
D.The number of Cached Partitions > the number of Spark Partitions
E.On Heap Memory Usage is within 75% of Off Heap Memory Usage

Explanation

Selected Answer: C C. Size on Disk is > 0 When using Spark's MEMORY_ONLY storage level, the ideal scenario is that the data is fully cached in memory, and the Size on Disk should be 0 (indicating that the data is not spilled to disk). If the Size on Disk is greater than 0, it suggests that some data has been spilled to disk, which can lead to degraded performance as reading from disk is slower than reading from memory.

Reference: examtopics_top_comment

Practice with progress tracking

Start tracked practice Pricing