A Fabric administrator is reviewing workspace settings. The data engineering team reports that Spark sessions are consuming excessive cluster resources during idle periods. The administrator wants to ensure that Spark sessions automatically release compute resources when idle, without terminating active jobs. Which Spark workspace setting should the administrator enable?
- A. High Concurrency mode for Spark sessions
- B. Starter pool configuration with reduced node count
- C. Session-level Spark pool customization
- D. Automatic executor scaling (dynamic allocation)✓ Correct
Explanation
Automatic executor scaling (dynamic allocation) allows Spark to automatically add or remove executors based on workload, releasing idle compute resources while keeping the session alive for active jobs. High Concurrency mode allows multiple users to share a Spark session but does not specifically manage idle resource release. Session-level Spark pool customization lets users override pool settings per session but doesn't automatically release idle resources. The Starter pool configuration adjusts initial pool size but doesn't dynamically scale during idle periods.