Operations | Monitoring | ITSM | DevOps | Cloud

July 2024

Myth #4 of Apache Spark Optimization: Manual Tuning

In this blog series we’ve been examining the Five Myths of Apache Spark Optimization. The fourth myth we’re considering relates to a common misunderstanding held by many Spark practitioners: Spark application tuning can eliminate all of the waste in my applications. Let’s dive into it.

Myth #4 of Apache Spark Optimization | Manual Tuning

Manual tuning can remediate some waste, but it doesn’t scale or address in-application waste. Watch this conversation to learn why manually tuning your Apache Spark applications is not the best approach to achieving optimization with price and performance in mind. Visit Pepperdata's page for information on real time, autonomous optimization for Apache Spark applications on Amazon EMR and EKS.