Wherever you are on your cloud journey, you’re likely under pressure to cut or at least control your cloud spend. You may be undertaking “quick fix” cost cutting tactics such as turning off resources or moving workloads to cheaper services. But, as you may be finding out, tactical, reactive cost management activities generally only work for the short term and do not solve systemic inefficiencies that only increase over time if not addressed.

At Spot by NetApp, we see an increasing number of customers embracing Azure either as their primary cloud or as part of a hybrid or multi-cloud deployment strategy. While these customers are reaping significant business benefits from Azure, optimization of application infrastructure, controlling costs and mitigating operational burden continue to challenge Azure cloud operations (CloudOps) teams.

Let’s get right to the point: most Kubernetes workloads are underutilizing CPU and memory, with 49% of containers using less CPU than their defined requests. Although Kubernetes provides users with the ability to define resource guidelines for containers based on CPU and memory needs, it can be difficult to define and maintain these resource requirements for dynamic applications, especially in fast scaling scenarios.

The release of Apache Spark 3.4 introduced Spark Connect, and we are thrilled to announce that we now offer full support for Spark Connect on Spot Ocean for Apache Spark. This blog will provide an overview of what Spark Connect is and how to effectively utilize it with Spot Ocean for Apache Spark.