Operations | Monitoring | ITSM | DevOps | Cloud

February 2022

Orchestrate Spark pipelines with Airflow on Ocean for Apache Spark

Running Apache Spark applications on Kubernetes has a lot of benefits, but operating and managing Kubernetes at scale has significant challenges for data teams. With the recent addition of Ocean for Apache Spark to Spot’s suite of Kubernetes solutions, data teams have the power and flexibility of Kubernetes without the complexities. A cloud-native managed service, Ocean Spark automates cloud infrastructure and application management for Spark-on-Kubernetes.

Continuous EC2 Spot Market Prediction for Continuous Optimization

Using spot instances for mission-critical workloads always carried the risk of interruptions, making their use, while financially attractive, less than ideal from a reliability perspective. Spot by NetApp has made it possible for cloud consumers to use spot instances for dramatic cost savings while ensuring high availability for all kinds of workloads. Spot Availability Scores are core to our cloud infrastructure offerings, which are leveraged to provide maximum availability while mitigating risks.

Ready to run! Get Started with Spark on Kubernetes

The Apache Spark and Kubernetes integration was recently officially declared Generally Available and Production Ready, generating a lot of interest from the community. More and more companies choose to run their big data workloads on Kubernetes to benefit from containerization and a standard cloud-native ecosystem.