The world of consumption has changed to everything as a service. Be it the way we consume movies and music to our office productivity tools, personal data storage, health and wellness, even our day-to-day shopping experience for consumer goods. The evolution of technology played a huge role in this. People no longer download songs or movies, nor do we buy CDs and DVDs for our personal entertainment. Rather we are now in a world where OTT (Over the Top) has become the default.

Ocean for Apache Spark has featured support for integration with Jupyter notebooks for quite some time – for details, please see our documentation. However, many developers would like to have this interactive notebook within their familiar IDE, such as VS Code, so that they can benefit from other IDE built-in features including Git integration. This article describes how to use VS Code to run Jupyter notebooks, while the code executes on an Ocean for Apache Spark cluster.

Cloud native architectures using Kubernetes and microservices have brought a lot of flexibility, enabling teams to release more applicative updates and do it multiple times a day for tens or even hundreds of services.

FinOps, or cloud financial operations, is a method that helps organizations bring financial accountability to their cloud’s operational expenses. To see success, the cross-functional teams leading FinOps efforts — including finance, IT, and business professionals — need the right tools to manage and optimize their cloud costs.

Spot by NetApp is excited to announce that Spot Security is now generally available. Delivering continuous, automated security, Spot Security analyzes, detects, and prioritizes threats to surface the most critical risks and anomalies, while providing prioritized recommendations, guided remediation, and compliance.

The Spot family has grown rapidly. Elastigroup, Ocean, and Eco have been joined by Spot PC providing Virtual Desktops, Ocean for Apache Spark, Spot Storage, and Security. Each of these solutions have individual space inside the Spot Console. Today we are excited to unveil a centralized dashboard that provides a full overview of your Spot organization. The new overview dashboard appears as the top option in the side navigation menu accessible to authenticated users.

Delight is a free, cross-platform monitoring UI for Apache Spark featuring: You can install it on top of any existing Spark infrastructure – EMR, Databricks, Spark-on-Kubernetes open-source, Cloudera/Hortonworks, … – by attaching an open-source agent to your Spark applications. Delight consists of an open-source agent attached to your Spark job, and a hosted backend accessible at delight.datamechanics.co.

Spot Ocean functions as an autopilot for the Kubernetes data-plane, as it delivers container-driven autoscaling to continuously monitor and optimize your cloud infrastructure for the cluster. Positioned at a busy crossroads in your application deployment pipeline, Ocean generates and maintains data in several manners/formats – data which is valuable when monitoring the containerized environment.

Spot’s onboarding process is simple: once you have a new AWS account, you are only a few clicks away from creating a Spot account and associating the two together. But for bigger organizations that are provisioning AWS accounts regularly, this repetitive process becomes a bit laborious. There’s also another quibble with manual onboarding: suppose you want to grant Spot new permissions for a new product you’d like to use in all of your AWS accounts.

The utilization of spot instances can be one of the fastest and easiest ways to drastically reduce the compute costs of your Spark infrastructure when you run in the cloud. In this article, we will discuss two techniques specific to Spark on Kubernetes, Enhanced Spot Instance Selection and Executor Decommissioning, that can both remove some pitfalls of spot instances and increase the reliability of your compute – resulting in faster, cheaper applications.