The promise of autoscaling is that workloads receive exactly the cloud computational resources they require at any given time, and you only pay for the server resources you need, when you need them.

Learn why Enterprise clients use Pepperdata products and Services: https://www.pepperdata.com/

#autoscaling #bigdataperfromance #pepperdata

Autoscaling enables applications to perform their best when demand changes, but depending on the application, performance varies. While some applications are constant and predictable, others are bound by CPU or memory, or “spiky” in nature. Autoscaling automatically addresses these variables to ensure optimal application performance. Amazon EMR, Azure HDInsight, and Google Cloud Dataproc all provide autoscaling for big data and Hadoop with a different approach.

Estimating the right number of cluster nodes for a workload is difficult; user-initiated cluster scaling requires manual intervention, and mistakes are often costly and disruptive.

Join Pepperdata Field Engineer Kirk Lewis for this discussion about operational challenges associated with maintaining optimal big data performance in the cloud, what milestones to set, and recommendations on how to create a successful cloud migration framework. Learn the following:

– What are the types of autoscaling?
– What does autoscaling do well?
– When should you use autoscaling?
– Does traditional autoscaling limit your success?
– What is optimized cloud autoscaling?

More on the episode:
So, today we're going to be talking about cost optimization, and keep in mind what do we mean by cost optimization it's a business-focused continuous discipline intended to maximize business value while reducing costs.

Keep a note on that continuous portion. We will discuss that more as we go along so today's agenda we're just going to do a quick overview of optimization types of auto-scaling in your big data cloud environments the advantage of using it and knowing when to use it um but also just don't let it impede your limit your big data cloud success.

So let's go ahead and jump right in so my cost optimization of cloud is complex if it was very simple you probably wouldn't be here listening to us so big data in the cloud has a lot of moving pieces a lot of overlap a lot of interdependencies so you need to make sense of that and on top of that you need to when you talk about cost optimization we're not just talking about stopping spend in a certain area what we're talking about is an ongoing action to maximize the utilization of the resources you are spending money on and to make sure you're spending money on the correct resources.

This came out the article just at the end of last week um cloud computing sticker shock is very real for many of you out there um there's a survey the fin ops this is people who specialize in uh cloud cost computing uh had a survey and this talks about where the real issues are understanding how to deal with shared costs accurate forecasting and spend and also just getting your engineers to act on cost optimization.

So how do you work with this well nearly half the respondents said they had little or no automation of managing cloud spend automation is generally how you do things like this
with those with automation only some of them had notifications um only a handful did spot use and automated right-sizing.

In other words, there's a lot of opportunities to optimize your cloud span that these survey authors find, and this survey is only a known part of the question.

Understanding automation and right-sizing and spot use is only part of it and we'll talk about the other portions as we move forward also make a note here the Pepperdata survey is also
available and should be seen below this.

So let's talk about the different types of auto-scaling really quickly you have fully ephemeral clusters versus kind of stem-based clusters.

In other words, do we completely shut everything down and restart every time that we launch and always have to reload in some sort of policy framework some basic software of small data sets that are housed separately, or do we leave something running all the time and just shut down most the nodes.

Then spin it back up depending on the demand of the workloads. How frequently do your workloads spin up? What's the cost of keeping that minimal space running...

Learn why Enterprise clients use Pepperdata products and Services - https://www.pepperdata.com/

Check out our blog: https://www.pepperdata.com/blog/

/////////////////////////////////////////////////////////////////////////////////////////
Connect with us:
Visit Pepperdata Website: https://www.pepperdata.com/
Follow Pepperdata on LinkedIn: https://www.linkedin.com/company/pepperdata
Follow Pepperdata on Twitter: https://twitter.com/pepperdata
Like Pepperdata on Facebook: https://www.facebook.com/pepperdata/