Operations | Monitoring | ITSM | DevOps | Cloud

Apache Ambari is an open source management tool that helps organizations operate Hadoop clusters at scale. Ambari provides a web UI and REST API to help users configure, spin up, and monitor Hadoop clusters with one centralized platform. As your Hadoop deployment grows in size and complexity, you need deep visibility into your clusters as well as the Ambari servers that manage them. If issues arise in Ambari, it can lead to problems in your data pipelines and cripple your ability to manage clusters.

Monitoring is an essential aspect of any IT system. System metrics such as CPU, RAM, disk usage, and network throughput are the basic building blocks of a monitoring setup. Nowadays, they are often supplemented by higher-level metrics that measure the performance of the application (or microservice) itself as seen by its users (human beings on the internet or other microservices in the same or different clusters).

Five worthy reads is a regular column on five noteworthy items we’ve discovered while researching trending and timeless topics. This week, we define what data sprawl is and how organizations can cope with it effectively. Data sprawl—defined as the proliferation of data into endpoints, servers, applications, BYODs, operating systems, network environments, and even other geo-servers—can be a challenge to monitor and control.

Research firm IDC predicts that overall data volumes will grow from 33 zettabytes in 2018 to 175 zettabytes by 2025. 50% of the data will come from IoT devices and nearly 30% of the data will get consumed in real-time by 2025. The data economy has changed the game on how enterprises will need to work with digital technologies to become more effective, competitive, and innovative.

As many of you know, Gartner is recognized as one of the premier analyst firms by most enterprise IT organizations. Given the broad and diverse set of customers Gartner serves -- many risk averse and conservative towards new technologies, Gartner generally tends to recommend incremental and measured changes. Imagine our surprise when Gartner made this very bold statement "on-premises is the new legacy.

Only about four months to go before nearly 60,000 people descend on Las Vegas for the eighth annual AWS re:Invent user conference. However, that doesn’t mean that Amazon is going to embargo new service announcements until then. The recent AWS Summit in NYC looked more like a mini-re:Invent with nearly 12,000 people in attendance.

As a little introduction for everyone who has not heard about the cube yet: The cube module is there to show statistics grouped by the custom variables that have been set for the hosts and services. They are then displayed in up to three dimensions for a quick overview to show the relations. The most prominent change is the addition of services: While it used to be only possible to have the hosts in a cube, the module has now been extended to provide full functionality with services as well.

When you start a company – or a third company as is the case for Lee and me – you start with a problem statement, a product you believe in, and a lot of hope. This means when growth goes as planned or exceeds expectations, you shouldn’t be surprised. This is what is supposed to happen. Great Product + Market Opportunity + Great Team = Successful Business. Intellectually, I know all this, but it is still exciting to see it come to fruition.

In a previous post, I explained the role Apache Kafka plays in production-grade ELK deployments, as a message broker and a transport layer deployed in front of Logstash. As I mentioned in that piece, Redis is another common option. I recently found out that it is even more popular than Kafka! Known for its flexibility, performance and wide language support, Redis is used both as a database and cache but also as a message broker.