Operations | Monitoring | ITSM | DevOps | Cloud

DevOps engineers wishing to troubleshoot Kubernetes applications can turn to log messages to pinpoint the cause of errors and their impact on the rest of the cluster. When troubleshooting a running application, engineers need real-time access to logs generated across multiple components. Collecting live streaming log data lets engineers: The challenge that engineers face is accessing comprehensive, live streams of Kubernetes log data.

Artificial Intelligence (AI) - or more specifically Machine Learning (ML) - and automation were big topics for many of our customers in 2022. Common reasons for the interest in AI and automation were to: increase efficiency, reduce manual processing, minimise human error and - especially for the use of ML - identify ‘unknown unknowns’.

Several tech giants like Amazon and Netflix have jumped from their monolithic applications to microservices. This has allowed them to expand their business interface tremendously and improve their services. Not only them, but most businesses today are dependent on microservices. Twitter currently has about a thousand such services working together, releasing meaningful outputs.

Over the past decade, Prometheus has become the most prominent open source monitoring tool in the world, allowing users to quickly and easily collect metrics on their systems and help identify issues in their cloud infrastructure and applications. Prometheus was originally developed by SoundCloud when the company felt their metrics and monitoring solutions weren’t meeting their needs.

We are using data wrong. In today’s data-driven world, we have learned to store data. Our data storage capabilities have grown exponentially over the decades, and everyone can now store petabytes of data. Let’s all collectively pat ourselves on the back. We have won the war on storing data! Congratulations!

Many business organizations begin their data analytics journey with great expectations of discovering hidden insights from data. The concept of unified storage — data lake technologies in the cloud — have gained momentum in recent years, especially with the exponential options for cost-effective cloud-based storage services. Big data is readily available. In fact, 2.5 quintillion (2.5 x 10^18 or 2.5 billion billion) bytes generated every day!