As discussed in our blog post the Rise of the Hybrid Cloud the dramatic growth of hybrid infrastructures is being driven by benefits such as enhanced flexibility, cost optimization opportunities, and support for the agile DevOps culture. But hybrid clouds also come with their challenges, such as determining how to consistently apply security and compliance processes and how to avoid performance issues resulting from the differences between private and public cloud SLAs.

Elasticsearch Aggregations provide you with the ability to group and perform calculations and statistics (such as sums and averages) on your data by using a simple search query. An aggregation can be viewed as a working unit that builds analytical information across a set of documents. Using aggregations, you can extract the data you want by running the GET method in Kibana UI’s Dev Tools. You can also use CURL or APIs in your code.

We’re happy to announce official support for Zeek in Logz.io Security Analytics for easier security monitoring! Logz.io Security Analytics provides a unified platform for security and operations designed for cloud and DevOps environments. It’s built on top of Logz.io’s enterprise-grade ELK Stack and is extremely easy to set up and integrate with.

Organizations of all sizes—but, in particular, the larger ones—view hybrid cloud infrastructures as the new normal. The Rightscale 2019 State of the Cloud Report from Flexera (registration required) surveyed close to 800 business, IT, and development professionals around the globe. They worked for both large and small organizations across a wide range of verticals.

RabbitMQ is an open source message broker that was built to implement AMQP in 2007, and over the past twelve years has grown to include HTTP, STOMP, SMTP and other protocols via an ever growing list of plugins.

Nginx is an extremely popular open-source web server serving millions of applications around the world. Second only to Apache, Nginx’s owes its popularity as a web server (it can also serve as a reverse proxy, HTTP cache and load balancer) to the way it efficiently serves static content and overall performance.

Logs need to be stored. In some cases, for a long period of time. Whether you’re using your own infrastructure or a cloud-based solution, this means that at some stage you’ll be getting a worried email from your CFO or CPO asking you to take a close look at your logging architecture. This, in turn, will push you to limit some data pipelines and maybe even totally shut off others. Maybe we don’t need those debug logs after all, right? Wrong.

The online world is full of contrasts. On the one hand, you have site reliability engineers whose job is to keep the business running by ensuring an app’s smooth operations. On the other hand, you have the DevOps staff, whose goal is to minimize cycle time—the time from business idea to feature in production. These two teams can have conflicting objectives.

Logs have always been a crucial part of applications, providing insight into an application’s every operation and auditing all of its activities. Yet to date, logs have been used primarily for researching incident details or applicative failures. Only recently have R&D and operations teams started paying closer attention to logs in an effort to identify incidents as they occur and recognize trends that can prevent future pitfalls.

Developer teams and even operational teams often ignore monitoring applications. Deadlines, inexperience, company culture, and management can lead to poor or neglected monitoring inside developing platforms. Automating all monitoring tasks is an excellent way to avoid this scenario. Automation leads to lower costs, less time spent solving issues, and more efficient teams.

Is monitoring in the cloud special enough to warrant a list of tips and best practices? We think so. On the one hand, monitoring in the cloud might seem easy since there is a large number of solutions to choose from. On the other hand, though, the dynamic and distributed nature of the cloud can make the process much more challenging. In this article, we’ll cover ten tips and best practices that will help you ace your cloud monitoring game.

As if the temperature this summer was not high enough, this new major release of the Elastic Stack promises turns it up a notch with some hot new features. Bundling new ETL capabilities in Elasticsearch, a bunch of improvements in Kibana and a lot of new integration goodness in Filebeat and Metricbeat, Elastic Stack 7.3 is worth 5 minutes of your time to stay up to date.

Hive and Spark are two very popular and successful products for processing large-scale data sets. In other words, they do big data analytics. This article focuses on describing the history and various features of both products. A comparison of their capabilities will illustrate the various complex data processing problems these two products can address.