Kafka and the ELK Stack — usually these two are part of the same architectural solution, Kafka acting as a buffer in front of Logstash to ensure resiliency. This article explores a different combination — using the ELK Stack to collect and analyze Kafka logs. As explained in a previous post, Kafka plays a key role in our architecture. As such, we’ve constructed a monitoring system to ensure data is flowing through the pipelines as expected.

We get it — errors suck. And you don’t want to spend too much of your time fixing them, dealing with them, investigating them, etc. In our Workflow blog post series, we’ll help you optimize your, well, workflow, from crash to resolution. To quote “the second-most frequently quoted writer in The Oxford Dictionary of Quotations after Shakespeare,” Alexander Pope, “to err is human.” There will always be errors, even in code written by the best developers.

Google Stackdriver is a cloud-based managed services platform designed to give you visibility into app and infrastructure services. Stackdriver’s monitoring, logging and APM tools make it easy to navigate between data sources to view performance details and find the root causes of any issues.

Kubernetes Pod Security Policy is a mechanism to enforce best security practices in Kubernetes. In this tutorial, we will explain how to enable Kubernetes Pod Security Policy across your cluster using kube-psp-advisor to address the practical challenges of building an adaptive and fine-grained security policy on Kubernetes in production.

If you are familiar with instrumenting applications, you may have heard of OpenMetrics, OpenTracing, and OpenCensus. These projects aim to create standards for application performance monitoring and collecting metric data. Although the projects do overlap in terms of their goals, they each take a different approach to observability and instrumentation.