This is a guest post by Ryan Ordway, DevOps Engineer at Oregon State University. At Oregon State University Libraries & Press (OSULP) we have been using Honeycomb for about 18 months. We were in the beginnings of automating our infrastructure and needed an APM solution that we could scale with. New Relic was becoming too expensive, and we couldn’t afford to monitor our whole infrastructure and trace all of our applications anymore. Thus began our Observability journey.

Kafka is a distributed, partitioned, replicated, log service developed by LinkedIn and open sourced in 2011. Basically it is a massively scalable pub/sub message queue architected as a distributed transaction log. It was created to provide “a unified platform for handling all the real-time data feeds a large company might have”.Kafka is used by many organizations, including LinkedIn, Pinterest, Twitter, and Datadog. The latest release is version 2.4.1.

If you’ve already read our guide to key Kafka performance metrics, you’ve seen that Kafka provides a vast array of metrics on performance and resource utilization, which are available in a number of different ways. You’ve also seen that no Kafka performance monitoring solution is complete without also monitoring ZooKeeper. This post covers some different options for collecting Kafka and ZooKeeper metrics, depending on your needs.

Kafka deployments often rely on additional software packages not included in the Kafka codebase itself—in particular, Apache ZooKeeper. A comprehensive monitoring implementation includes all the layers of your deployment so you have visibility into your Kafka cluster and your ZooKeeper ensemble, as well as your producer and consumer applications and the hosts that run them all.

Jenkins is an open source, Java-based continuous integration server that helps organizations build, test, and deploy projects automatically. Jenkins is widely used, having been adopted by organizations like GitHub, Etsy, LinkedIn, and Datadog. You can set up Jenkins to test and deploy your software projects every time you commit changes, to trigger new builds upon successful completion of other builds, and to run jobs on a regular schedule.

One day you’re grabbing your to-go latte, responding to Slack messages on your subway commute, and arriving at work to find a coworker waiting by your desk with a broken computer… and the next, you find yourself at home, sipping on plain old drip coffee while making huge decisions about remote work that will affect your entire organization – all with only a day’s notice.

While SaaS has made digital transformation a cakewalk, the virtual private network (VPN) takes credit when it comes to remote work. A lot of enterprises, as well as small and medium-sized businesses, continue their seamless operations remotely and securely through their VPN. VPNs enable private networks to communicate with the compute resources of public and shared networks.

Even though search is the primary function of Elasticsearch, getting search right can be tough — and sometimes even confusing. To retrieve your data in the most efficient way from Elasticsearch, sometimes you’ll need to overcome some Lucene’s obstacles. While you need to familiarize yourself with Lucene Query Syntax for advanced Kibana use, Lucene’s implementation within Elasticsearch still has some challenges.

We’re in the middle of a scary, uncertain time, and we hope those of you reading are staying safe and healthy. Despite the current challenges, the 40+ members of the remote-first Netdata team have been hard at work on the next version of the Netdata Agent: v1.21.0. This release is foundational: While we do have fantastic new collectors and three new ways to export your metrics for long-term storage, many of the most significant changes aren’t even those you’ll notice.

The world of work has had to quickly adapt to the new rules of society. Now more than ever traditional businesses are having to look beyond their immediate goals of sales and revenue and put society at the heart of their business operations. Government legislation on COVID-19 means as many people as possible must work remotely to protect staff, customers and the community.