Logs contain some of the most valuable data available to developers, DevOps practitioners, Site Reliability Engineers (SREs) and security teams, particularly when troubleshooting an incident. It’s not always easy to extract and use, though. One common challenge is that many log entries are blobs of unstructured text, making it difficult to extract the relevant information when you need it.

What the Beats family of log shippers are to Logstash, so Fluent Bit is to Fluentd — a lightweight log collector, that can be installed as an agent on edge servers in a logging architecture, shipping to a selection of output destinations.

Elasticsearch comes with good out-of-the-box Garbage Collection settings. So good in fact that the Definitive Guide recommends not changing them. While we agree that most use-cases wouldn’t benefit from GC tuning, especially when it turns out there simply isn’t enough heap, there are exceptions. We found that G1 GC, for example, works well on big heaps. This allows you to have less, bigger nodes, which in turn means less network traffic in a large cluster.

Providing the ultimate customer experience is the goal of every modern company, and to do that they need complete visibility into every aspect of their business. At Sumo Logic, we make it our mission to democratize machine data and make it available for everyone, which allows organizations to gain the required visibility at each step. That’s why today, we are excited to announce the availability of Search Templates to our customers.

In a world where IT infrastructure becomes more complex with each additional layer, knowing what is happening in your infrastructure becomes more complicated every day.

While logs can tell us whether a specific request failed to execute or not and metrics can help us monitor how many times this request failed and how long the failed request took, traces help us debug the reason why the request failed, or took so long to execute by breaking up the execution flow and dissecting it into smaller events.

The rather broad definition of personal data in the GDPR requires paying special attention to log data. GDPR and personal data in web server logs is a popular topic in many GDPR fora. For example, IP addresses or cookies might be considered personal data. Consequently, such data must be stored only with the consent of customers for a limited time. It is highly recommended to anonymize personal data before you hand over the logs to any 3rd party to minimize risk.