Operations | Monitoring | ITSM | DevOps | Cloud

Coralogix

Parsing Multiline Logs - The Complete Guide

In the context of logging, multiline logs happen when a single log is written as multiple lines in the log file. When logs are sent to 3rd party log monitoring platforms like Coralogix using standard shipping methods (e.g. Fluentd, Filebeat), which read log files line-by-line, every new line creates a new log entry, making these logs unreadable for the user.

Elasticsearch Disk and Data Storage Optimizations with Benchmarks

Out of the four basic computing resources (storage, memory, compute, network), storage tends to be positioned as the foremost one to focus on for any architect optimizing an Elasticsearch cluster. Let’s take a closer look at a couple of interesting aspects in relation to the Elasticsearch storage optimization and let’s do some hands-on tests along the way to get actionable insights. The storage topic consists of two general perspectives.

42 Elasticsearch Query Examples - Hands-on Tutorial

Elasticsearch provides a powerful set of options for querying documents for various use cases so it’s useful to know which query to apply to a specific case. The following is a hands-on tutorial to help you take advantage of the most important queries that Elasticsearch has to offer. In this guide, you’ll learn 42 popular query examples with detailed explanations, but before we get started, here’s a summary of what the types of queries we’ll tackle.

Overcoming DNS barriers for Kubernetes Scaling

It was a cloudy winter morning when I had arrived at the office and found, to our horror, that a Kubernetes cluster was suffering from extremely high CPU and network usage and had become almost completely non-functional. To make things worse, restarting the nodes (the go-to DevOp solution), seemed to have absolutely no effect on the issue. Something was poisoning the network and we had to find out what it was and fast.

How SIEM is evolving in 2020

The evolution of Security Information and Event Management (SIEM) is deeply intertwined with cloud computing, both in terms of technological breakthroughs the cloud provided and from its inherent security challenges. With the rise of cloud computing, we no longer rely on long-lived resources. An ephemeral infrastructure obscures the identity of the components and, even if you do have the visibility it doesn’t necessarily mean you can comprehend the meaning behind the components.

Dynamic alerts

The power and value that’s embedded in logs are reflected by the status and behavior of our applications and infrastructure. Many times we would like to be alerted when the application or its components show abnormal behavior. This behavior can be reflected by the application sending some logs at a higher than usual volume. Figuring out exactly what ‘higher than usual’ means, or in other words, setting the threshold value at which the alert should trigger can be a daunting task.

Optimizing a Golang service to reduce over 40% CPU

10 years ago, Google was facing a critical bottleneck caused by extremely prolonged C++ compilation times and needed a totally new way to solve it. Google engineers tackled the challenge by creating a new language called Go (aka Golang). The new language of Go borrows the best parts of C++, (most notably it’s performance and security features) and combines it with the speed of Python to enable Go to quickly use multiple cores while being able to implement concurrency.

How to get the most out of your ELB logs

Amazon ELB (Elastic Load Balancing) allows you to make your applications highly available by using health checks and intelligently distributing traffic across a number of instances. It distributes incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, IP addresses, and Lambda functions. You might have heard the terms, CLB, ALB, and NLB. All of them are types of load balancers under the ELB umbrella.

Guide: Smarter AWS Traffic Mirroring for Stronger Cloud Security

So, you’ve installed Coralogix’s STA and you would like to start analyzing your traffic and getting valuable insights but you’re not sure that you’re mirroring enough traffic or wondering if you might be mirroring too much data and could be getting more for less. The harsh truth is that in order to be able to detect everything, you have to capture everything and in order to be able to investigate security issues thoroughly, you need to capture every network packet.

Using Auditbeat to protect your critical infrastructure

Beats are lightweight, purpose-built agents that acquire data and then feed it to Elasticsearch. Beats use the libbeat framework that makes it easy to create customized beats for any type of data you’d like to send to Elasticsearch. Auditbeat is a lightweight shipper from the Beats family that you can install on your servers to audit the activities of users and processes on your systems.