Ingesting various events and documents into Elasticsearch is great for detailed analysis but when it comes to the common need to analyze data from a higher level, we need to aggregate the individual event data for more interesting insights. This is where Elasticsearch Data Frames come in. Aggregation queries do a lot of this heavy lifting, but sometimes we need to prebake the aggregations for better performance and more options for analysis and machine learning.

Log maintenance has a hidden cost. Engineers optimize their instance types, storage, networking, dependencies, and much more. However, we rarely consider the engineers themselves. A DevOps culture encourages engineers to own the solutions they build. While this increases team autonomy, it risks splitting the precious bandwidth that the team has. Automation is what makes the DevOps cycle work, and it has to cover log analysis to do a thorough job of catching issues.

Fundamentally, there are logs that will be of intrinsic value to you, and others that are less business-critical. Are you aware of the logging cost to handle, analyze and store these different types of logs? Should you really have the same approach for mission-critical logs as you do for info or telemetry logs? Differentiating your approach for different logs is challenging. If no two logs are truly the same then why should you treat them the same?

The ELK stack has become a staple of log analytics in recent years, but so too have the stories of complex maintenance and poor scalability. We’re going to discuss some of the problems with a self-hosted ELK stack and the advantages of a SaaS offering like Coralogix.

Observability is an indispensable concept in continuous delivery, but it can be a little bewildering. Luckily for us, there are a number of tools and techniques to make our job easier! One way to aid in improving observability in a continuous delivery environment is by monitoring and analyzing key metrics from builds and deploys. With tools such as Prometheus and their integrations into CI/CD pipelines, gathering and analysis of metrics is simple. Tracking these things early on is essential.

Java is used by at least 7.6 Million developers worldwide. Java logging has been a staple of the Java platform since day one, boasting extensive, resourceful documentation and rich API’s. The cornerstone of monitoring your application is efficient and widespread logging. At Coralogix, we know that logs have become one of the most important components of a modern monitoring function.

Log files are infamous for being “noisy”. Without the right management solution, trying to find a specific piece of information or using them to reproduce a critical error is a complex undertaking. If you’re working with CI/CD, how do you attribute new errors to a particular release? How do you investigate those errors and make sure that your customers aren’t being impacted? Faster releases mean shorter development and testing cycles before new code reaches production.

The cost of logging is one of the big problems of a scaled software system. Logging solutions now need to support far more than they ever have. You need to make a real investment in a logging solution that can support these initiatives. However, the up-front costs of a custom-built logging solution are prohibitive for many organizations. No business wants its bottom line affected by logging costs. That’s where Coralogix comes in.

Logging is a key requirement of any production application. .NET Core offers support for outputting logs from your application. It delivers this capability through a middleware approach that makes use of the modular library design. Some of these libraries are already built and supported by Microsoft and can be installed via the NuGet package manager, but a third party or even custom extensions can also be used for your .NET logging.

Logs in continuous delivery pipelines are often entirely ignored, right up until something goes wrong. We usually find ourselves wishing we’d put some thought into our logs, once we’re in the midst of trawling through thousands of lines. In order to try to prevent this, we can add DevOps metrics into our logs, which will provide us with greater observability, and give insight into anything going wrong in our pipelines.