Logging is a key requirement of any production application. .NET Core offers support for outputting logs from your application. It delivers this capability through a middleware approach that makes use of the modular library design. Some of these libraries are already built and supported by Microsoft and can be installed via the NuGet package manager, but a third party or even custom extensions can also be used for your .NET logging.

Logs in continuous delivery pipelines are often entirely ignored, right up until something goes wrong. We usually find ourselves wishing we’d put some thought into our logs, once we’re in the midst of trawling through thousands of lines. In order to try to prevent this, we can add DevOps metrics into our logs, which will provide us with greater observability, and give insight into anything going wrong in our pipelines.

Kubernetes (or “K8s”) is an open-source container orchestration tool developed by Google. In this tutorial, we will be leveraging the power of Kubernetes to look at how we can overcome some of the operational challenges of working with the Elastic Stack.

It is tempting to consider logging as a simple, solved problem. We write a log, check our file and, boom, we’ve cracked it. Yet those of us who have sat up at three in the morning, trawling through log files over an unreliable SSH connection, know that this is simply not enough. As your system scales, so too must the sophistication of your tooling. Your logging best practices must be scalable and ready to support your efforts.

The CSV file format is widely used across the business and engineering world as a common file for data exchange. The basic concepts of it are fairly simple, but unlike JSON which is more standardized, you’re likely to encounter various flavors of CSV data. This lesson will prepare you to understand how to import and parse CSV using Logstash before being indexed into Elasticsearch.

CIOs see data costs as their greatest logging challenge to overcome, according to this survey we collaborated on with IDC. If you’re running significant production operations, you’re almost certainly generating 100’s of GB of log data every day. Naturally, you’re also monitoring those logs and querying for incident investigations. However, most log data is never queried or analyzed, yet makes up the majority of logging costs.

Running a successful company relies on current and accurate information about the underlying systems. Much of this information is contained within your application logs. By investing in your log management solution, you can unlock these crucial insights and access a wealth of powerful data. This post presents a series of goals that will allow you to make the best possible use of your application logs.