Organizations that handle logging at scale eventually run into the same problem: too many events are being generated, and logging components can’t keep up. Even with persistent queues and other mitigating features enabled, there’s simply not enough of a buffer between log generators and log ingesters to handle the volume of log lines coming in.

Application Performance Monitoring, aka APM, is one of the most common methods used by engineers today to measure the availability, response times and behavior of applications and services. There are a variety of APM solutions in the market but if you’re familiar with the ELK Stack or are a Logz.io user, this article describes using a relatively new open source-based solution — Elastic APM.

Logging is an important topic in software development, especially if you need to analyze bugs and other unexpected events in your production environment. Implementing your logging often seems easy. But as you probably experienced yourself, logging is far more complex than it might seem. That’s why you can find lots of articles about it here on the blog.

Ever since humankind developed the ability to write, much of our progress has been made thanks to recording and using data. In ages long past, notes were made on the production and gathering of resources, the exact number of available soldiers and other important personnel, and were compiled and stored by hand. Because of this documentation method, important information was also prone to being misplaced, lost, or even mishandled.