One of the major developments in software design and delivery over the last few years has been a movement away from monolith applications towards microservices. One of the sticking points I’ve seen on numerous microservice applications is logging. There are some unique challenges with microservice logging that need to be addressed. In this article, we’ll look at how we can make logging in a microservice as painless as in a monolith.

With so many choices available to us today, choosing log management software that’s just right for us has never been simpler. That is, if you know exactly what it is you are looking for. But for many users, the sheer amount of computer programs that perform the same tasks, and seem so similar(sometimes almost identical) to each other, can quickly become off-putting and confusing.

Every year, we take time out of our everyday activities to join thousands of techies just like us for the biggest event of the season–AWS re:Invent. For one week at the end of November, industry leaders gather in Las Vegas to learn about new technologies from AWS as well as other partners in the cloud, DevOps, and IT spaces. The new technology, innovative ideas, and smart, ambitious people we have met over the past few years have been priceless for all of us at Logz.io.

In the world of agile there’s a demand to solve grey areas throughout the design process at lightning speed. Prototypes help the scrum team test ideas and refine them. Without prototypes, we can’t test ideas until the feature or product has been built which can be a recipe for disaster. It’s like running a marathon without training.

Organizations that handle logging at scale eventually run into the same problem: too many events are being generated, and logging components can’t keep up. Even with persistent queues and other mitigating features enabled, there’s simply not enough of a buffer between log generators and log ingesters to handle the volume of log lines coming in.

Application Performance Monitoring, aka APM, is one of the most common methods used by engineers today to measure the availability, response times and behavior of applications and services. There are a variety of APM solutions in the market but if you’re familiar with the ELK Stack or are a Logz.io user, this article describes using a relatively new open source-based solution — Elastic APM.

Logging is an important topic in software development, especially if you need to analyze bugs and other unexpected events in your production environment. Implementing your logging often seems easy. But as you probably experienced yourself, logging is far more complex than it might seem. That’s why you can find lots of articles about it here on the blog.

Ever since humankind developed the ability to write, much of our progress has been made thanks to recording and using data. In ages long past, notes were made on the production and gathering of resources, the exact number of available soldiers and other important personnel, and were compiled and stored by hand. Because of this documentation method, important information was also prone to being misplaced, lost, or even mishandled.

I hosted a webinar where I covered why logging is important, how to choose a logging provider. And then shared our experience of setting up logging on Kubernetes containers, the Kubernetes logging framework and the logging best practices we’ve implemented internally and supported our customers who run Kubernetes in production.

Last month, we announced Logz.io Security Analytics — a security app built on top of the ELK Stack, offering out-of-the-box security features such as threat intelligence, correlation, and premade integrations and dashboards. In this article, I’d like to show an example of using both the ELK Stack and Logz.io Security Analytics to secure an AWS environment.