Logs are an important part of troubleshooting and it’s critical to have them when you need them. When it comes to logging, Google Kubernetes Engine (GKE) is integrated with Google Cloud’s Logging service. But perhaps you’ve never investigated your GKE logs, or Cloud Logging? Here’s an overview of how logging works in GKE, and how to configure, find, and interact effectively with the GKE logs stored in Cloud Logging.
Editor’s note: This is a follow up to a recent post on how to use Cloud Logging with containerized applications running in Google Kubernetes Engine. In this post, we’ll focus on how DevOps teams can use Cloud Monitoring and Logging to find issues quickly. Running containerized apps on Google Kubernetes Engine (GKE) is a way for a DevOps team to focus on developing apps, rather than on the operational tasks required to run a secure, scalable and highly available Kubernetes cluster.
Whether you’re a developer debugging an application or on the DevOps team monitoring applications across several production clusters, logs are the lifeblood of the IT organization. And if you run on top of Google Kubernetes Engine (GKE), you can use Cloud Logging, one of the many services integrated into GKE, to find that useful information. Cloud Logging, and its companion tool Cloud Monitoring, are full featured products that are both deeply integrated into GKE.
We’ve heard from our customers that you need visibility into metrics and logs from Google Cloud, other clouds, and on-prem in one place. Google Cloud has partnered with Blue Medora to bring you a single solution to save time and money in managing your logs in a single place. Google Cloud’s operations management suite gives you the same scalable core platform that powers all internal and Google Cloud observability.
Monitoring your cloud infrastructure is an essential part of making sure your operations are running smoothly. Since announcing the new Cloud Logging interface in February, we’ve heard from users that the new interface is making it faster and easier to meet logging needs, including troubleshooting issues, verifying deployments, and ensuring compliance. One of those users, Arne Claus, is a site reliability engineer at trivago, and has taken advantage of the new interface already.
Data pipelines provide the ability to operate on streams of real-time data and process large data volumes. Monitoring data pipelines can present a challenge because many of the important metrics are unique. For example, with data pipelines, you need to understand the throughput of the pipeline, how long it takes data to flow through it and whether your data pipeline is resource-constrained.
Using dashboards in Cloud Monitoring makes it easy for you to track important system metrics. Creating dashboards by hand in the Monitoring UI can be a time-consuming process, especially if you want to use them in multiple different Monitoring Workspaces. With the recent GA announcement for the Cloud Monitoring dashboards API, you now have a way to programmatically create dashboards.
During my career (in technology), I have dealt with many clients to whom security was one of the main areas of concern. As such, there’s always room for improvement but without a shed of a doubt, communications direction and stateful firewalls are some of the very first elements to consider. When it comes to logging and audit information, as a rule of thumb, it’s good to have a log aggregator stored outside of the scope of a cloud provider. A great log correlation out there is Splunk.
Our suite of operations products has come a long way since the acquisition of Stackdriver back in 2014. The suite has constantly evolved with significant new capabilities since then, and today we reach another important milestone with complete integration into the Google Cloud Console. We’re now saying goodbye to the Stackdriver brand, and announcing an operations suite of products, which includes Cloud Logging, Cloud Monitoring, Cloud Trace, Cloud Debugger, and Cloud Profiler.
One of the main benefits of using an all-in-one observability suite like Stackdriver is that it provides all of the capabilities you may need. Specifically, your metrics, traces, and logs are all in one place, and with the GA release of Monitoring in the Cloud Console, that’s more true than ever before. However, for the most part, each of these data elements are still mostly independent, and I wanted to attempt to try to unify two of them — traces and logs.
