With more and more applications moving to the cloud, an increasing amount of telemetry data (logs, metrics, traces) is being collected, which can help improve application performance, operational efficiencies, and business KPIs. However, analyzing this data is extremely tedious and time consuming given the tremendous amounts of data being generated. Traditional methods of alerting and simple pattern matching (visual or simple searching etc) are not sufficient for IT Operations teams and SREs.

Historically, software development and SRE have worked in silos with different cultural perspectives and priorities. The goal of DevOps is to establish common and complementary practices across software development and operations. Sadly, in some organizations true collaboration is rare and we still have a way to go to build effective DevOps partnerships.

Artificial intelligence for IT Operations (or AIOps) has been playing an expanding role in helping SREs, DevOps, and developers effectively navigate the challenges around application and infrastructure complexity, pace of change, and data volume that characterize the operations landscape.

Elastic Observability provides a full-stack observability solution, by supporting metrics, traces, and logs for applications and infrastructure. In a previous blog, I showed you how to monitor your AWS infrastructure running a three-tier application. Specifically we reviewed metrics ingest and analysis on Elastic Observability for EC2, VPC, ELB, and RDS.

Does your organization’s data include sensitive information, like intellectual property or personally identifiable information (PII)? Do you want to protect your data from being stolen and sent (i.e., exfiltrated) to external web services? If the answer to these questions is yes, then Elastic’s Data Exfiltration Detection package can help you identify when critical enterprise data is being stolen and exfiltrated.

Unlock the full potential of your observability stack with continuous profiling Identifying performance bottlenecks and wasteful computations can be a complex and challenging task, particularly in modern cloud-native environments. As the complexity of cloud-native environments increases, so does the need for effective observability solutions.

In an earlier blog post, Log monitoring and unstructured log data, moving beyond tail -f, we talked about collecting and working with unstructured log data. We learned that it’s very easy to add data to the Elastic Stack. So far the only parsing we did was to extract the timestamp from this data, so older data gets backfilled correctly. We also talked about searching this unstructured data toward the end of the blog.

Elastic Observability 8.6 introduces a set of capabilities improving production operations through the introduction of host (EC2/GCP compute/Azure compute) observability, application dependency operations views (insights into databases, caches, etc), and a new connector for Opsgenie. These new features allow customers to: Elastic Observability 8.6 is available now on Elastic Cloud — the only hosted Elasticsearch offering to include all of the new features in this latest release.

Elastic Enterprise Search 8.6 enables customers to index searchable content on file systems, network drives, MongoDB, and Amazon S3. With new connectors for network drives and Amazon S3, content indexed can easily be transformed for natural language processing (NLP) use cases with intuitive tooling to test and tune your search experience with the trained model of your choice.

One tool for all your Python performance tracking needs We're building this neat service in Python to ingest data in Elasticsearch from various sources (MySQL, Network Drive, AWS, etc.) for Enterprise Search. Sucking data from a third-party service to Elasticsearch is usually an I/O-bound activity. Your code sits on opened sockets and passes data from one end to the other. That's a great use case for an asynchronous application in Python, but it needs to be carefully crafted.