A sequel to our first post, Automating the installation of Elastic Cloud Enterprise with Ansible, this blog shows how to extend automation to cloud provisioning with Terraform. In the first post, we detailed how to deploy and configure Elastic Cloud Enterprise (ECE) across three availability zones in AWS using Ansible. However, the provisioning of the underlying EC2 instances and configuration of the security groups was all manual.

The Elastic Common Schema (ECS) defines a common set of fields for ingesting data into Elasticsearch. A common schema helps you correlate data from sources like logs and metrics or IT operations analytics and security analytics. Further information on ECS can be found in the official Elastic documentation, GitHub repository, or the Introducing Elastic Common Schema article.

Elastic Cloud on Kubernetes (ECK) is an operator that allows you to automate the deployment of the Elastic Stack — including Elasticsearch, Kibana, and Elastic APM, Elastic SIEM, and more — using Kubernetes. By using this ECK, you can quickly and easily deploy Elasticsearch clusters with Kubernetes, as well as secure and upgrade your Elasticsearch clusters. It is the only official Elasticsearch operator.

Logs from a variety of different AWS services can be stored in S3 buckets, like S3 server access logs, ELB access logs, CloudWatch logs, and VPC flow logs. S3 server access logs, for example, provide detailed records for the requests that are made to a bucket. This is very useful information, but unfortunately, AWS creates multiple .txt files for multiple operations, making it difficult to see exactly what operations are recorded in the log files without opening every single .txt file separately.

Elasticsearch and the rest of the Elastic Stack are commonly used for log and metric aggregation in various environments, including Kubernetes. In addition, the Elastic Stack is frequently being used for uptime tracking, with Heartbeat, as well as Application Performance Monitoring (APM), with agents supporting common programming languages, including Java.

Elasticsearch cross-cluster replication (CCR) was released as a beta feature in Elasticsearch 6.5, and as a Generally Available (GA) feature in Elasticsearch 6.7. CCR allows multiple indices to be replicated to one or more Elasticsearch clusters. Replicating indices to additional Elasticsearch clusters solves several use cases, including high availability (HA) across datacenters, disaster recovery (DR), and CDN-like architectures to co-locate data closer to application servers (and users).

Hey, there. This is part six of the Elastic SIEM for home and small business blog series. If you haven’t read the first, second, and third blogs, you may want to before going any further. In the Getting started blog, we created our Elasticsearch Service deployment and started collecting data from one of our computers using Winlogbeat. In the Securing cluster access blog, we secured access to our cluster by restricting privileges for users and Beats.

When making investments in our tech stack, we tend to have doubts about companies that don’t use their own products and services. At Elastic, we deploy the full suite of our technology across the enterprise. We do so because our technology not only works, but it makes us more efficient and flexible on so many levels. And it can do the same for you and your business, too.

Starting April 21, 2020, all requests to Elasticsearch Service on Elastic Cloud must use HTTP over TLS (HTTPS) with support for TLS 1.2. We’ve decided to make this change in the best interest of our users so we can ensure the security of data in transit and stay up to date with modern encryption, security protocols, and practices.

Hey, there. This is part five of the Elastic SIEM for home and small business blog series. If you haven’t read the first, second, and third blogs, you may want to before going any further. In the Getting started blog, we created our Elasticsearch Service deployment and started collecting data from one of our computers using Winlogbeat. In the Securing cluster access blog, we secured access to our cluster by restricting privileges for users and Beats.

In the previous post, we covered some of the frameworks accessible by kernel extensions that provide information about file system, process, and network events. These frameworks included the Mandatory Access Control Framework, the KAuth framework, and the IP/socket filter frameworks. In this post, we will go into the various tips and tricks that can be used in order to obtain even more information regarding system events.

This year at BSidesDFW, my local security conference, I highlighted a continuing trend of adversaries using open source offensive tools. The talk reviewed one of these post-exploitation frameworks named Koadic and walked through different ways defenders can build behavioral detections through the use of Event Query Language (EQL).

We're happy to announce an initial alpha release of a new Rust client for Elasticsearch! You can find it on crates.io with the crate name elasticsearch and dive into the documentation to get started.

As part of Elasticsearch 7.5.0, we introduced a couple of ways to control the index age math that’s used by index lifecycle management (ILM) for phase timings calculations using the origination_date index lifecycle settings. This means you can now tell Elasticsearch how old your data is, which is pretty handy if you’re indexing data that’s older than today-days-old.

KeyBank is one of the largest banks in the United States. And as the bank has grown, so has their end-to-end monitoring system. With more than 1,100 branches and 1,400 ATMs stretching across 15 states, KeyBank’s infrastructure had evolved into a “Noah’s Ark of design,” says Mick Miller, Senior Product Manager, Cloud Native at KeyBank. In other words, they had two of everything, resulting in 21 different data islands.

Curious about how to write more idiomatic concurrent code in Go? It’s not always easy or intuitive, even if you’ve done lots of concurrent programming in other languages. I’ve been lucky to have worked in a well-written code base, and had the expert advice of Beats core area lead Steffen Siering along the way. In this post I’ll walk you through how we implemented a new scheduler for Heartbeat that is part of our upcoming 7.6.0 release.

When it comes to having visibility and detecting threats on macOS, one of the best sources of information for file system events, process events, and network events is the kernel. MacOS kernel extensions provide the ability to receive data about these events in real time with great detail. This is good for providing quick visibility into detecting anomalies and identifying possible threats.