Operations | Monitoring | ITSM | DevOps | Cloud

As well as being a search engine, Elasticsearch is also a powerful analytics engine. However, in order to take full advantage of the near real-time analytics capabilities of Elasticsearch, it is often useful to add structure to your data as it is ingested into Elasticsearch. The reasons for this are explained very well in our schema on write vs. schema on read blog post, and for the remainder of this blog series, when I talk about structuring data, I am referring to schema on write.

Amazon Kinesis Data Firehose is a service for ingesting, processing, and loading data from large, distributed sources such as clickstreams into multiple consumers for storage and real-time analytics. AWS recently launched a new Kinesis feature that allows users to ingest AWS service logs from CloudWatch and stream them directly to a third-party service for further analysis.

Monitoring your system and infrastructure is critical to ensure the performance of your services. In fact, as software development moves faster and faster, alerting and monitoring becomes an indispensable practice for modern DevOps teams. Why is that exactly? That’s what I’m going to discuss today.

As you build more complicated Splunk apps, you might wonder, “What is the best way to make the features in my app more usable?” If you’re adding new SPL commands or creating ways to input new data sources, the answer is straightforward. But imagine you’re trying to address one of the following scenarios: For cases like these, consider extending the Splunk REST API with custom endpoints.

With the ever-growing volume of application logs and analysis tools available, it can be time-consuming to set up your observability tools to keep up with best practices. Every new piece of infrastructure deployed also causes another piece of dashboard and monitoring that needs to be put in place to ensure stability and reliability.

With open source in our roots, we’re always excited about integrations with tools like OpenVAS, a popular open source vulnerability scanner that Greenbone Networks has maintained since 2009. If you’re not currently using OpenVAS, you can find the project here. OpenVAS contains more than 50,000 vulnerability tests with a community constantly updating its feed to adapt to the ever-evolving security landscape.