Operations | Monitoring | ITSM | DevOps | Cloud

Apache Hive is an open source interface that allows users to query and analyze distributed datasets using SQL commands. Hive compiles SQL commands into an execution plan, which it then runs against your Hadoop deployment. You can customize Hive by using a number of pluggable components (e.g., HDFS and HBase for storage, Spark and MapReduce for execution). With our new integration, you can monitor Hive metrics and logs in context with the rest of your big data infrastructure.

Dashboards provide critical visibility into the performance and health of your environment. But if your organization uses hundreds or thousands of dashboards, or if you’ve recently transitioned to a new company or different team, it’s not always easy to understand the full significance of the data shown on every single dashboard.

Ansible is an automation tool for provisioning, managing, and deploying infrastructure and applications. When building large-scale applications, Ansible enables users to manage and configure their infrastructure across platforms like AWS. Whether you rely on temporary or dedicated hosts, you can use Ansible to create a repeatable process for configuring them with the Datadog Agent.

Apache Ambari is an open source management tool that helps organizations operate Hadoop clusters at scale. Ambari provides a web UI and REST API to help users configure, spin up, and monitor Hadoop clusters with one centralized platform. As your Hadoop deployment grows in size and complexity, you need deep visibility into your clusters as well as the Ambari servers that manage them. If issues arise in Ambari, it can lead to problems in your data pipelines and cripple your ability to manage clusters.

Log management solutions can make it easy to filter, aggregate, and analyze your log data. Whether you leverage JSON format or process your logs in order to extract attributes, you can slice and dice your logs using the information they provide such as timestamp, HTTP status code, or database user. But different technologies and data sources often label similar information differently, making it difficult to aggregate data across multiple sources.

Monitoring backend issues is critical for ensuring that requests are handled in a timely manner, and validating that your services are accessible to users. But if you’re not tracking client-side errors and events to get visibility into the frontend, you won’t have any idea how often these issues prompt users to refresh the page—or worse, abandon your website altogether.

As your application grows in size and complexity, it becomes increasingly difficult to manage the number of logs it generates and the cost of ingesting, processing, and analyzing them. Organizations often have little control over fluctuations in the volume of logs generated—and the resulting costs of collecting them—so they are forced to limit the number of logs generated by their applications, or to pre-filter logs before sending them to their log management platform.

Network Performance Monitoring is currently available in private beta. Request access here. Your applications and infrastructure components rely on one another in an increasingly complex fabric, regardless of whether you run a monolithic application or microservices, and whether you deploy to cloud infrastructure, private data centers, or both.

At Dash 2019, we are excited to share a number of new products and features on the Datadog platform. With the addition of Network Performance Monitoring, Real User Monitoring, support for collecting browser logs, and single-pane-of-glass visibility for serverless environments, Datadog now provides even broader coverage of the modern application stack, from frontend to backend.

Signal Sciences is proud to announce our integration with the Datadog platform. This integration furthers our mission of producing the leading application security offering that empowers operations and development teams to proactively see and respond to web attacks—wherever and however they deploy their apps, APIs, and microservices.