Datadog on Building an Event Storage System
When Datadog introduced its Log Management product, it required a new event data storage platform, as storing logs and events is a completely different problem from storing metrics, which was the first Datadog product.
Over time, Datadog introduced more and more products that needed to store and index multi-kilobyte timeseries “events”, re-using the Event Platform infrastructure from Log Management. The increased use of the Event Platform and the new feature requirements coming from new products started exposing the limitations of the legacy system and the need for a new approach
In this session Ara Pulido, Staff Developer Advocate, will chat with Ryan Worl, Senior Software Engineer, and Guillaume Duranceau, Senior Software Engineer, both part of the Event Platform team, about the evolution of the events stores at Datadog. They’ll discuss the original log storage system and its limitations that led to the design and development of Husky, a complete rewrite of Datadog’s event store.
By the end of the session you will learn what to consider when designing event store systems and how to plan a full migration without downtime for your users.