Are Your Data Pipelines Up to Commercial Standards?
In the data business, we often refer to the series of steps or processes used to collect, transform, and analyze data as “pipelines.” As a data scientist, I find this analogy fitting, as my concerns around data closely mirror those most people have with water: Where is it coming from? What’s in it? How can we optimize its quality, quantity, and pressure for its intended use? And, crucially, is it leaking anywhere?