What the Heck Happened to the MDS?
Not too long ago, the Modern Data Stack (MDS) was hailed as a game-changer in the data engineering world. Its modular approach—featuring tools like Fivetran for ingestion, dbt for transformations, Snowflake for warehousing, and reverse ETL solutions such as Hightouch—promised a revolution. It offered flexibility, scalability, and a chance for organizations to democratize data. But today, the buzz has sorta died down. So, what happened?
From Game-Changer to Table Stakes
The MDS quickly became the go-to architecture for many companies, and in doing so, it transitioned from being modern to simply expected. What was once revolutionary became routine. Tools like Snowflake and dbt are now common place, and many of the stack’s principles have been internalized across the industry.
Yet, alongside its success came its (many) pitfalls, here are just some:
Complexity: The "six-vendor problem" emerged, where organizations needed to integrate have a dozen tools from separate vendors. (We will talk about my bet that the pendulum will start swinging in a bit)
Cost: At the beginning it was a bit cool to brag about your fancy tool stack, but now people know that they each came with a fat price tag as well.
Operational Overhead: Managing multiple contracts, SLAs, and integrations introduced friction that undercut the promised efficiency of the MDS.
The Buzzword Problem
Tristan Handy and Benn Stancil, in their dbt Roundup episode on the MDS, argue that the term "Modern Data Stack" was always more of a marketing construct than a clearly defined standard. It promised the allure of best-in-class modularity but often left organizations struggling to piece together a fragmented ecosystem.
Stancil succinctly summarized the problem: "The Modern Data Stack wasn’t a product—it was a collection of vendors selling different pieces of the puzzle." And while modularity was a strength, it became a liability when too many tools needed to be orchestrated together.
The Pendulum is Swinging: The Shift Towards Consolidation
A key development in the MDS’s evolution is platform consolidation. Organizations began moving away from assembling their stacks tool-by-tool, opting instead for single platforms that could manage multiple layers of the pipeline:
Snowflake: Once primarily a warehouse, Snowflake now offers tools for app-building and more integrated transformation capabilities.
Databricks: Initially focused on machine learning, it has expanded into data warehousing with Databricks SQL, offering an all-in-one lakehouse solution.
These shifts address the core weaknesses of the MDS—cost, complexity, and interoperability—by simplifying vendor relationships and unifying tooling.
The AI Effect
The rise of AI has also stolen the spotlight. Companies are now focused on LLMs, GenAI, and AI-native data architectures that promise insights far beyond traditional analytics. While the MDS once captured the zeitgeist of innovation, it has been overshadowed by this next wave of excitement.
In Handy’s view, "The conversation has shifted." The energy that once fuelled the MDS is now powering AI-driven architectures.
Before we get too deep into AI let’s take a look at some lessons learned from the MDS…
"Modern" Is a Moving Target: The term modern dates pretty poorly. Tools that feel cutting-edge today will probably be legacy tomorrow.
Integration Fatigue Is Real: Too many vendors and tools create operational friction, even if each tool excels individually.
Adaptability Wins: Platforms that evolve to simplify workflows—offering consolidated, efficiency and end-to-end solutions—tend to thrive in the long term.
Culture Matters: No matter how modern your stack is, you still need modern data culture.
So, Is the Modern Data Stack Dead?
Not quite—but it’s definitely evolved. The Modern Data Stack isn’t a failure; it’s a stepping stone. It’s taught the industry valuable lessons about modularity, scalability, and the importance of flexibility. Today, many organizations still use MDS-inspired architectures, albeit with fewer tools and more reliance on integrated platforms.
The data landscape will continue to change, whether through the rise of AI-native stacks, further platform consolidation, or entirely new paradigms. But the spirit of the Modern Data Stack—solving hard problems with creative, scalable solutions—remains alive and well.
So, what happened to the MDS? It matured, and in doing so, it paved the way for whatever comes next. Whether that’s called the "Next-Gen Data Stack" or something else, one thing is clear: data engineering isn’t slowing down anytime soon.