Over the past few years, Data Mesh has been a hot topic in the world of data engineering, promising decentralisation, autonomy, and domain-driven ownership.
However, recent conversations with data leaders suggest a shift in the opposite direction—many teams are moving back to a centralised data model, favouring Data Lakes and Data Warehouses.
Why Are Companies Reverting to Centralised Models?
Several key challenges with Data Mesh are prompting organisations to reconsider their approach:
Scalability of Self-Service Data Products
- Building scalable, self-service data products for the entire organisation is proving more manageable in a centralised model, where data teams can work together rather than in silos.
Developing a Semantic Layer
- A centralised model enables a unified semantic layer, making it easier for non-technical users to access and interpret data without domain-specific barriers.
Consistency in Data Governance & Cataloguing
- Ensuring consistent governance and metadata management across multiple decentralised teams is a challenge. A central repository allows for more structured and standardised cataloguing.
Simplifying AI & GenAI Implementation
- With AI and GenAI becoming more prevalent, having a single data repository makes it significantly easier to implement machine learning models without fragmented data sources.
Was Data Mesh Over-Sold?
While Data Mesh introduced valuable principles around domain ownership and federated governance, some argue it was presented as a universal solution rather than a framework suited to specific organisational structures. A few key points raised by data leaders include:
- End-to-End Use Cases Are Harder in a Decentralised Model
- Many organisations struggle with cross-domain data use cases, which are significantly more complex in a decentralised structure.
- Data Mesh Doesn’t Solve Integration Challenges
- Some argue that while Data Mesh decentralises ownership, it doesn’t inherently make data integration easier—if anything, it adds more friction when attempting to unify datasets.
- Companies Confused Architecture with Organisation
- Data Mesh is not about decentralising data storage, but rather decentralising data ownership. Some organisations misunderstood this and ended up with highly fragmented systems.
The Future: Centralised, Decentralised, or Hybrid?
While some are returning to centralised models, others suggest a hybrid approach—leveraging Data Mesh principles for ownership and accountability while maintaining a central data repository for consistency and AI scalability.
This shift raises important questions:
Are you seeing the same trend in your organisation?
How do you structure your data teams—Data Mesh, Centralised, or a Hybrid approach?