Home TECHNOLOGY Data Mesh: An Alternative Model To Fully Exploit Data Potential

Data Mesh: An Alternative Model To Fully Exploit Data Potential

With the digitization of the corporate world, ever more significant amounts of data are being collected. Central solutions are often used for this, which, however, hurt the agility and quality of the data. The data mesh concept offers an alternative. Every device and system generates new data that needs to be evaluated, analyzed, and used, from the online shop to social media to IoT devices. Even within the modern, networked factory, given the wide variety of data and information levels, transparency and automated processes are becoming increasingly important to reduce costs and increase productivity because only those who can react quickly to new developments will also be able to avoid significant disruptions and thus hold their own in the market in the long term.

Data warehouse or data lake technologies have long been considered the solution for storing a company’s data and making it available to all areas. Many companies implemented corresponding centralized platforms – hoping to avoid siloing and get the most excellent possible value from their available data. However, this often turned out to be a fallacy: This is usually due to the central data team, which stands between the many data sources on the one hand and the constantly growing number of data consumers on the other. Although it is intended to integrate all data streams in a central depot, it often does not have sufficient knowledge of the technical content and the required structure.

Practically Structuring Data Volumes And Making Them More Usable

Based on proven software engineering principles, the data mesh concept offers a solution. It combines them with decentralized approaches from modern software development – domain-driven design (DDD), product-centric thinking, self-service infrastructure platforms, and federated governance – and applies them to data. One of the key benefits of the data mesh approach is specialized, domain-centric teams.

They know the data and data sources in their respective domains both technically and commercially and can thus integrate new data or structural changes more quickly and make them available in the company. Since the individual teams act independently, the typical bottlenecks of a central data organization are avoided. As a result, data agility in the company increases, and data users are supplied with relevant data more quickly. This, in turn, increases business agility and shortens the time to value.

The Principles Of The Data Mesh

The definition of Zhamak Dehghani includes the following four principles:

  • Domain ownership
  • Data as a product
  • Self-Service Data Platform
  • Federated governance

Domain ownership means that data management is divided into technical business areas, each of which has a dedicated data team responsible for it. This way, it can respond better to the requirements and integrate new data sources more quickly. Since the team has the necessary expertise, it can react more quickly to changes – for example, when purchasing external data. Following the data-as-a-product principle, domain teams create data products that other teams can easily access.

To do this, the data must be well documented and easy to find, offer easy access, be high-quality, and be based on user requirements. In short: The team develops and maintains its data product like a natural product. The self-service platform describes the underlying IT platform, which should provide uniform, domain-independent tools with which the domain teams can independently create, maintain, and offer their data products. These should be easy to use and require little or no highly specialized knowledge. This is the only way to reduce the costs and complexity of the processes used to create the data products.

In addition, the quality, interchangeability, and integrability, i.e., the interoperability of the data products, must be guaranteed. This ensures that all domains comply with uniform quality standards and that their data products can be used across the board. The four principles must complement each other. Data-as-a-product and federated governance are crucial to preventing domains from becoming data silos. Federated governance also ensures the quality of the data products and defines a uniform set of rules within which the domains can act autonomously but not uncontrolled. The self-service data platform, in turn, must be functionally powerful and easy to use so that the domains can produce their data products independently and cost-effectively.

Tech Cults
Tech Cults is a global technology news platform that provides the trending updates related to the upcoming technology trends, latest business strategies, trending gadgets in the market, latest marketing strategies, telecom sectors, and many other categories.

RECENT POSTS

Hybrid Cloud: What Is It, How Does It Work, And What Are Its Advantages?

You’ve heard of cloud computing, haven’t you? Also called cloud computing, this solution refers to providing computing services, such as storage, software, database, and...

High-Performance And Secure Network For Fast Data Transfer In The Company

Compared to MPLS-based wide area networks, SD-WAN is more flexible, easier to administer, and cost-effective. But how to explain the superiority of SD-WAN compared...

Selecting Project Management Software Using These Tips

There are a variety of software tools available to help project managers see their plans through to completion. Despite this, there is still a...

The Green Digital World – Suggestions For More Sustainable Success

We are in the transition phase to a hyper-connected digital world in which sustainability is a top priority. Companies should take a step away...

How Do I Design My Affiliate Strategy?

Sooner or later, every online shop and every B2C provider gets to the point where affiliate marketing becomes an exciting option. Be it because...

What Is OwnCloud? OwnCloud vs. Nextcloud

In today's world, almost every one of us has many photos, videos, and files. Due to their simplicity, most of them are stored in...