Soumya Samuel – Solution Architect | Sanjay Sreenivas – Solution Architect |
Pramod Padmaprabhaiah – VP of Global Delivery
November 14, 2022
Executive Summary
An organization is entitled to identify, define, model, and standardize its data as part of a data management framework. However, defining industry-specific and standardized data dictionaries can be an excruciating task. Many organizations hesitate to establish a good data management program without a good business case and, in turn, lack support from leadership. A clean and standardized data template can jump-start a data management and governance program and lay the foundations for an enterprise data architecture.
Information Asset has assembled a retail industry-specific template with Azure Purview. The template includes a standardized set of data definitions, regulations, and policies to kick-start a data governance program with Azure Purview. The retail Industry template can be enhanced and molded per an organization’s requirements to build a solid foundation. The retail Industry template is easy to adapt, customize, and implement.
Along with the retail industry-specific template, a common theme we have heard from many of our customers is an interest in having Azure Purview co-exist with other data governance platforms. We have developed an integration layer for data management platforms (Meta-mesh), which harmonizes information across various platforms, including Azure Purview and Collibra. This solution is built on Azure and can integrate other enterprise-wide data sources.
Inventory of Retail Business Terms
We have built an inventory with over 100 retail glossary terms listed, defined, and enriched with attributes to carry out data governance in Azure Purview. The glossary is described in business terms with a standard format for ease of understanding, whether the consumer of this data is from a business or technical background. Critical Data Elements (CDE) and Personally Identifiable Information (PII) are identified and tagged accordingly within the glossary (see Figure 1).
Data Quality Rules
The template includes a set of data quality rules for completeness and validity. You can extend this template to cover rules to measure accuracy, uniqueness, integrity, and consistency (see Figure 2).
Figure 3 illustrates the association of the data quality rule to the business term.
Inventory of Governance Policies
The template includes an inventory of the policies per the standard data governance framework. These policies are categorized based on specific domains like data quality and stewardship (see Figure 4).
Figure 5 illustrates the association of a policy with a glossary term.
Inventory of California Consumer Privacy Act Regulations
The most critical aspect of data governance is to ensure compliance with relevant regulations. Retail industry templates have defined around 100 privacy clauses for the California Consumer Privacy Act (CCPA) (see Figure 6).
Figure 7 illustrates the association of the regulation to the glossary terms.
Metadata Exchange Platform (Meta-mesh)
Information Asset has built a Metadata Exchange Platform, an integration layer for data management platforms. This platform is built on Azure and uses the Azure Event hub to seamlessly move metadata from one platform to another. This platform integrates Azure Purview and Collibra Data Governance, providing a unidirectional flow of metadata such as a glossary, data quality rules, Policies, and regulations from Collibra to Purview. This solution is available as a single-app service in Azure. The meta-mesh platform is flexible to use the Kafka data pipeline should an organization choose. Near real-time visualization of data in these pipelines is achievable using Stream Analytics on Azure or integrating third-party tools if an organization chooses to track and report the metadata quality moving between data management platforms.
Figure 8 shows the approved glossary terms in Collibra.
Figure 9 shows the same glossary terms migrated to Azure Purview using Meta-mesh. The platform provides the ability to map OOTB and customer attributes and relationships in Collibra to Purview.
