How to Minimize Knowledge Debt and Make Agile Data Governance an Everyday Practice
Source: Data.World Published: July 8, 2020
Agile Data Governance is the process of creating and improving data assets by iteratively capturing knowledge as data producers and consumers work together so that everyone can benefit. It adapts the deeply proven best practices of Agile and Open software development to data and analytics.
In generating data assets, many companies have accrued what I call “knowledge debt”. That’s when data and analysis isn’t documented, has no metadata, and isn’t comprehensible. We can all understand why the many people tasked with creating data-driven cultures try to pay down this debt with a silver bullet. Yet, a healthy data-driven culture minimizes knowledge debt as part of the process of doing the work. Capturing metadata and documentation in the flow of normal work fuels reproducibility and reuse. Adding roles like data stewards, data product managers, and knowledge scientists makes this process easier because they act as scrum masters or product owners would if they were developing software, but instead they’re building data assets. As with Agile Software Development, Agile Data Governance needs tools that respect—and promote—the agile process and these roles. According to Gartner:
Effective data management and governance are people-driven practices. They require consistent and high-quality interaction between a variety of roles, and these roles have grown more diverse and distributed over time. Maintaining communication and collaboration is even more critical in the current conditions, creating an opportunity for data and analytics teams to add value by furthering the adoption of new types of tools and approaches.