I will throw caution to the wind and presume you know just how crucial data is right now.
If you don’t, you really should.
It’s the key driver of growth for modern businesses, but you can forget about outsmarting your competitors if you don’t manage it well.
Seriously. It’s that important.
But, don’t take my word for it—let’s take a look at the numbers. Thirty years ago, we were still in what I like to refer to as the ‘filing cabinet phase.’
Those days are gone. Since the advent of the internet, the amount of stored data has exploded. In fact, by 2013, 90% of the world’s data had been created in just two years prior.
By 2025, analysts predict that users will create 463 exabytes of data every day—that’s the same amount of information stored on 212,765,957 DVDs.
The data age is here, but what does that mean for businesses?
In a nutshell, you need to up your data governance game.
And here’s how to do it.
Don’t worry. I won’t make the next part too painful. Data governance is a lot easier to define than you might think.
I’ve done my best to keep this as straightforward as possible, so here we go:
Let’s break that down.
So, let’s recap. I’ve established how important data has become. I’ve explained what data governance is. However, I haven’t told you why it’s so important.
Data is being created and stored at lightning speeds, and with this stockpile of data comes responsibility.
It’s pretty simple. If you’re responsible for any third-party data, you are obligated by law to govern it correctly.
Compliance is one of the critical drivers of data governance, but there are others.
Another significant catalyst is big data management. It’s easy to mix up data governance and data management, but the two terms are different.
Whereas governing big data refers to introducing company-wide policies and processes, management involves enacting them on a day-to-day basis.
The volume of big data within an organization can be overwhelming, and the need to manage it and make it useful is a prime catalyst for data governance programs.
Another important driver is customer satisfaction. Tedious link?
Okay, it sounds like it, but it becomes a lot clearer when you drill a little deeper.
When you govern data efficiently, it’s much easier to share it. If a customer requests a data set—that could be anything from PII or performance data on a particular stock or asset—the quicker they can get access to it, the more satisfied they’ll likely be.
Even the fact that a business can share this information at all is a benefit.
The final key driver we’ll cover in this section is decision-making.
Critical business decisions are better when you make them using quality, governed data.
And this translates directly into a business benefit because efficient decision-making practices lead to growth.
When an organization has access to governed data, it’s far easier to make better judgment calls. With qualified data, businesses can determine what has worked in the past, what hasn’t, and everything in between.
Another vital business benefit is data-driven innovation.
Data accelerates the growth of the world’s most successful companies. When everyone in an organization has easy access to it, it’s possible to develop profitable, innovative strategies across the board.
Ok, so if you’ve got a thinking cap, now might be a good time to put it on.
Traditional data governance models are effective, in-depth, and give data governance managers massive control over data sets. However, you have to be a data expert to get the most out of them. They aren’t straight forward. They’re suitably academic!
If you want to implement these models company-wide and manage them on an ongoing basis, you’re going to have a lot of work on your hands.
It’s necessary to implement traditional strategies over multiple systems and tools, and, by design, they focus on one primary driver: compliance. Consequently, these strategies don’t help much with data literacy, the most significant factor in wide-spread data use.
Traditional governance follows the DAMA framework.
DAMA International has been in the data governance game for over three decades, and they have done some incredible things during that time.
But there is a problem with their framework of governance. Not only is it prescriptive, but at times it’s intrusive too.
Today, deadlines drive data departments, so they can’t usually follow DAMA’s directions to the letter. When intrusive, the DAMA framework creates a bottleneck in the delivery of data projects, hindering growth.
Let’s go ahead and dissect the terminology described in the framework—piece by piece.
In simple terms, data architecture is about identifying the data needs of an enterprise and designing and maintaining the master blueprints required to meet those needs.
These master blueprints enable users to manage data integration, control data assets and align data investments with business strategies.
You require a data architecture group to:
Data architecture groups play a vital role in an organization’s overall data governance strategy. Usually, this group is responsible for all data governance responsibilities in a company. At the very least, they work intimately with data governance team members, like data stewards.
From a governance perspective, data architects are responsible for:
Data modeling and design processes are directly comparable to data architecture. However, where data architecture processes provide an overview of a company’s data management requirements, data modeling, and design are secondary outcomes.
Essentially, data modeling and design involve the production of graphs, diagrams, and other documentation—physical, logical, or conceptual—that demonstrate and communicate a company’s data assets.
Data modeling is a highly complex process and is the responsibility of the data scientists or data engineers who build these data models.
Data governance duties:
Data storage and operation are about maximizing data’s value through optimal design, implementation, and support.
Most organizations have various databases (SQL, No-SQL, data lakes, etc.), maintenance systems, backups, encryption protocols, and various other activities.
Teams responsible for data storage and operation must:
Database administrators (DBA) play a pivotal role in data storage and operations. The DBA role is probably the best-established and most widely adopted data role in the industry.
As a result, database administration practices are the most mature of all data management disciplines.
From a data governance perspective, the following should be accessible:
The process of appraising stored data using fixed acceptance standards to ascertain its quality is called data auditing and validation.
Once an organization decides on its data storage methods, the challenge is to ensure that the data remains secure. When data is stored on-prem, it’s down to dedicated IT professionals to develop security systems that prevent third-party access or alteration.
But this challenge doesn’t end with external threats. Data security protocols should also prevent unauthorized users within an organization from accessing or manipulating prohibited data sets too.
Data security goals:
IT security teams use various tools and techniques like encryption, antivirus software, malware attack prevention, and more to achieve these goals.
Database administrators and IT security teams are usually responsible for managing data security. However, regulatory guidelines make it more difficult for security teams to find all available privacy-focused data assets and govern them appropriately.
Moreover, granting individual users access to specific data creates a backlog that the security team must manage.
Data integration is the process of funneling data between data stores, applications, and organizations. Data integration is the most common process a business needs to initiate to build any data solution, and data engineers the most sought role.
Data engineers are usually responsible for creating and managing these data pipelines.
Data integration goals:
Data integration requirements:
Data exists in many formats. It could be a PDF, text file, JPG, or one of many other document types.
Like MP3 or MP4, everything else can be considered content and storable in any format other than RDBMS (Relational Database Management Systems).
Several steps must be followed, including organizing and categorizing data, developing storage solutions, implementing workflow protocols, editing the data, publishing, and archiving.
Unstructured data requires governance, and here’s why:
Although similar, reference and master data are two separate things.
Master data is the core data within an organization and could be customer data, data referring to inventory or stock, primary analytical data, or something similar. Master data is characterized by how it is stored (on multiple systems) and shared (by numerous members of an organization).
For example, in the retail industry, consolidating customer information is a master data management (MDM) activity.
Reference data, on the other hand, is the set of values used to structure this master data with a focus on shared or common indicators.
For example, in the global stock market, a trader may well be aware of the tickers representing each stock even if they don’t possess any other detailed information about the stock itself.
I bet you didn’t know M stood for Macy’s on the New York Stock Exchange?
Aggregating master data and categorizing reference data is a complex process that requires various tools and techniques.
All of the following are the MDM activities:
All MDM activities should have a governance focus. Without governance, both reference and master data solutions are unable to deliver their full potential.
Business intelligence (BI) refers to organizations’ strategies and technologies to analyze business-critical data, while data warehousing is a pivotal component of BI.
A data warehouse can contain all of a company’s data, current and historical, from numerous sources inputted by multiple users. From here, data analysts are theoretically able to access any data they need to make vital business decisions.
Traditionally, IT teams used an Extract, Transform, and Load process (ETL) to upload and store data in a data warehouse. This way, data is moved in batches and on daily schedules.
But there’s a limit to how much data you can move at once, so in a traditional data governance model, data warehouses often require updating. This method also requires a lot of resources, including CPU, memory, and bandwidth.
Commonly, governance in BI groups is the primary driver for an entire data governance program.
BI groups can enable business acceptance by:
Key objectives include:
Metadata is data in the fine print—the information used to find and categorize information. As well as making data discoverable, you can use metadata to find common relationships between data sets too.
Metadata is intrinsically linked to data quality because the information contained within it gives data provenance. But without a system in place that automatically analyzes metadata and uses it to categorize and qualify this provenance, it’s impossible to get the most from metadata on a large scale.
Metadata management objectives:
The governance team must establish metadata standards and guidelines.
Good data quality improves the overall usage of data and makes data-driven decisions a reality. Consequently, quality data is one of the primary objectives of a data governance program.
It needs to be tagged correctly for provenance, it needs to be stored safely and securely, and it needs to be well-referenced.
Data quality team objectives:
If you follow all of the processes explored in this blog post, you’ll end up with quality data.
Traditionally, data governance programs were so expensive that stakeholders needed a clear justification for the investment.
The return on investment (ROI) of a traditional data governance program is pretty hard to calculate (we’ll get to calculating the ROI on modern examples later), so the maturity model was developed to better communicate the process with sponsors and stakeholders.
These models were conceived in the early-2000s, except IBM’s (2007) and Gartner’s (2008).
So what does the maturity model look like? Here’s IBM’s version:
Level 1: Initial
There is no awareness, lots of silos, and no governance program in place.
Level 2: Managed
An organization begins to realize the importance of data and how it can benefit from it. Companies start seeing data as an asset. Usually, they have purchased a tool has already and implemented it.
Level 3: Defined
Data regulation and management guidelines are better defined, and integration with existing company processes has started, while regulatory rules are refined and made less ambiguous.
Technology is used in a more efficient way to manage data, and data management practices are implemented widely throughout the organization.
Level 4: Quantitatively Managed
At this stage, all projects follow the data governance guidelines and principles, while data models are documented and made available throughout the organization.
Assessable quality goals are set for each project, data process, and maintenance task. The performance of business operations is measured continuously against these objectives.
Level 5: Optimizing
There is a reduction in the cost of data management, and data becomes easier to administer. Operations are streamlined and easier to navigate. Data governance becomes an enterprise-wide effort that improves productivity and efficiency
As we covered (in great detail) earlier on in this blog post, traditional data governance deals with multiple departments and various functions.
Trying to align departments with data sources, not to mention one another, is a massive headache. Traditional data governance approaches don’t provide an easy way to measure the success of a data governance program.
So, it’s often difficult to justify the investment.
Although incredibly complex, traditional data governance is outdated. Today, it cannot achieve the efficiency, cost-effectiveness, and simplicity of modern data governance tools.
What’s required is a centralized, value-driven platform that’s easy to implement and manage.
But what does that look like?
Don’t forget, most traditional data governance practices were developed in the late 1990s due to compliance requirements for the banking industry. Since then, a lot has changed.
Today, data is everywhere, and data governance protocols have had to adapt.
Now, data governance is defined by the level of value it can bring to an organization—especially when quality data is the foundation of this value.
In the early days of data governance, there was a great deal of focus on developing specific data architecture. Now, using modern data governance technologies, architecture diagrams are automatically built using raw data.
To advance data-driven decision making in an organization through trusted insights.
To ensure data compliance across various data privacy laws and internal data policies.
To improve the efficiency and productivity of IT and data teams.
It’s pretty simple. You can encourage data-driven innovation in a company and make better business decisions when data is:
But don’t get ahead of yourself. Before an organization can innovate with its data, there are a few extra steps to take first.
Data literacy is all about education.
It’s the process by which an organization puts in place measures to ensure all data users within that organization receive education to enable them to consume data confidently.
A comprehensive data literacy strategy enables companies to avoid mixed messaging and cross-department confusion.
Look at it like this – having a dedicated data team is a powerful asset. But give everyone in a company the means to locate and utilize this data, and you can transform a business from the inside out.
To build a culture where users can utilize data effectively, the way custodians distribute, store, and manage the data must be transparent.
And transparency leads to trust.
But transparency doesn’t mean making all data in an organization available to everyone—that would make your data almost impossible to govern correctly!
Instead, it’s vital that a company clearly states where and what its data is, where it’s coming from, who is using it, who owns it, and whom to contact if you need access to it.
With technology like this in place, organizations can build trust by displaying the location and content of the data available to users without necessarily giving them access.
Talking of access—once you have a data-literate staff, transparent data sets, and a culture of trust in that data, the next step is to make it accessible.
Without access to the data they need to develop new concepts and approaches, users can’t innovate.
So what’s an ideal scenario?
All users have equal access to the data they require—as long as no restrictions are in place to protect PII or other information. You can achieve this through smart cataloging and classification.
Self-service analysis happens when business users develop into business analysts—we’ll talk about other roles and responsibilities a little later in this section.
After giving users access to data, they can train, experiment, and innovate. Eventually, these users will transform into analysts using the data available to make better business decisions.
In modern data governance, self-service is one of the primary use cases for end-users. The other two are data literacy and data discovery.
The more processes that are put in place to streamline the data governance process and track KPIs, the better the data’s quality becomes.
Organizations need to make a concerted effort to improve their data quality for users to get the best from it. However, you must ask yourself what you want from your data before determining how good it is.
Once a data-literate staff can access and analyze this data, they can determine the specific KPIs required to track its performance.
Compliance is a driving force behind data governance practices today. There are three key areas to consider if you wish to address compliance issues in modern data governance.
Standardizing data is a crucial step to ensuring compliance. When you standardize data, it is easier to track and compare.
A primary compliance requirement is to ensure an organization can confirm user data’s location and security. By standardizing data, analysts can easily categorize and trace it, ensuring it conforms to various regulatory requirements.
Once standardized, data is easier to identify, enabling organizations to classify and tag it.
Understanding data is vital if you want to ensure compliance. You need to know what it is and what it means. Classification is a crucial part of this process.
Data lineage refers to the lifecycle of data—where it comes from and where it’s been
With processes to confirm the lineage of data, it’s possible to see if and how it’s been altered, who it belongs to, and what important information it contains.
From a compliance perspective, data lineage enables organizations to achieve many objectives, including more efficient regulatory reporting, improved data governance through access to historical data, and the ability to expose any discrepancies or potential security threats.
Chief Information Officers (CIO) and Chief Data Officers (CDO) are expected to do more with fewer resources. Fact.
There is a powerful drive to transform existing data models into modern systems, but several fundamental processes are required to achieve this.
Data discovery processes ensure that an organization’s data is easy to find, access, and understand regardless of where it is stored. The best way to achieve this is through a data discovery platform, but how do provisions like these improve efficiency?
Generally, in an organization, data is stored in multiple locations with countless different admittance measures in place. This fact makes it incredibly hard to find and access the data.
With data discovery systems in operation, data is easy to locate because it is searchable. This process slashes the time it takes to find a particular data set.
On top of this, comprehensive categorization makes the data within an organization easier to understand as the requirement to trawl through countless data sets to find specific information disappears.
Finally, when data is discoverable, it can be collaborated on. Data and IT teams can work together to use the data available to them to develop data-driven growth strategies. Simultaneously, business users can access the platform independently without help from these specialized staff members.
Impact analysis, in this context, concerns the processes IT and data teams undertake to determine the impact of data management decisions downstream.
Impact analysis enables these teams to work more efficiently before rolling out a major data management protocol, they can systematically weigh up the pros and cons of any imminent decision.
The first step of impact analysis is to do a business assessment. Using smart tools, both data and IT teams can quickly assess how introducing specific changes will impact profits, workflow, and more. It is a critical BI process, and one that is becoming easier and easier with the advent of AI-driven tools that can quickly analyze the available data.
Of course, it is necessary to have data correctly stored, categorized, and managed first.
Metadata provides context to information enabling users to work with it more effectively.
You must fully understand the data you are using if you want to get the most out of it. That’s why in modern data governance, one of the most important drivers of efficient data analysis is managing this metadata.
Modern metadata management programs provide links to context, cataloging metadata in a way that makes finding these links straightforward and fast.
When managed correctly, metadata makes every aspect of modern data governance more effective. It provides accurate information to calculate impact analysis. It includes detailed user information required for regulatory compliance. It enables advanced users to track data lineage, and it provides analysts with more comparison points.
Now you know what a modern data governance model is, it’s a good time to talk about who uses it.
At the top-end of the scale, the most cutting-edge modern data governance programs allow for progressive implementation, enabling users to develop data governance programs at their own pace.
Progressive data governance allows companies to implement strategies gradually, at a pace they desire. It’s possible to specify the scope of data used, the specific data governance programs you wish to implement, and how users will access these assets through self-service.
Ok, so it’s not easy, but it is possible to calculate the ROI of a data governance program.
What makes it difficult is that the ROI of a data governance program is value-driven, always use-case specific, and not intrinsically tied up with tangible profits—at least not in every circumstance.
To calculate ROI, you have to look at the governance program as a whole, and there are no fancy ROI calculators that can do the job for you.
But fear not, using the three pillars of modern data governance—data-driven decisions, compliance, and efficiency—I can explain the ROI of a modern data governance strategy.
It’s straightforward to calculate the ROI in regards to improved efficiency because you’ll quickly learn how much time you’re saving your data teams, and, of course—the time is money.
Self-service has a significant impact on ROI. When users gain access to platforms that make it easy for them to find and use data independently, the economic impact can be huge.
One report by Forrester included analysis from seven companies that had used a modern data governance tool. Over three years, it found that:
From a compliance perspective, it is difficult to calculate an exact ROI, but it’s easy to work out the savings you could make by not falling foul of regulatory guidelines.
There are lots of data protection laws but let’s look at the ones with the biggest fines attached:
And these aren’t just empty threats. The worst rule-breakers of the EU’s GDPR got hit with the following penalties:
The most challenging ROI to calculate surrounds data-driven innovation. Firstly, this is a company by company consideration, and every single organization will use its data to innovate in different ways.
How much they get from the data is dependent on how much they are willing to invest in data governance strategies.
Secondly, data-driven decisions drive value over the long-term, so it takes at least a year—more likely two to three years—to get to a stage where it’s possible to analyze outcomes.
However, you can split the ROI into two—benefits to business leaders and business users.
As we mentioned earlier in this mega-blog when business leaders make decisions backed by trusted insights, it leads to better outcomes.
And THIS directly affects a company’s top or bottom line.
With business users, even when there is a trusted data delivery platform in place and all the data required to innovate is at a user’s fingertips, it’s difficult to predict when and how innovation will happen.
Over time, teams will build more use cases with the technology available to them. Eventually, there will come a pivot point where this new use case, say a recommendation engine, for example, is rolled out.
Even then, you need to have people use the technology first to find out how popular it is and what the ROI will be.
Here’s a scenario. You own a land plot, and underneath this land, unbeknownst to you, is an untapped reserve of crude oil. You know the value of crude oil, but you don’t know that this great wealth is right below you.
Until you explore it, you will never be able to realize its value.
The first step in your data governance journey is finding the best governance tool for the job.
You’ll need to get a little introspective and figure out what you want to get out of your data governance program. Find out what you need and go with a tool that meets these expectations.
The winning-tool should support most, if not all of your data sources and enable you to realize your key goals—within budget!
Remember, start small. Don’t go overboard and get too excited. Find a tool that can grow at your pace and roll with it.
Hey, at OvalEdge, we are determined to make a business find and work wonders with data that matters. Would you like us to work with you?ask for a demo
Fill the information below to set up a demo.