Poor data quality is a problem many organizations face, but few will acknowledge. However, this head-in-the-sand attitude only allows data issues to grow and become increasingly complex as more data enters the organization from a rising number of sources. Ignoring data quality problems can lead to significant waste, errors, and re-work, which can eventually become impossible to ignore. Overcoming the denial stage is the first step in a journey towards improving data quality and reaching business goals.
Causes of Poor Data Quality
Many factors contribute to poor data quality, making it easy for company executives to overlook. Data entry errors caused by employees, customers, partners, or vendors can doom the data right from the beginning. Companies that do not adequately monitor data input or neglect to use tools to normalize or correct data as it enters the organization are likely to have downstream remediation issues. A good example of how to prevent bad data is a postal address validation program that provides real-time corrections via an API as employees or customers enter the data.
Another source of data quality problems are inconsistent data definitions across applications which lead to multiple data formats, each tuned to serve a specific purpose. Later projects to re-purpose data from one system to another can multiply the impact of bad data or prevent data from being loaded into some databases.
Data decay is another common issue, such as with customer postal addresses, where about 15% of the US consumer population moves every year. Addresses may get corrected in extracted mailing files, but the new address data rarely makes it back to the source database. Uncorrected address information can also prevent attempts to merge files, which is a necessary step towards building consolidated customer views and personalized communications. Sales volumes, product purchases, and email addresses are just a few other examples of data that can also decay.
Poor Data Quality Symptoms
So, how do you know if you have a data quality problem? Here are common symptoms of data quality issues:
· Employees inventing home-grown workarounds to deal with data inconsistencies.
· Reports generated from different systems or divisions that do not match as they should.
· An army of staffers going over reports and charts and “fixing” them before passing them along to media, investors, or company executives.
· Having to reject requests for information or making purposely vague statements because you don’t trust the data.
Evaluate the Impact of Bad Data
Once an organization recognizes it has a data quality problem, it should determine the cost of operating with bad data. Data quality is not just about the wasteful practices of paying for duplicate mail pieces, excess data storage, or confined to personalization errors. A piece of data might support many actions. Flaws that might be inconsequential in one case can severely impact outcomes in another. Unrecognized data quality issues can adversely affect business operations, such as staffing decisions, site location selection, money management strategies, and inventory administration.
Fixing Data Quality Problems
Once you have acknowledged the need to fix your data quality issues and have performed an assessment, it's time to develop a plan of action. Here are the steps to take:
1. Executive sponsorship: Secure executive buy-in and support to ensure the success of your data quality initiative. This involves budget approvals, setting goals, and aligning your data quality efforts with overall business objectives.
2. Define data governance policies: A data governance program is a framework that provides the structure, policies, and procedures to ensure organizations manage and use data effectively. This includes defining roles and responsibilities, establishing data management policies, and providing the tools and resources necessary to carry out those policies. A data governance program also helps make sure that data quality is a priority, and the company allocates the resources to improve the quality of their data.
3. Develop a data quality plan: Develop a comprehensive data quality plan that outlines the steps to be taken to improve data quality. Include the timeline for each step and the resources required. This plan should include data profiling, data cleansing, data standardization, data enrichment, and data governance.
4. Implement data quality tools: Install data quality tools that can automate data quality processes and provide data quality metrics and reporting. Tools that offer functions such as data profiling, data validation, and data reconciliation can help you identify and resolve data quality issues.
5. Data quality education and training: Supply data quality education and training to employees, including best practices for data entry and data management. This will help reduce data entry errors, improve data accuracy, and ensure data is properly maintained.
6. Continuous improvement: Establish a continuous improvement process that monitors and reports on data quality, provides ongoing training and support, and regularly assesses the effectiveness of your data quality efforts.
Denying the existence of a data quality problem creates a false sense of security. Organizations must acknowledge the issue and take steps to address data quality to make sure their data supports the organization’s mission and overall business objectives. Executive sponsorship, clear data governance policies, a comprehensive data quality plan, implementing data quality tools, and continuous improvement are key elements of successful data quality implementation. By investing in data quality, organizations can improve the accuracy of their data, reduce waste and errors, and make better data-driven decisions that drive business success.
Ken Kucera is the managing principal of Firstlogic Solutions, delivering world-class address and data quality software to data-driven companies across the USA. With 40 years of industry experience, Ken leads the team that innovates and delivers address correction, data cleansing, data enhancement, and data matching/consolidation software at Firstlogic. Reach Ken at firstlogic.com or follow him on LinkedIn.
This article originally appeared in the July/August, 2023 issue of Mailing Systems Technology.
