Need for data quality

The media provides many stories of new and developing technologies and applications. Usually not mentioned is the critical role that accurate data has in the value of outputs and the work required to ensure data accuracy. For end users to have trust in the data requires a commitment throughout an enterprise that data is an asset to be managed.

This objective is easy to write and difficult to implement. The quality of data in an organisation is typically not a topic at management and board meetings. The topic therefore remains at user levels, where correcting duplicated, outdated, incomplete and missing data is often considered as part of the job.

The term ‘data’ is a collection of facts (numbers, measurements, observations) and words, that are structured to enable processing by IT. In times past, incorrect data was considered as a cost cost of doing business. However, the extent of eCommerce, implementation of the Industrial Internet of Things (IIoT) and the drive for ‘visibility’, means that data quality should be taking its rightful place.

Data between organisations

Global and domestic supply chains are increasingly required to exchange data and information. At each node (data collection point) in a supply chain, there are transactions within and between organisations. Incorrect data can therefore be uploaded and transmitted at each node, compounding the problem.

Master Data includes relatively static reference data e.g. customer, location and product. With the Master Data is Product Master Data, which identifies the product classification and attributes of an item.

A 2006 study in the US of 91,000 new and changed products found that more than 60 percent of the data entries required correction. These were missing or incorrect product attributes such as the wrong weight, dimensions or sizing; incorrect product classifications; multiple versions of the same product and incorrect conversions of internal descriptions to international standards. Also mistakes in spelling, punctuation and abbreviation.

A 2009 study in the UK by GS1 (the global product identification standards organisation) identified over 80 per cent of transactions between suppliers and retailers had inconsistencies in what should have been identical data. A similar study in 2014,by GS1 in the US, identified about 50% of the data surveyed was inaccurate. Product Master Data accuracy concerning attributes such as weights and dimensions continued to be an issue.

Even in basic Master Data, discrepancies can occur. What Master Data designation for a business – for example, should it be Procter & Gamble, Procter and Gamble or P&G? Calendar dates must be in one format – for October 18, 2021 it can be written as 10/18/2021 or 18/10/2021 or 2021/10/18. Addresses can be correct, but have multiple formats and titles; for example ‘New York City’ can be written as ‘NYC’, ‘New York’ or ‘NY NY’ (which incorporates the State). When values for the same data point are inconsistent, it prevents grouping and summarizing of data into useful information.

But what is an acceptable error rate for data? The six Sigma target for physical items is a defect parts per million (DDPM) rate of 3.4 defects per million opportunities. But, an acceptable range of 97 or more correct records from 100 data points has been stated as more realistic; although defect-free data should be the ultimate goal.

Internal supply chain data

A successful supply chain group relies on Data Quality. But how many have a process to manage this? A US study of process compliance in a business found that the supply chains flowed crossed 117 disconnected document formats written in Access, Excel and Google Analytics. Because problems with data cuts across department boundaries, there is rarely a person in the supply chains group with responsibility for product data quality.

Procurement: At one company, an analysis found that 88 percent of the items from 93 percent of suppliers contained errors. This may not be typical, but illustrates the possible challenge for Procurement professionals. Internally, there are areas for resolution and a process required for resolution by Procurement:

• Suppliers can trade under multiple divisions and businesses, although ultimately owned or controlled by one entity
• A supplier’s name can be entered differently in the buying organisation’s accounts payable files
• Possibly multiple general ledger and cost centre names
• Categories and standard industry (SIC) codes may be used in one part of the organisation, but not others
• Descriptions of the product or service can vary, depending on the user

Operations Planning:, As planners using Enterprise Resources Planning (ERP) and Supply Network Analysis and Planning (SNAP) applications, they rely on accurate input data, spreadsheets and learned experience, which maybe wrong. An example being a large transport organisation where an audit of the planning spreadsheets identified that more than 90 percent of the formulas used contained at least one error.

In organisations that have implemented Sales & Operations Planning (S&OP), the Master Planner, as facilitator for the process, relies on the quality and reliability of the data aggregated from multiple sources within the organization.

Logistics: Multiple sales channels and the reduction expected in delivery lead times, requires accuracy in the data received and transmitted between suppliers and customers. The Data Quality process is required to ensure that product attributes remain accurate throughout the lifecycle of a product.

An area of concern for supply chains is the projected growth of Artificial Intelligence (AI) ‘solutions’. At their heart are algorithms that use data to make predictions and refine models, as new data is generated. Due to the design by developers, AI models can have bias built-in and the model then continues to learn and modify, with minimal intervention once deployed. As a model is unable to differentiate the accuracy of data supplied, calculated predictions and options can be wrong and there is no way that supply chain professionals (or anyone else) can trace back through the iterations to identify errors.

In this new era of more integrated and potentially autonomous systems, Data Quality will be critical for having trust in the outputs from applications that affect competitiveness, revenue and customer relationships. Achieving a substantial improvement in your organisation will require commitment and effort. To start, undertake an audit of processes in your supply chains.