Sometimes a level of specialised data cleaning is needed that cannot be achieved by automated enhancement techniques

How our approach works

Designed for each client’s data challenges

Our approach begins with combining the organisation’s experience of its data quality challenges with our own more formal data quality assessment. From these we develop the enhancement regime. This typically includes a number of our standard cleaning elements plus other, client-specific elements.

Driven by business rules wherever possible

As much of the enhancement regime as possible is documented in advance, in business rules that ensure complete clarity about most of what will happen to the data. They will normally be supplemented by human judgement but always form the basis of the core enhancement activity.

Combines computer speed with human judgement

Everything that can be reliably programmed into database queries and routines is carried out early in the process. Even at this stage exception management puts any suspect situations in front of human eyes for a judgement. Later stages invariably involve a far higher level of human involvement.

Application of carefully controlled common sense

Common sense is applied to the enhancement activity to a degree that has been agreed with the client. This may be limited to the correction of common mis-spellings and expansion of an agreed list of abbreviations. It can also be extended to making judgements about suspected duplicate records and identifying industry-specific errors.

Supported by research where needed

The cleaning routines can be enhanced to include on-line or telephone research if required. This can involve checking organisation websites and company records to understand corporate hierarchies or collaborating on contacting individual customers to confirm and update their details.

Learning can be fed into ongoing data capture and loading

As well as delivering the enhanced data itself our approach can document all the business rules used and the learning from any manual interventions that have been frequently applied. This can form the basis of a specification to the IT Team supporting the ongoing loading of data to prevent future problems occurring. It can also drive the development of a data capture guide for colleagues who collect and enter customer data.