Data Cleansing and Data Quality: A Primer

by Jessica Banks Marketing Professional


Duplicate data can lead to huge headaches in an organization. Similarly, corrupt or incorrect data also leads to issues in the day to day functioning of an organization. A process that detects and removes these records is necessary, and such a process is called data cleansing. Data cleansing is also called data cleaning or data scrubbing.

Data cleansing does not simply mean clearing out old data to make space for new data. That is called data purging. The point of data cleansing is to ensure maximum accuracy of data in the system. Errors arise due to user entry errors, corruption in transmission or storage and use of different standards in the same organization.

The process itself usually involves removing typographical errors by checking against a known list of values. The process can be fine-tuned to be as tight or as loose as the user wants.

Data auditing is the first step of data cleansing. Statistical and database methods are used to record the characteristics of data and any anomalies present. Checks are made with the help of constraints specified by the user. The second process is called workflow, where the anomalies and errors are removed. Cause of these anomalies have to be considered. This process is essential for high quality data. Workflow execution is the process of implementing the workflow. Post processing is the last step, where the results are inspected keenly to verify how well the workflow has performed. This entire process is repeated as often as is necessary for data cleansing.

Data quality is another aspect that has to be considered. The name is self-explanatory, and organizations have to ensure that the data in their databases is of consistently high quality. There is a set of criteria that data has to pass through for it to be considered high quality. Among other things, this involves validation, accuracy, decleansing, completeness, consistency and uniformity.

Data governance is the process of creating a simple procedure to retrieve and store data. Data governance makes ensuring data quality simple. One leads to another, and regular and prompt data governance also helps with data cleansing.

Verdantis can help with data quality management. Verdantis Harmonize is an extremely configurable and easy to use solution to manage and ensure quality of data. It uses clustering algorithms and fuzzy logic that helps you process thousands of records in a matter of hours. It involves minimal training to master.



Jessica is one of the most passionate marketing professionals in Verdantis. She is a strong proponent of Data Quality Improvement for large enterprises. For her, data drives performance.

Sponsor Ads

• Ad Space Available - Rent it Now!

Created on Dec 31st 1969 18:00. Viewed 0 times.

Comments