Archived Data Cleansing and Duplicate Information in the Healthcare Industry

binary stream

The upcoming warm weather brings with it spring cleaning initiatives across walks of life. For healthcare organizations, spring cleaning should include its vast stores of data. According to AHIMA, only 20% of all healthcare data sits in automated and managed systems, leaving the other 80% of healthcare information in an unstructured, unknown state that may or may not even be relevant to the organization anymore.

Healthcare organizations should be looking at this 80% of unstructured data if they want to make strides in increased efficiency. This efficiency can take the form of either reduced latency or better data access, as well as the reduction of the amount of overall data that needs to be stored and secured.

Developing a Data Cleansing Plan

Developing a data cleansing plan starts by identifying where your data is, what it is for, and any legal or regulatory retention concerns. This will also be the most difficult step, one for which you will need input from all aspects of your organization. A typical Champion data cleansing engagement sees up to 50% of the project time spent on just this one task. Once this is complete, the second part of this process will involve seeking data quality errors and data that is out of date or incorrect.

Once these two points are identified, you will be better able to gauge the scope of your data cleansing effort. It will be critical that communications across all departments remain open and concise to prevent any data management mishaps.

Two of Everything

A common find during any spring cleaning is discovery item redundancy: having multiples of one thing or many things. For example, the patient management document that everyone in Accounts Payable saved to differing folders, when one copy would suffice, is a prime example of how data can quickly grow due to unnecessary duplication.

Getting rid of item redundancy will require you to work with all business stakeholders within your organization, but can be made much easier thanks to data de-duplication tools like IBM’s ProtecTIER solution. Not only will this save your team and business valuable resources in terms of time spent, but will act as a great preventative measure to add to your overall data management policy.

How Solid is Your Data?

As your data cleansing project progresses, it is important to create monitoring and validation procedures to ensure that any data that is impacted by your efforts remains accurate and secure. While this will vary from organization to organization, having a system of concise and standard data validation checks is vital for project success.

This can mean anything from making sure files and folders retain expected file sizes and names to performing encryption validation checks against encryption hashes. Data validation will be especially important when consolidating or merging data into other systems. Utilizing a good data cleansing tool, such as IBM’s InfoSphere or Watson Analytics, will automate the process to again save valuable staff time while providing further methods to proactively manage your data.

Managing Data Post Project

This last point is especially important in terms of keeping your data organized and up to date. After all of your data cleansing efforts, the last thing you want to see is your stored data getting out of control again. One of the steps we like to add to a data cleansing project is creating a data maintenance plan.   

Businesses that put significant effort into a data cleansing project should also have a maintenance plan established and live prior to any additional project efforts.  

Driving Towards a Successful House Cleaning

Our data engineers work closely with IT and business teams within the healthcare industry to lead successful data cleansing efforts which center around the removal of duplicated and erroneous data while implementing automation tools and procedural changes to leave your data warehouse clean for quarters to come. Reach out to our experts today to begin you data spring cleaning efforts.


How To Make HIPAA and Windows 10 Play Nice During Your Healthcare Organization’s Data Migration


Readying Your Organization for the FDA’s Medical Device Security Recommendations