Dealing with Bad Data: the Pitfalls and Opportunities to Manage It

You could say data runs the world. Need to make a decision? What does the data say. Going to invest? What is the data telling you. Starting a new business? I have no idea where to find the data! Data helps to inform good decisions, but what about when the data is not good?

Data is a set of quantitative and qualitative variables – it is facts. Facts inform via information, which is interpreted and evaluated to make decisions. That seems simple enough.

How can data be bad?

Bad data is data that can be inaccurate, incomplete, inconsistent, and duplicated information. It corrupts and disrupts. Bad data can be a part of the ignorance of the process, where data was not requested or was inconsistently gathered.

When you collect data, it should have some meaning to your deliverables. Why are you collecting hat you are collecting? Where is the data going? Do you understand the system that is collating the data?

Lots of questions flying around, but let’s analyse the pitfalls of having bad data and the opportunities available to repair and reengage.

What are the pitfalls of having bad data?

Pitfall 1 – you don’t acknowledge there is a problem

A head in the sand approach does not work with data. You are going to keep gathering bad data and not be able to use it effectively. If a decision was made that impacted the collection of data, that decision may not want to be admitted. People are human, it happens but by covering it up inevitably it will come out eventually.

Pitfall 2 – you don’t fix it

Ok, you had some bad data. Hopefully, you’ve identified where the data issue has come from and know how to fix it. If you haven’t yet worked out where the bad data is coming from, find it and fix it!

If you don’t fix it, you’ll continue to have bad data being collected. Failure to fix it is just bad form and begs the question – why are you collecting the data in the first place? It can’t be useful if you won’t accept it needs to be fixed.

Pitfall 3 – you don’t know how to fix it

This pitfall is an area that may not be entirely within your own control and may need external expertise to support. With the development of APIs and automations data is much easier to gather than ever before but you may not know what you’re collecting, or you could be collecting information you do not and should not need.

Pitfall 4 – you don’t secure your data because you don’t think its valuable

Data leaks should not happen, but they do. According to Security Magazine, a hacker attacks every 39 seconds. They do not care how bad your data is, just what they can get and how much. If you don’t take protection of your data seriously, it doesn’t matter how bad it is, you shouldn’t be collecting it.

If you collect banking information, how secure is the system you store your data? If you do not know, find out right now. More and more emails are being sent advising of hacks and data exploitation.

Any data you collect is valuable, never forget that. It’s valuable to the person who provided it to you. If you don’t collect the right data that’s your responsibility, not the person who gave it to you. If you are collecting someone’s date of birth, be very clear on why. If it’s just because you thought that would be cool data to collect but no plan to use it then that’s a big red flag.

Every piece of data you collect from someone is valuable and keep that in the back of your mind when you make decisions on what data you have, and why you think it’s bad. If you don’t need it, don’t collect it.

What are the opportunities to deal with bad data and make it useable?

Opportunity 1 – you’re being data aware

What is the best way to deal with bad data? Acknowledge it and take steps to rectify the problem. If you forgot to add an essential field to a form and have gaps in your data, add in the field and recognise whether you need to go back to people and ask them to provide the data you are missing.

Being data aware is a big asset. If you don’t know what data you are collecting or what you’re doing with it then ask questions and find out.

Be data smart.

Opportunity 2 – getting a CRM will help you manage your data effectively

By collecting data and copying and pasting it into an excel, you’re on entry level data collection. If you’re using complex systems which are collecting the right information through automations and providing tailored reporting that analyses and guides you on how to use that data then you are a few steps further up on the data collection rung.

If you are a small or medium business you can be data smart by using a CRM or a data system to collect your data into, with a mechanism to understand and interpret it. A CRM, or customer relationship management tool is a great way of helping you manage your data effectively to minimise bad data issues.

You can determine what data is most useful to you, customising the CRM to your requirements. Smart tools like Zapier and Automate.io help you streamline how information gets into your CRM, and also how to get it back out again.

Opportunity 3 – data reporting can be a big asset if you let it

There is big money in data analysis because it’s an area that’s difficult to understand. And, to be very honest, you probably don’t need to know too much about how it works. It can also be a very expensive area to dip into. But, the value of getting good and useful data should be your goal.

Social media tools have some reporting functionality embedded so you can read what the data is telling you about who is looking at your profiles, pages and groups. CRMs will have some reporting as well but be aware many offer reporting as a premium or extra so know what reporting you will get out of any tool you use.

If you’re serious about your data, you may want to hire a specialist to help you understand and use your data to guide your decisions. They’ll be able to help you analyse and articulate what data you have, and also what you may not be collecting that could be of value.

When data collection is tainted – data is deemed bad?

Scandals such as The Facebook Cambridge Analytica make people distrust your reasons why you are collecting data so be very clear on why you are asking for it and what you plan to do with it. In the situation with Cambridge Analytica, data was being collected without knowledge or clear recognition of what data was being collected and what it was being used for. This data was used for psychological profiling to be used for targeted ads and messaging. See Tech Republic for a good analysis of the situation with the data.

CSO provide a good list of the biggest data breaches over the past 10 years. These are companies you may not have thought twice about giving data to your yourself. These sites collect payments, ask for a range of data, and if you have used two or three of them and hackers get your information, they are filling in data gaps you may not even realise they have the ability to define.

Bad data doesn’t have to remain bad if you acknowledge, identify and take steps to become data aware. Any data you collect is valuable so with some thought and planning you can make your data work for you.