Data is Not Enough For UX

In my previous blog post, we covered why it is important to collect both qualitative and quantitative data. 

Let’s assume you followed that advice. Congratulations, you now have the data!  But what do you do with it?

You can’t just collect data and hand it off, otherwise, you risk creating insights (and making poor UX decisions) based on false data.

Take web analytics. This is the good stuff that shows how people are using your site, pages they visit, where they’re from, what device they use. It can help inform decisions to build responsive-first for mobile (which is always a good idea anyway) or highlight places where users are struggling and abandon the site. 

The thing is, you can’t do any of that without first making sure the data is valid.

Quantity Tells One Story

Imagine that you own a chain of bookstores across New England. 

You decide that, as the tourist season is over for the coastal towns where you have shops, you want to make up the shortfall in revenue with more online sales. In an act of brilliance, you hire Phase2 to help. 

One of the first things we’d do is take a deep dive into your existing analytics to see what’s going on with users on the site.

We look at trends across the site for the past year and notice that there was a huge 5-day spike in traffic to the site last October. We ask if you had a big sale, or if anything happened that would’ve caused the spike. When no reason can be identified to explain this behavior, we look deeper. 

Digging through the data, we notice that Japan is the third highest region that your visitors are from; this seems odd because you do not ship internationally. Upon further review, we discover that all these visits originating in Japan coincided with that 5-day spike in site traffic, and that they were from automated bots crawling your site, acting as desktop users.

Given that the spike was caused by invalid traffic, we can omit this from the data set. This not only has an affect on the site traffic numbers, but also on the top geographic regions, pages visited, year-over-year engagement increase, etc.

Quality Insights

Now that our quantitative data is set up correctly, we can start reading through it to discover trends and areas where the qualitative data can lend a hand. All this data tells a story, and reading it from cover-to-cover will produce the best insights.

With the newly corrected data, we now see that you had a higher percentage of mobile visitors to your site than was originally represented. We also see that omitting the false data had an impact on your cart abandonment rate, which represents the set of users who add books to their cart but do not complete the transaction. While this rate is lower than originally thought, it is still evident that the cart abandonment rate has been increasing over the past year, especially on the mobile traffic we can now identify.

When asked about it, you remark that your sales have been a bit lower than usual lately, but the amount of phone orders you’ve been getting has actually increased.

Looking at all these pieces of data together, the insight formed is that there is a problem with your site checkout process. For some reason, your customers feel they can’t use the mobile site to buy their books, and have resorted to alternative purchasing methods or not buying anything at all. Now we can work together and begin to identify solutions to this problem. 

Back to The Real World

While it’s likely you don’t own a chain of bookstores, data best practices still apply. 

It is not enough to just collect the data and say you’re done. You need to ensure that the data has been collected correctly, in order to use it to form valid insights; otherwise, why even collect it at all?

Once your data is ready, you need to look at all elements that can impact the part of the business you’re looking at. When you see the whole picture, you can form the best insights.