conversion rate can easily lie

How conversion rate can easily lie

Business summary: Conversion rate can be very misleading metric when calculated from sessions instead of users.

Conversion rate has been here since the Web Analytics as such exists. The holy grail of online marketing, every single one of us uses it and hopes that it grows in time. No doubts about its universal usage when evaluating almost any dimension view. Although there are couple of cases when we shouldn't rely on it at all. Let's have a look at one example when it seems that our business is perfectly fine, but slightly different perspective will show that reality is far cry from what we see in standard view.

Traditional view

Imagine we have a hypothetical e-commerce project and we're about to evaluate the whole year. The very common view we all use is to look at conversion rate trend in time. This is how it looks in the most of the websites. It's growing the closer we are to the Christmas.

chart trendline
conversion rates table

So far, so good, everything as we expected. Well, we're still a bit suspicious, right? We want to reassure that the business is growing, so we look at total revenue trendline.

revenue line
revenue table

As clear from the chart we had a huge growth in the end of the year, so we can open a champaign and celebrate.

Slightly different view

What if we only do one change and instead of using sessions we start all the calculations with users. Not that massive switch, right? Every enlightened analyst should use it! Let's have a closer look how a charts will look like. For better illustration I left there also conversion rate from the first chart based on sessions as we all know it, as well as data from previous table.

users and sessions chart
users and sessions table

As obvious we might not be as happy as from previous "reassuring" charts. All we did here is that we just added number that actually makes more sense - User's conversion rate. We have the same volume of sessions and same amount of conversions. Of course we have hypothetical data, but I've seen couple of e-commerce projects where this wasn't far from truth, at all. What basically happened in the end of the year is that we weren't able to convert as many users as in its beginning. Even though, we were able to generate amount of money we expected, it's obvious that something happened in the second part of the year. Let's add one more chart to easier understand the problem.

revenue per user chart
revenue per user table

"Revenue per User" confirms the fact that we didn't just converted less users, but we also dropped in IMHO one of the most important calculated e-commerce metrics. There can be many reasons like worse product inventory, missing additional services, change in pricing policy, different channels mix, not-working new loyalty program and hundreds more.

Combined view

We all know that it's not possible to use user scope conversion rate every time. The good example are traffic channels. It's impossible to divide users by them. So, the solution is if possible to combine the session view with at least one user scope dimension view. It would be a mistake to cut corners just by being lazy to create a custom report to see what's important. As we see in the hypothetical example above it's very easy to get fooled by session scoped conversion rate. Mentioned user scope metrics will give more realistic view on business and help you identify that something might be wrong. User scope analytics definitely isn't just a to-be-cool-analyst myth. The ball is in your court.

Happy user scoping :)

Related posts: