Are You Missing Referral Traffic In Universal Analytics?
Don’t fall for that old Jedi mind trick and simply ignore what Universal Analytics tells you to ignore… they might be the referrals you are looking for.
Did you know that Universal Analytics’ default setting is not to count referrals from your domain? That’s right, Universal Analytics is going to ignore self-referrals by default. This may not be a good thing if you need the information to fix coding errors, but that’s another story.
Today’s story is how to make sure that your idea of self-referrals matches what Universal Analytics is calling a self-referral. If it doesn’t, you may be ignoring some referrals that you didn’t want to ignore. Depending on your situation, you may need to change a setting or even add some custom code to see all the referrals you want.
Read more to understand what’s really going on with referrals in Universal Analytics so you can make an informed decision about what to ignore.
What’s in your referral exclusion list?
When you start a new property, Universal Analytics automatically adds your domain to that property’s referral exclusion list. This list controls which referrals are ignored. Right now you should take a look at what’s in there, if you don’t already know.
Here’s how Universal Analytics decides what to put in your referral exclusion list by default: It takes the domain you entered when you created the property, such as jobs.tatooine.com, and uses only the second-level and top-level parts, so that the default domain in the list would be tatooine.com. If you’re upgrading to Universal, this may not happen automatically, but you should still check out the list to confirm.
Sounds reasonable, right? In fact, this default setting works out fine as long as you want to ignore referrals from every hostname that contains tatooine.com.
If there are third-level or higher domains, the default setting ignores them all: www.tatooine.com, vacations.tatooine.com, as well as jobs.tatooine.com. The default setting also ignores referrals from second-level domains that include tatooine.com, such as visittatooine.com or cantinatatooine.com.
Whoa, whoa, whoa… you may be thinking, “I didn’t want to go that far. I’ll just change my referral exclusion list to specify exactly the domain I want to ignore.” And so you might enter jobs.tatooine.com – if that’s the property you’re collecting data for and you want to see referrals from everywhere else – and think your job is done.
Unfortunately, the result is that now you’re not excluding any referrals because the Referral Exclusion List accepts only the second-level and top-level parts of any domain. If you enter only third-level or higher domains, it’s like entering nothing at all.
Do you need to ignore any referrals?
In the simplest, ideal scenario, your tracking code would be implemented perfectly and your traffic reports would contain nothing or next to nothing in the way of self-referrals. It’s possible you wouldn’t need to enter anything in your referral exclusion list, i.e. you could remove the default entry. Or you might need only the domain of your third-party shopping cart in the list.
Here are a couple of scenarios with related domains where things could be more complicated:
Scenario 1: Ignore one subdomain, but not main domain or other subdomains
Your property collects data for jobs.tatooine.com. You want to ignore referrals only for that hostname, and you want to see referral traffic from everywhere else, including tatooine.com.
Scenario 2: Ignore main domain and one subdomain but not another
Your property collects data for tatooine.com and vacations.tatooine.com. You want to ignore referrals from these two hostnames, because your tracking counts a user who views pages from both as part of the same session. On the other hand, you want to see referral traffic from other hostnames such as jobs.tatooine.com.
What are your options in each case? As mentioned earlier, the default exclusion tatooine.com is going to ignore referrals from every hostname that contains tatooine.com, and entering anything more specific is like entering nothing at all.
Things look pretty hopeless. Maybe someday Universal Analytics will allow you to specify exact hostnames in the referral exclusion list, but until then…
Custom JavaScript to the rescue
Our solution is to come up with our own mind trick. With some custom JavaScript, we’ll make Universal Analytics think, “This is not the hostname you’re looking for.”
What makes this work is Universal Analytics’ Processing Flow for Campaigns and Traffic Sources.
When Universal Analytics is deciding what traffic information to record for a single hit, one of the things it may look at is the document referrer, i.e. the page the user was viewing when he or she clicked to arrive at the current page. Universal Analytics writes the document referrer into its own parameter named documentReferrer (clever, eh?) and then compares that parameter with your referral exclusion list.
Our trick is to set documentReferrer to a different name – one we still recognize, but that will not match the name(s) in the referral exclusion list. This can be anything that we make up, as long as WE know what it means. In this case, I’ll use tatooinetheplanet.com instead of tatooine.com.
Step 1: Keep the default domain in the referral exclusion list.
For either of my two scenarios, I want to have tatooine.com in the referral exclusion list. Remember, this is going to match everything with tatooine.com, so the next step is the tricky part.
Step 2: Conditionally rewrite the document referrer.
In Google Tag Manager, create a custom JavaScript macro named something like {{rewritten referrer}} with the function shown below. Or, if you’re not using Google Tag Manager, include the function in your page source. The function checks to see if the document referrer matches one of the actual domains we want to exclude. (Here’s where we cover specific domains that we cannot include in the referral exclusion list.)
function() {
var theReferrer = document.referrer;
if (theReferrer) {
var myProtocol = theReferrer.match(/(https?:\/\/)/);
var myDomain = theReferrer.match(/\/\/tatooine\.com(.+)/);
var mySubdomain = theReferrer.match(/\/\/(.+)\.tatooine\.com(.+)/);
var myPath = "";
if (mySubdomain) {
if (mySubdomain[2]) myPath = mySubdomain[2];
// exclude referrals from the jobs subdomain
// see referrals from all other subdomains
if (mySubdomain[1]=="jobs") {
return myProtocol[1]+"jobs.tatooine.com"+myPath;
} else {
return myProtocol[1]+mySubdomain[1]+".tatooinetheplanet.com"+myPath;
}
}
// keep the else-if statement to see referrals from the main domain
// remove it to exclude referrals from the main domain
else if (myDomain) {
if (myDomain[1]) myPath = myDomain[1];
return myProtocol[1]+"tatooinetheplanet.com"+myPath;
}
}
return document.referrer;
}
If the referrer matches tatooine.com but does not match our specific domains, then we’ll trick Universal Analytics into thinking it also does not match the domain in the referral exclusion list by rewriting it as something like tatooinetheplanet.com.
Thanks to my colleague Alex Moore for writing the original code, which I’ve slightly modified here.
Step 3: Set the document referrer field to the value returned in Step 2.
Do this in Google Tag Manager by entering your macro as the value for Document Referrer, which you’ll find under Basic Configuration in your Universal Analytics tags. Or write it into your page source between the create and send methods, as described in the documentation for Document Referrer: ga(‘set’,’referrer’,VALUE); …replacing VALUE with the returned value of your function from Step 2.
Step 4: Use a Search & Replace filter to restore the referrer (optional).
Your rewritten document referrer will end up as one of the sources shown in the traffic reports. If you’d rather see the original hostname instead, you can use a Search & Replace filter to reverse engineer the Campaign Source field and put it back to the way it was. For example, search for tatooinetheplanet.com and replace it with tatooine.com.
Have you been ignoring referrals that you didn’t mean to ignore? If so, have you addressed the issue in other ways than the ones described here? Please share in the comments.
