A Better Alternative To Exclude Filters In Google Analytics

I originally wrote this post back in 2014, but the concept and themes have proven valuable over the years, even if the screenshots have slipped out of date. I’ve updated the screenshots and the title. Originally, this post was titled “Say Goodbye to Exclude Filters in Google Analytics!” – which understandably elicited mixed reactions on social media and accusations of click-baiting. Exclude Filters aren’t going away anytime soon, but after reading this post you may not WANT to use Exclude Filters as frequently.  There are still many valid reasons why you may need to set them up, but when possible – it might be time to eliminate them.

Just as a quick refresher, you can use the Exclude Filters in Google Analytics to block traffic data from certain sources from showing up in a particular view.  Sometimes these are used to partition data into one view or another, for example, think of creating separate views for Internal or External Traffic.  For these use cases, Filters work beautifully. You can filter based off of IP Address, Hostname, Service Provider, etc…

But then there are those occasions where you want to block out traffic completely.  Just as easily, you can set up an Exclude filter for each of your views, and poof! The data has disappeared!  Except, it hasn’t really.

The raw, unfiltered data is stored by Google Analytics. Anytime you request a more specific report, either by adding a secondary dimension or applying an Advanced Segment, GA will go back to that unfiltered data at the property level to rerun your report, then filtering the results for your specific view.  Why is this important?

Even filtered data will increase sampling in your reports.

In the free version of Google Analytics, if you try to run a non-standard report that includes more than 500K sessions, Google Analytics will use a sampling algorithm. So all of this raw data you think you’re filtering out is actually going to continue to stick around and cause sampling issues for you. Click here for more information about how sampling works with Google Analytics.

Google Analytics 360 customers have a few secret weapons against sampling, specifically – sampling is calculated at the View level, after filters. However, that doesn’t mean that the hit goes away. Again, it’s just filtered from a particular view. Those hits count towards your hit quota and can affect how much you’re charged or if you’re nearing a particular tier.

Filtered data will count against your hit quota.

In cases where you want to filter out traffic completely, my advice is to stop sending that data to Google Analytics. Consider the difference between source separated recycling or single stream recycling. In source separated, you manually split your recycling before you send it to the recycling plant. In single stream recycling, it all goes together and the recycling plant has a machine that helps sort it for you.

Modified from Better World Betty

The answer here is Google Tag Manager, which is going to help us sort our traffic before it goes to Google Analytics. If you’re not familiar with Google Tag Manager, we’ve got a few articles on our blog that will help get you up to speed. Long answer short, we love it here and it will change your (analytics) life.

Let’s go over how data gets from your website into Google Analytics. In a standard implementation, using Google Tag Manager and Google Analytics, the following order of steps happen.

1. Visitor Reaches Your Page
2. Google Tag Manager Loads
3. GTM Executes a Google Analytics Tag
4. Pageview Is Sent to Google Analytics
5. Google Analytics Receives the Pageview
6. GA Applies Any Filters for Each View against the Pageview
7. Pageview Shows Up In Your Analytics View

Depending on the outcome of Step 6, the Pageviews and Events will or will not show up in your Google Analytics reports. Rather than waiting until the Pageview hits the filter to decide that it’s not what we want, I propose we recreate the Exclude Filters using custom Variables and Triggers in Google Tag Manager.  If we can determine that we don’t want a particular visitor to show up in our Analytics, let’s not even serve them the Google Analytics tag. We’ll use Google Tag Manager as a shield in this situation, to keep that data out of our analytics entirely.

Fancy GTM shield graphic circa 2014.
Get it?!

If there’s a reason you want to see this data in the Google Analytics interface, let’s say from a development site or from internal traffic, once your triggers are created you can always serve it a different Analytics tag or use a dynamic variable to change the UA number, so that it won’t count against your main property’s hit limits or sampling limits.

Note: It will still be helpful to have Exclude Filters on your account as a backup. This blog post will help keep traffic from reaching your account, but it won’t stop traffic from any sites that aren’t loading your Google Tag Manager.

Our new process looks like this:

1. Visitor Reaches Your Page
2. Google Tag Manager Loads
3. GTM Looks at Value on Data Layer
4. GTM Decides Not To Execute a Google Analytics Tag
5. Pageview Is Sent to Google Analytics
6. Google Analytics Receives the Pageview
7. GA Applies Any Filters for Each View against the Pageview
8. Pageview Shows Up In Your Analytics View

Benefits Of Using Google Tag Manager instead of Google Analytics

Setting up View Filters will always have its place, but I encourage complementing View Filters with GTM Triggers for most of my clients and training attendees, for a few reasons. I mentioned above that View Filters won’t help influence your sampling limits or your hit limits.

Additionally, most people aren’t JUST using Google Analytics. By using View Filters, you’re putting logic in place that only exists in one tool – Google Analytics. Often, the reasons we want to exclude traffic from our reporting can apply to the other tags that we have in Google Tag Manager. If we don’t want these people getting our GA Tag, why should they continue to get our Google AdWords tag, our Facebook conversion pixel, our CrazyEgg Tag, etc.

Consider the following:

• Advertising and Remarketing – Surely – we don’t want our employees being targeted by our own advertising. Use these tactics to block the tags on your site that adds visitors to remarketing lists. Spend less and spend more intelligently.
• Surveys/Pop-Ups –  If you’re using a Tag to launch 3rd party surveys or ads, chances are you’re receiving some sort of report about how well they’re performing. It would make sense in this case to block these tags from even firing for Internal Employees. This should help improve your response/clickthrough rate.
• E-Commerce – With your new Variables and Triggers, you can effectively block any test transactions from Internal Traffic from ever getting to Google Analytics.

So let’s get down to business. Two of the more popular Include/Exclude Filters are based on Hostname or on IP Address.

Using Google Tag Manager to Exclude by Hostname

First up, we can create a Variable/Trigger in Tag Manager that we can use to block anything that isn’t our main site. For this example, let’s assume that our site has the following subdomains:

• www.mysite.com or mysite.com
• dev.mysite.com
• admin.mysite.com

We only want the traffic to our main site to be sent into Google Analytics. We can do this through a simple Variable and a new Trigger.

1. Create a Page Hostname Variable

Google Tag Manager has a number of variables that are built-in and just need to be enabled! In this case, just make sure the Page Hostname variable is enabled.

2. Create the Trigger – “Traffic – Bad Hostname”

In Google Analytics, we have the option of using Include or Exclude. We could do the same thing here, but I’ve found that it’s much easier to create this trigger as a standalone trigger and use it to Block tags from firing. If you wanted to, you could add it to all the triggers that control firing, but then you’re duplicating the same logic over and over. I’ve written about this concept here: The Art of the Double Negative – Using Trigger Exceptions in GTM.

I’m calling this Trigger “Traffic – Bad Hostname” as a catchall for any hostname that isn’t www.mysite.com or mysite.com.

1. Head to the Triggers section and click New.
2. Name this Trigger “Traffic – Bad Hostname”
3. Choose Custom Event for the Trigger type, and enter “.*”, and check “Use regex matching.”
   Because this is a blocking trigger, we’re going to block everything from this hostname. More details
4. Choose the “Page Hostname” variable from the drop down list.
5. In the next box, choose the “does not match RegEx (ignore case)” option
6. In the next box, enter the following regular expression. This accounts for the fact that sometimes my site can be loaded with or without the www.

^(www\.)?mysite\.com$

Use that Trigger to as a Trigger Exception For Pageview Tags

Now that we know this traffic isn’t the general public coming to our main site, we can safely block certain tags from firing. In this case, that will be the basic Google Analytics tracking tag.

1. Open the tag you want to block with this Trigger.
2. Under Triggering, hover until you see the Pencil icon, and click. (I click on the word Triggering.)
3. Click “Add Exception”
4. Select the “Traffic – Bad Hostname” Trigger
5. Save!

Using Google Tag Manager to Exclude by IP Address

Now that we’ve set up our Exclude by Hostname capabilities in Tag Manager, let’s look at IP Address. This one is a little harder to implement and will require some server-side logic. By default, IP Addresses aren’t available through JavaScript on a webpage. Depending on the platform your site is running on, this should be fairly easy to implement. You’ll need to look up the appropriate way to get the visitor’s IP address depending on how your site is configured. Once you get that IP address, then it’s time to write it to the dataLayer.

Google Tag Manager utilizes something called the dataLayer to pass information from the page into Tag Manager. Since 2014, we’ve written extensively about the data layer. For more information, check out either of these posts:

• A Developer’s Guide To Implementing The Data Layer
• Unlock the Data Layer: A Non-Developer’s Guide to Google Tag Manager

The idea here is that you collect any sort of information about the visitor, the session, etc… and write it on the dataLayer when the page loads. These are values that your server knows, but that your page does not. When Google Tag Manager loads, it can see this information and using Variables, use this information for Triggers and other Variables.

For the IP Address Exclude, I’ve created a more complicated Variable to allow you to identify multiple types of visitors by their IP addresses. This will let you use these types of visitors in Firing and Blocking Triggers.

Add IP Address to the dataLayer

The dataLayer needs to be loaded ABOVE the Google Tag Manager snippet, so that this data is available when Tag Manager loads. You can use something like the following code:

dataLayer = [{
 'visitorIP': '155.55.155.155'
 }];

Create the Visitor IP Variable

1. Now, back in Google Tag Manager – head to the Variables section and create a new User-Defined Variable
2. Name this Variable “DLV – Visitor IP”
3. Select Data Layer Variable as the type of Variable.
4. In the Data Layer Variable Name, enter whatever you called the IP Address. In my case, this would be visitorIP.

Create the Visitor Type Variable

function() {
 var newIP = '{{DLV - Visitor IP}}';
 //INTERNAL TRAFFIC
 var patt = new RegExp("^28.28.128.1[0-9]?$");
 if(patt.test(newIP) == true) {return('internal');}
 //MONITORING SYSTEM
 patt = new RegExp("^65.65.65.12[0-9]$");
 if(patt.test(newIP) == true) {return('monitoring');}
 return('external');
 }

1. Create a new User-Defined Variable
2. Name this Variable “CU – Visitor Type”
3. Select Custom JavaScript as the type of Variable.
4. Paste in the following code.
5. Alter the code to suit your needs.  Regular expressions will come in handy here, just like in Google Analytics Filters.
6. Save!

Note: I used a Custom JavaScript variable here to enable regular expressions. There a few tools that might be helpful – check out my old post on creating Google Tag Manager Lookup Tables, which also includes a link to a tool created by Seer Interactive that makes this process easier.

Create the Trigger – “Traffic – Not External”

Again, I’ll create a trigger as catch-all for everything that is not our “external” visitors that we want to track. It might be helpful at some point to create special trigger for different visitor types, but in this case, I’ll lump them all together.

1. Create a new Trigger
2. Name this “Traffic – Not External”
3. Choose the “CU – Visitor Type” variable from the drop down list.
4. In the next box, choose the “does not equal”
5. In the next box, enter the word “external”

Use that Trigger to Block Tags from Firing

1. Open the tag you want to block with this Trigger.
2. Under Triggering, hover until you see the Pencil icon, and click. (I click on the word Triggering.)
3. Click “Add Exception”
4. Select the “Traffic – Not External” Trigger
5. Save!

General Conclusions

So there you have it, two ways to use Google Tag Manager to shield your Analytics data from unwanted traffic. I’ve written extensively about blocking out internal traffic, but consider any other decisions you can make inside of Google Tag Manager to determine whether or not to fire tags.

Sometimes you want to see the data going into Google Analytics to make sure everything is working properly. Filter may have helped you create separate views, but Lookup Tables and trigger exceptions can help create a similar setup using a test property with a different UA number for all testing/QA. This way your test data isn’t contributing to increased sampling and you can safely monitor your test cases. You can use these Variables to block one tag from firing, and the trigger another tag to fire instead.

Last note of caution – if you have a high traffic site and you’re constantly hitting sampling, this fix may not do much to help your situation. Consider Google Analytics 360 which increases the amount of visits you can include in your sampled reports, as well as giving you the option to request unsampled custom reports.