BLOG

How to Filter Out Fake Referrals and Other Google Analytics Spam

Published January 23, 2015
If you work with Google Analytics, chances are you’ve run into some of these websites in your Referrals report (Acquisition > All Traffic > Referrals) lately:
  • semalt.semalt.com
  • buttons-for-website.com
  • forum.topic31342700.darodar.com
  • make-money-online.7makemoneyonline.com
  • anticrawler.org
What are these sites and why are they linking to your site? Well, actually, they’re not linking to you at all. These sites represent fake referrals. They are created in your Google Analytics account to trick you into visiting spammy websites. If you open one of these URLs in your browser, you will likely be redirected to an online store, marketing scam or malware site. Nice, right?
Welcome to the world of Google Analytics spam, where spammers mess with your Google Analytics data to drive traffic to shady websites!
In this post, we look at the impact these spam sites have on your metrics as well as steps you can take to eliminate the spam from Google Analytics. If you are looking for ways to filter this spam out of your Megalytic reports, see: Removing Semalt and other Referer-Spam from Megalytic Reports.

Google Analytics Spam

 

Is Google Analytics Spam Messing Up Your Metrics? (Probably)

What’s the impact of a little spam data on your Google Analytics metrics? If you run a large website with tens of thousands of visitors or more per day, then maybe not much. However, if your site is smaller, there’s a good chance spam may be seriously skewing your metrics.

Below is the Acquisition > All Traffic > Referrals report from a small non-profit. I’ve checked the spam referral sources, and clicked “Plot Rows” to see the daily level of traffic from these spammers.

 

Google Analytics Referrals Reports Showing Spam

 

In the table above, you can see that spam accounts for the top two slots in this Referrals Report! Not only is this annoying, but it messes up the metrics pretty badly.

To analyze the spam’s impact on the non-profit’s metrics, I exported this table into Excel and did some calculations. In the results below, the Sources highlighted in yellow are spam referrals; the two summary lines at the bottom show metric calculations with and without spam.

 

Spreadsheet Analysis of Google Analytics Spam

 

The first thing to note is that 144 out of 283 referral Sessions are spam – that’s 50.9%! The impact on small websites like this one is huge as these spam visits throw off the engagement metrics. As you can see from the spreadsheet, the Bounce Rate for most spam referrals sources is 100%, the Pages/Session is close to 1.0 and the Avg. Session Duration is close to 0.00. When more than 50% of the referral traffic is spam, it is seriously dragging down the engagement numbers and giving you a false impression of the quality of your traffic.

Compare the Bounce Rate, Pages/Session, and Avg. Session Duration for “Including Spam” vs “Excluding Spam” (numbers inside the red rectangle). The spam is making these metrics look much worse than they really are. Bounce Rate, for example, is reading 77.74%. But, when we exclude the spam, the Bounce Rate is a much better 55.4%.

Other than exporting data to Excel and re-calculating all the numbers, is there any way we can stop these spam referrals from polluting our Google Analytics data?

Filtering Out Google Analytics Spam

The techniques for removing spam rely on using Google Analytics View Filters. I first read about these techniques in this excellent article from the Analytics Edge blog: Removing Referral Spam from Google Analytics.

As explained in that article, there are two basic groups of spammers using two different techniques, and you need to use slightly different filters to combat each technique.

Eliminating Ghost Referrals

The first group is what people are starting to call “Ghost Referrals.” These are referrals generated in your reports by fake visits. In this scenario, the spammers don’t even visit your website. Instead, they transmit spammy data directly to Google Analytics that gets added to your reports.

To start cleaning this up, we create a new view and then add some filters. As shown below, you can create a view in the Admin section of Google Analytics. Pick the Account and Property where you want to create a spam-free view. [Note: Views do not contain historical data older than the date on which they are created. If you create a view on Jan 2nd, there will be no data in that view prior to Jan 2nd. So, this new spam-free view will not help clean up the historical data - only the new data coming in.]

 

Creating a New View in Google Analytics

 

Next, we are going to create a list of the valid hostnames that should be showing up in your Google Analytics reports. The key to removing ghost referrals is that they come from hostnames that are not yours – and you can use that weakness to filter them out.

Below is a list of the valid hostnames of visits to our Megalytic website:

  • megalytic.com
  • blog.megalytic.com
  • support.megalytic.com
  • megalytic.com
  • translate.googleusercontent.com

Note the last one – translate.googleusercontent.com. This is the hostname that shows when a user views your website through Google Translate – you do not want to filter that out.

If you are not sure of your list of valid hostnames, you can look at the Audience > Technology > Network report and select Hostname as the primary dimension. Set a long time range in the calendar – like a year or more if you have that much data. This will ensure that you capture all the valid hostnames.

Here is what that report looks like for Megalytic. The valid hostnames have little red arrows next to them. The rest (e.g., apple.com, iedit.ilovevitaly.com) are from spammers!

 

Google Analytics Report on Hostnames

 

Once you have your list of valid hostnames, put them in a single line of text, separated by the “|” – OR character. Also put a backslash in front of all the “.” – PERIOD characters. This creates a regular expression that will match on your good hostnames and exclude all the spammer hostnames.

For example, here is what we use:

megalytic\.com|blog\.megalytic\.com|old\.megalytic\.com|forums\.megalytic\.com|client\.meglytic.com|
translate\.googleusercontent\.com|support\.megalytic\.com

Before you put this filter expression to use we recommend that you build a segment to test it out on your historical data to see how it looks. Filters permanently alter the data in a view, so it’s a good idea to test filter expressions using non-permanent segments on your historical data before using them in filters.

Another benefit of testing your filter expression in a segment is that you can use this segment to look at your historical data without the ghost referrals.

Here is the testing segment created for Megalytic, which we named “My Hosts.”

 

Google Analytics Use Segment to Test FIlter

 

And here are the results, filtered by using the “My Hosts” segment:

 

Google Analytics Report Comparing Segments

 

As you can see, some of the sessions have been filtered out – the “My Hosts” segment has 19,934 Sessions vs 20,235 in “All Sessions.”

Next, apply the “My Hosts” segment to your Audience > Technology > Network report and select Hostname as the primary dimension. Check to see that only valid hostnames are showing up. Below are the results for Megalytic.

 

Google Analytics Segment Filters Out Spam

 

Once you are confident your filter expression is working correctly, add it to your new view. We called our new view “Spam Free.” You can see below how we selected the “Filters” section to create a filter on this view, and then pasted in our filter expression as a Custom Filter Type. Make sure to select “Include” and to filter on the “Hostname” field.

 

Google Analytics Filter Definition with Hostnames

 

Save this filter and you should be all set. This new view will now exclude all ghost-referral spam. Unfortunately, filtered views only include data from the date they were created. So, you cannot use this view to look back at historical data. However, you can use the segment “My Hosts” created during the testing process to view spam-free historical results.

Eliminating the non-Ghost Referral Spam

Unlike the ghost referrals, some of the spammer bots, like Semalt, actually visit your website. These will not be removed using the hostname filter described above. To remove these, you will need to create another filter that will exclude a list of known referral spam domains.

So, to clarify, the first filter INCLUDES only your valid hostnames. That kills the ghost-referral spammers. This second filter will EXCLUDE known spammer domains.

To find the non-ghost spammers visiting your website, open Acquisition > All Traffic > Referrals and add Hostname as a secondary dimension. Spam sources where the Hostname is valid (in our case, megalytic.com) are the non-ghost spammer domains we need to exclude.

 

Google Analytics Spam Referral Sites - Non Ghosts

 

From this list, you can see that semalt.com and buttons-for-website.com should go on our list. As before, create a filter, but this time use Referral as the Filter Field, and the filter is:

semalt\.com|buttons-for-website\.com

As shown below, we name this filter “Exclude non-Ghost Referral Spam.”

 

Google Analytics Filter to Exclude Spam Referrals

 

You should check your Acquisition > All Traffic > Referrals report periodically to identify any new spam referral domains that start showing up. Add these new ones to your filter as necessary to keep your data as spam-free as possible.

Another approach to filtering out the non-ghost spammers is to stop them from visiting your website at all. If you are hosting your website on the Apache web server, this kind of blocking can be accomplished by modifying the .htaccess file, as described here: How to block referrer spam traffic.

If you are running WordPress, there is now a plugin that will do this for you: SpamReferrerBlock. One advantage of using this plugin is that they claim to keep a “blacklist” of domains that are spammers and filter those visits for you, so you do not have to keep your filters up to date.

Conclusions

Referral spam is becoming a serious problem and I expect that Google will soon introduce new features to help us protect the integrity of our Google Analytics data. Until then, you can use the filtering techniques described in this post to create a view that is relatively free of referral spam.

Appendix

Its been almost 2 years since I wrote this, and the Google Analytics spam problem is still with us! If you are looking for more details on this subject, I suggest that you check out Carlos Escalera's post: Ultimate Guide to Getting Rid of All the Spam in Google Analytics.

Update on March 23, 2017 ...
I've seen a few articles indicating that Google is taking action to solve this problem. If you have noticed an improvement, let me know in the comments.

ALSO IN THIS BLOG

As a digital marketer, juggling a variety of advertising platforms is just a part of everyday life. From Google Ads and Facebook to Twitter, LinkedIn, AdRoll, and more, you’re used to having a lot on your plate.
 
But when you have to worry about installing performance-tracking tags and pixels on your website so you can optimize for each platform, things can get overwhelming. Not only do you have to add this time-consuming task to your IT team’s already long to-do list, but it creates a lot of additional code to manage, maintain, and update.
There are lots of ways to drive traffic to a website these days – social media, pay-per-click ads, display ads, email marketing, etc. But, for many websites, organic search remains the largest source of traffic.
 
As an agency, you need to help your clients understand what’s driving organic search traffic to their websites and how they can grow that traffic. This means you need a source of data that measures organic search traffic. For traffic from Google Search, there is an excellent free tool from Google providing that data: Google Search Console (GSC). While other tools "scrape" data from the Google search engine result pages (SERPs), GSC provides the official data from Google.
In this blog post, we’ll look at what kind of data you can pull from Google Search Console, how to use it for SEO, and how to best share it with your clients.
Facebook lead ads are incredibly useful when you want to create a connection with potential prospects. You can use lead ads to gather emails and other contact information, provide quotes and estimates, get people to sign up for your newsletter, and just about anything else you can think of. The benefit of lead ads is that they help you engage with your audience on a deeper level by collecting contact information.
But how can you create lead ads that are more likely to convert? Luckily, there is a lot you can do to optimize your lead ads. Whether you’re new to Facebook lead ads or simply want your current ads to perform better, this post will help you improve your conversion rates.