Opened 6 years ago
Closed 6 years ago
#4138 closed defect (bug) (maybelater)
PROPOSAL: Maintain a blacklist of obviously nefarious traffic sources
Reported by: | jonoaldersonwp | Owned by: | |
---|---|---|---|
Milestone: | Priority: | low | |
Component: | General | Keywords: | analytics |
Cc: |
Description
We monitor where traffic to wordpress.org comes from, in order to help understand our marketing performance and to prioritise strategies/tactics. ~40% of our traffic comes in the form of referrals from other sites.
A large proportion of that traffic is obviously fake, and/or nefarious by nature; general spam bots, or, more maliciously targeted fake traffic, designed to confuse, elicit clicks, or obfuscate other behaviours.
Some of this, we can see pretty easily. For example, between Jan 22nd-24th:
- Nearly 8% of our referral traffic (46,000 users) originated from this **NSFW** site.
- 2.5% of our referral visits (12,000 users) came from [gamefullpc.net]
- 1.5% came from (9,000 users) came from http://onlinetrafficbot.com
Further investigation into the behaviour of these particular visits shows significant evidence that they're bots - browsing patterns and meta data associated with the visits sticks out like a sore thumb.
These are bad sites, sending fake users, costing us time and money, and muddying our understanding of _actual_ user behaviour.
My original suggestion was going to be that we should filter out visits from these blacklisted sources/domains in Google Analytics ("ignore traffic with X referring source") - however, upon reflection, perhaps we should block these from the site entirely?
My suggestion, then, is that:
- We undergo a regular review process of referrering sources in GA.
- If they're obviously _bad_, we add them to a list to block them on the load balancer level, based on the
referrer
. - If it's less obvious what's going on, we consider filtering out in GA on a case-by-case basis.
Thoughts appreciated.
Change History (9)
#3
@
6 years ago
How much of what - these referrers specifically, or, all referral traffic?
This isn't data which has been submitted via measurement protocol, they're hits triggered by bot visits.
#4
@
6 years ago
Oh. If these are spam bots that do real visits in order to infect the stats with fake referrals, then it would just be right to block them at the perimeter.
#5
@
6 years ago
Could some of these be browsers aggressively pre-loading links on the page?
I ask because I notice that in the case of gamefullpc net, the referrals mostly go to the two pages linked to from that site in the footer. Note that the themezee link redirects to the w.org page.
#6
@
6 years ago
Oh, now that's an interesting idea. I can't see any evidence of it, though. Nothing obvious in headers, network connections, or JS.
#7
@
6 years ago
Bot/fraud detection is a specialised area that can easily turn into a game of whack-a-mole. If Google Analytics can't detect these bots, how are we going to successfully manage it going forward? Google has resources and expertise that we don't.
#8
@
6 years ago
Agreed. This is whack-a-mole. However, we don't need to detect the bots; if we just filter out particularly high volume referring domains of obviously bad traffic, and we'd easily clean up a big chunk.
I can do/maintain this via GTM if we'd prefer (nice interface, access control, change logs) and chip away at it happily - but that only hides the issue. Feels like if we're going that far, we should probably have a process to block them from the site (rather than just hide them from our tracking, save the bandwidth/processing overhead, etc)...
@Otto42 The plot thickens, I think all of the sites in question have been hacked, and are running some nasty obfuscated JS. Check the first script which gamefullpc loads. Can't see any evidence that it's preloading in/from there, but I might be missing something?
How much if this is REAL traffic?
There are spammers that just send virtual hits directly to GA, without ever actually visiting the site.