Ok so I finally got around to doing all the things I meant to do for a long time…
I also added the latest comments plugin on the index page, which although it’s totally useless, it’s pretty neat anyway.
The latest problem I’ve been having is with Referal Spam and Comment Spam.
For those that don’t know:
Referer
A misspelling of “referrer” which somehow made it into the HTTP standard. A given web page’s referer (sic) is the URL of whatever web page contains the link that the user followed to the current page. Most browsers pass this information as part of a request.
My referals were starting to look like this:
Which is not a good thing…
I found an addition to the .htaccess file which may stop this, I only started using it today though, so I’ll wait and see the results..
This code sends back the ‘403 Forbidden’ response for anyone who’s asking for a header only and who is sending a referer.
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http [NC]
RewriteCond %{REQUEST_METHOD} ^HEAD$ [NC]
RewriteRule .* / [F,L]
There are some things out there which use the HEAD command like local proxies and webcaches but they generally don’t send a referer when they do this.
Source is HERE
There is also one to block on specific words/domains:
SetEnvIfNoCase Referer ".*(anal|adulthost|latina|shemale|viagra|valium|fioricet|ebony|
hydrocodone|3d.net|v33|brad.com|ambien|xrated|tranny|phentermine|vicodin|credit|canadianlabels|
8gold|texas-hold|hold-em|holdem|fidelityfunding|condo|sportsparent|mortgage|spoodles|money|
cash|hotel|houseofseven|stmaryonline|newtruths|popwow|oiline|flafeber|thatwhichis|tmsathai|pisoc|
crepesuzette|mediavisor|commerce|easymoney|911|////.vi|gb////.com|4free|macsurfer|teen|pussy|
discount|blogincome|lillystar|aizzo|webdevsquare|laser-eye|escal8|xopy|vixen1|linkerdome|
youradulthosting|fick|inkjet-toner|fuck|ime.nu|perfume-cologne|italiancharmsbracelets|
shoesdiscount|psnarones|hasfun|casino|gambling|poker|porn|sex|paris|gabriola|nude|xxx|
hilton|pics|video|adminshop|devaddict|iaea|empathica|insuranceinfo|atelebanon|handy-sms|
peng|just-deals|pisx|rimpim).*" BadReferrer
order deny,allow
deny from env=BadReferrer
Will wait and see if it works, might have to tweak it a bit, note the first section should be all one line, so remove the carriage returns.
There is more info:
Sample .htaccess file
Referer Spam
So my finished current (tested) .htaccess file looks like so (remember the spam words is all one line):
## shaolintiger.com Anti-spam .htaccess file © 2005
RewriteEngine On
RewriteBase /
## List of dirty spamming websites, matches any part of referer
SetEnvIfNoCase Referer ".*(anal|adulthost|latina|shemale|viagra|valium|fioricet|ebony|hydrocodone|
3d.net|v33|brad.com|ambien|xrated|tranny|phentermine|vicodin|credit|canadianlabels|8gold|texas-hold|
hold-em|holdem|fidelityfunding|condo|sportsparent|mortgage|spoodles|money|cash|hotel|houseofseven|
stmaryonline|newtruths|popwow|oiline|flafeber|thatwhichis|tmsathai|pisoc|crepesuzette|mediavisor|
commerce|easymoney|911|////.vi|gb////.com|4free|macsurfer|teen|pussy|discount|blogincome|lillystar|
aizzo|webdevsquare|laser-eye|escal8|xopy|vixen1|linkerdome|youradulthosting|fick|inkjet-toner|fuck|
ime.nu|perfume-cologne|italiancharmsbracelets|shoesdiscount|psnarones|hasfun|casino|gambling|poker|
porn|sex|paris|gabriola|nude|xxx|hilton|pics|video|adminshop|devaddict|iaea|empathica|insuranceinfo|
atelebanon|handy-sms|peng|just-deals|pisx|rimpim).*" BadReferrer
## List of bad User-Agents
SetEnvIfNoCase User-Agent ".*(bdfetch|npbot).*" BadUA
order deny,allow
deny from env=BadUA
deny from env=BadReferrer
## This rule will block anyone that just gets the HEAD and sends a REFERER without actually viewing the page.
RewriteCond %{HTTP_REFERER} ^http [NC]
RewriteCond %{REQUEST_METHOD} ^HEAD$ [NC]
RewriteRule .* / [F,L]
## Tells spammers to bugger off if appending underscore to file names
RewriteRule ^.*_$ - [F,L]
Seems to be ok so far, I had to tweak it a bit though as it was giving me an infinite loop somewhere resulting in the famous:
“Redirection limit for this URL exceeded. Unable to load page requested. This may be caused by the cookies that are blocked.”
As for comment spam, I’ve just enabled it so anyone posting a URL or using HTML will be moderated, but the spammers have gotten smart…they now break the HTML tags onto two lines..so the regex doesn’t find it.
Perhaps if I get too many, I’ll implement captcha images.
Anyway that’s all for now, I’m gonna go delete the spammers from my database..
Good thing you have sitemeter in addition to server logs. The pixel based tracking overlooks referal spams but I’m sure the spammers are working on it this moment.
I wonder how come no nerds have used it as a creative method of proposal? Imagine his gf finding a “Marry Me” message in her logs! :p
Haha, yeah actually I found most it from a referal plugin I was using, which was interesting..
But then it started getting flooded with spam as above, so I took it out, but I missed it, so tried to find some way to stop the spammers.
The referal page is hidden at /ref.php at the moment. 99% success so far, my error log is full of 403’s!
Seems to be working well so far, I like Sitemeter for the quick view daily averages and browser share stats 🙂
Proposal, certainly interesting. But how many chicks would actually check their referals? Apart from blog ho’s like Suan 😀
Maybe “Referer” is the new American spelling. Up there with Color, Neighbor, and Fonetic! 🙂
Nah it’s not the yank spelling, it’s the techy spelling, somehow made it into the HTTP standard 😀
cipet…
*blink* *blink*
hmmm….blog page this has fade away from…..portal site it has become….yes….heavy the load i see, hard to navigate i sense..hmmm hmmmm…..keep the simplicity, keep youmust
Been trying to make it easier to find stuff actually haha…
Maybe declutter a bit later.