[Update: More information about NPBot and Nameprotect]
Found a link to this cute little bot in my referer log: NameProtect Inc :: NPBot Notification
Interesting to see someone so openly telling people they are looking for “intellectual property violations”. I wonder if they respect my copyright..
They say they identify with the NPBot user-agent string, which would make filtering out any attempts by the bot pretty easy, or even better – serving up content that did not in-fridge on any copyright if you are into giving away stolen goods. So what is the deal? Are they only looking for the regular Joe Schmo, stupid enough to just publish MP3’s and other “goodies” on their homepages?
[Update]: A little more information about the NPBot‘s behaviour. It seems like Nameprotect isn’t all that interested in playing “fair”, and that they are lying when saying they respect the Robots.txt file. I am testing it myself to see if NPBot will actually accept not to spider this site. And I have added this to the robots.txt file:
User Agent: NPBot
Disallow: /
I am contemplating starting to disallow the IP-range of Nameprotects servers if it doesn’t behave.
More interesting note about robots. I am seeing a LOT of traffic from a spider only identifying itself as “Robot”, it seems to be very agressive, and are operating from a japanese server/machine (p1223-ipbf02hodogaya.kanagawa.ocn.ne.jp). Anyone else seen that kind of behaviour?
Nice catch, Jarle…
Interestingly enough they even say that they respect the robots.txt protocol so one wouldn’t even have to set up cloaking [1] or something else server-side to prevent them from catching you (if you had malicious intents, that is)…
[1] http://www.jacobsen.no/anders/blog/archives/2002/10/08/what_is_cloaking.html
it does seem to respect robots.txt. i added it a few days ago and sure enough, the next visit *only* requested /robots.txt. now what kind of alarms blocking this specific spider will set of remains to be seen :-)
From a moral perspective, I have blocked access to NPBot through .htaccess. I will not contribute to keeping a fundamentally flawed IP system.
Besides, even if they are respecting robots.txt, the behavior it showed was close to hammering my site.
exactly what user agent did you put in the robots.txt file for NameProtect?
Which if these is right:
(A)
User Agent: NPBot-1/2.0 (http://www.nameprotect.com/botinfo.html)
Disallow: /
or (B)
User Agent: NPBot
Disallow: /
I am testing the B alternative. Another way to deal with the Nameprotect bot is to block all access from their IP addresses (See the update and the link there for more information about what to put in your .htaccess file).
I had just launched a website (with a common misspelling of a wellknown searchengine in the domainname) for 3 days when I discovered the first request from this bot. I wonder how they were able to find my side so fast. either they grab whois/dns data or they have strong connections to some search engines.
I use the following in .htaccess deny access for NPBot and Nameprotect.com
SetEnvIf Remote_Addr 12\.40\.85\.([0-2][0-5][0-5]) ban
SetEnvIf Remote_Addr 12\.148\.196\.(12[8-9]|1[3-9][0-9]|2[0-4][0-9]|25[0-5]) ban
SetEnvIf Remote_Addr 12\.148\.209\.(19[2-9]|2[0-4][0-9]|25[0-5]) ban
<Files ~ “^.*$”>
order allow,deny
allow from all
deny from env=ban
deny from env=bad_bot
</Files>
This is my opinion of Name Protect… As a tech for several domains that are only used for e-mail services it really is interesting to watch who will hit a site after a DNS update, etc without ever being invited. The number one IP to hit the server is always Name Protect and then VigilActive – one in the same usually running on US West Internet. You can read more about their “partners” here:
http://www.ltutech.com/News_press_releases_NP.htm
or:
http://www.ltutech.com
The other uninvited visitors that hit the server uninvited are:
Mozilla/5.0 (Slurp/cat; [email protected]; http://www.inktomi.com/slurp.html
And: Scooter/3.3
I have read a lot about privacy issues in regards to Google, as in the link below, but I have yet to ever have a site hit by Google uninvited – that doesn’t mean that there are concerns but the bot comes invited and obeys robot text. I have NEVER seen Inktomi obey anything and their polocies are not listed and all attempts to have their bot halted fail; that always makes me very unhappy.
http://news.bbc.co.uk/1/hi/technology/2786761.stm
Scooter belogs to AltaVista, but it is still uninvited, does NOT look for robots text first. From my point of view I don’t care who owns the bot when it does not look for robots text FIRST.
Other things I’ve seen and heard about are all of the web content rating bots – there are rumors that many pass the information off to the government under subcontracts as several of these companies that operate in California and Oregon were almost bankrupt then along came Ashcroft to save the day.
Just hit em with
RewriteCond %{REMOTE_ADDR} ^12\.148\.(12[8-9]