NPBot – Nameprotect lying about respecting the robots.txt file

From what I have read about Nameprotect and their web-bots, I shouldn’t have been surprised that they are lying when they claim to respect robots.txt:

[…]Practices the following best practices to ensure non-invasive crawling:

Honoring robots.txt files – to exclude the NPBot crawler, please use “NPBot” as the user-agent name in your robots.txt file […]

I have added the necessary line to my robots.txt file to exclude the NPBot from my site, but they are still indexing without caring about what the robots.txt file says.

Its also worth noting that Nameprotect have been (and probably still are) using spiders which are lying about who they are. (more information about “crawler918.com” / Nameprotect)

So my suggestion, if you want to get rid of the Nameprotect bots, is to block their IP-range from accessing your web pages:

Add the following to your .htaccess file:

Deny from 12.148.209.192/26

See my earlier post about NPBot

7 comments

  1. This is my opinion of Name Protect… As a tech for several domains that are only used for e-mail services it really is interesting to watch who will hit a site after a DNS update, etc without ever being invited. The number one IP to hit the server is always Name Protect and then VigilActive – one in the same usually running on US West Internet. You can read more about their “partners” here:

    http://www.ltutech.com/News_press_releases_NP.htm

    or:

    http://www.ltutech.com

    The other uninvited visitors that hit the server uninvited are:

    Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; http://www.inktomi.com/slurp.html

    And: Scooter/3.3

    I have read a lot about privacy issues in regards to Google, as in the link below, but I have yet to ever have a site hit by Google uninvited – that doesn’t mean that there are concerns but the bot comes invited and obeys robot text. I have NEVER seen Inktomi obey anything and their polocies are not listed and all attempts to have their bot halted fail; that always makes me very unhappy.

    http://news.bbc.co.uk/1/hi/technology/2786761.stm

    Scooter belogs to AltaVista, but it is still uninvited, does NOT look for robots text first. From my point of view I don’t care who owns the bot when it does not look for robots text FIRST.

    Other things I’ve seen and heard about are all of the web content rating bots – there are rumors that many pass the information off to the government under subcontracts as several of these companies that operate in California and Oregon were almost bankrupt then along came Ashcroft to save the day.

  2. I would also strongly urge blocking all IP’s associated with the following using a Qwest – US West Communications netblock:

    Cyveillance

    Other bad bots include RuleSpace and Web Content International as I’ve read in several places on the Internet that RuleSpace and Web Content International are the same and that some Mormon friend of the RuleSpace guy has registered IP’s used by that company under the name Web Content International. If true, US West QWEST should not allow this to happen!

    Other people have posted remarks that Name Protect and VigilActive may have connections with the former since the IP’s and bots from Name Protect and VigilActive have shown up in logs more and more.

    I’m not sure if this is true, but I think there are some companies that may be offering a service that will block a lot of this junk and traffic from their servers by using dynamic updates of the IP’s these companies use. I heard it will use the same technology that these companies but in reverse. If true it serves them right!

    In any case I would never use U.S. West QWEST services (dial-up, DSL, phone, hosting, etc.) based on the junk I’ve seen in my logs from them and the things they allow on their networks. Funny, effective in 1999 these two companies “merged” so junk from QWEST is junk from U.S. West! Then we have the securities fraud cases against Philip Anschutz… So, it’s not just bad bots and junk IP traffic that comes from these people.

  3. Dear “VigilActivity”

    This is just not true about Google in the BBC story on your web site story above. Don’t you think it is rather odd that google-watch talks a lot about WebmasterWorld as it is some “other” website. Don’t you think it is funny that webmasterworld and google-watch use the same servers WESTSERVER.NET in Providence, Utah. Vigil, Providence, Utah is a nowhere place in the Cache Valley of Utah near Logan. The more I find out about google-watch and webmasterworld and phd software LOL the more I wonder about who really runs these things? the American FBI, CIA or NSA or the Mormon Mafia. I would never use google-watch and I think you should do a little more research on these things as everything is not what they seem. The same with webmasterworld that is headquarted in a town of less than 2,000 people in the middle of nowhere U.S.A. Vigil, things are not always what they seem and before you think I’m a conspiracy nut, one of the most used proxy sites a few years ago turned out to be funded through the CIA. Of course the American FBI and CIA just love to recruit in Utah and if you think I’m kidding, do a few searches on that we hope without using google-watch. NPBot, they have partners who also have U.S.A. contracts. Knowledge is power and you need to make sure you know who you are dealing with, NPBot or google-watch.

  4. Migel is right!

    I came here looking for information on NPBot and I ended up more interested in the google-watch and webmasterworld story. It really does make you wonder who these people really are and how long it will take to hide their name servers now that the Utah link has been compromised. I’ve looked at some information on webmasterworld and to be honest it was my opinion that much of the information is suspect. On the other hand, why does google allow google-watch to use the proxy? Something doesn’t add up here and I’m sure that Daniel Brandt and Brett Tabke will inform us soon as to why they both have the same connection to Providence, Utah LOL If they and/or google are the American Government it won’t be long until this information is deleted in bots but I received the following e-mail yesterday evening so I guess the word will stay out there:

    Subject: Webmasters Alert! Alert! Alert!

    The number one google lover in the world, Brett Tabke, who runs webmasterworld http://www.webmasterworld.com and the number one google hater in the world, Daniel Brandt, who runs google-watch http://www.google-watch.org have their servers traced to the same address, 200 N Spring Creek Parkway, Providence, UT 84332 The odds of this being random may range a million-to-one and if something doesn’t look right, it usually isn’t right. Please pass this information to other interested webmasters ASAP!

    INFORMATION FOR WEBMASTERWORLD THE WORLD’S NUMBER ONE LOVER OF GOOGLE:

    Domain Name: WEBMASTERWORLD.COM

    Registrar: IHOLDINGS.COM, INC. D/B/A DOTREGISTRAR.COM

    Whois Server: whois.dotregistrar.com

    Referral URL: http://www.dotregistrar.com

    Name Server: NS.WESTHOST.NET

    Name Server: NS2.WESTHOST.NET

    Status: ACTIVE

    Updated Date: 18-aug-2002

    Creation Date: 11-dec-1999

    Expiration Date: 11-dec-2004

    INFORMATION FOR WESTHOST.NET:

    Domain name: westhost.net

    Registrant Contact:

    WestHost Inc.

    WestHost Hostmaster (hostmaster@westhost.com)

    +1.4357553433

    Fax: +1.4357553449

    200 N Spring Creek Parkway

    Providence, UT 84332

    US

    ///////////////////////////////////////////////////////////////////////

    INFORMATION FOR GOOGLE-WATCH THE WORLD’S NUMBER ONE “HATER” OF GOOGLE:

    Domain ID:D87793486-LROR

    Domain Name:GOOGLE-WATCH.ORG

    Created On:23-Jun-2002 14:52:35 UTC

    Last Updated On:12-Sep-2003 23:04:55 UTC

    Expiration Date:23-Jun-2004 14:57:46 UTC

    Sponsoring Registrar:R91-LROR

    Status:OK

    Registrant ID:GODA-01182811

    Registrant Name:Daniel Brandt

    Registrant Street1:PO Box 680635

    Registrant City:San Antonio

    Registrant State/Province:Texas

    Registrant Postal Code:78268-0635

    Registrant Country:US

    Registrant Phone:+1.2105093160

    Registrant FAX:+1.2105093161

    Registrant Email:namebase@earthlink.net

    Admin ID:GODA-21182811

    Admin Name:Daniel Brandt

    Admin Street1:PO Box 680635

    Admin City:San Antonio

    Admin State/Province:Texas

    Admin Postal Code:78268-0635

    Admin Country:US

    Admin Phone:+1.2105093160

    Admin Email:namebase@earthlink.net

    Tech ID:GODA-11182811

    Tech Name:Daniel Brandt

    Tech Street1:PO Box 680635

    Tech City:San Antonio

    Tech State/Province:Texas

    Tech Postal Code:78268-0635

    Tech Country:US

    Tech Phone:+1.2105093160

    Tech Email:namebase@earthlink.net

    Name Server:NS3.WESTSERVER.NET

    Name Server:NS4.WESTSERVER.NET

    INFORMATION FOR WESTSERVER.NET:

    Billing Contact:

    WestHost Inc.

    WestHost Hostmaster (hostmaster@westhost.com)

    +1.4357553433

    Fax: +1.4357553449

    200 N Spring Creek Parkway

    Providence, UT 84332

    US

    Status: registrar-lock

    Name Servers:

    NS3.westserver.net

    ns4.westserver.net

    Another note on this e-mail that I received

    I did some looking around on both of the sites last night and it is like their views of google are totally different. So why do they both end up traced to a server company in Providence, Utah? How many people do you know who use server companies in Providence, Utah. I’ll admit, none of this makes sense to me! FBI, CIA, it looks odd so user beware.

Comments are closed.