• Kaizeku Crawler Maps

      Crawler List for November 2007

      Crawlers User agent
      Xirq xirq/0.1-beta (xirq; http://www.xirq.com; xirq@xirq.com)
      WebSearchBench WebSearchBench WebCrawler V1.0 (Beta), Prof. Dr.-Ing. Christoph Lindemann, Universität Dortmund, cl@cs.uni-dortmund.de, http://websearchbench.cs.uni-dortmund.de/
      Yahoo Search Japan robot Y!J-BSC/1.0 (http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html)
      NimbleCrawler Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) NimbleCrawler 1.11 obeys UserAgent NimbleCrawler For problems contact: crawler_at_dataalchemy.com
      Fastbot fastbot crawler beta 2.0 (+http://www.fastbot.de)
      Gigabot Gigabot/2.0/gigablast.com/spider.html
      Jambot Jambot/0.1.1 (Jambot; http://www.jambot.com/blog; crawler@jambot.com)
      Netluchs Netluchs/0.8-dev ( ; http://www.netluchs.de/; ___don\’t___spam_me_@netluchs.de)
      NutchEC2Test NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com)
      Bigsearch Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com)
      UKWizz UKWizz/Nutch-0.8.1 (UKWizz Nutch crawler; http://www.ukwizz.com/)
      Ilial/Nutch ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; crawl@ilial.com)
      Pmoz Mozilla/5.0 (compatible; pmoz.info ODP link checker; +http://pmoz.info/doc/botinfo.htm)
      Holmes holmes/3.11 (OnetSzukaj/5.0; +http://szukaj.onet.pl)
      Flatlandbot flatlandbot/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; jason@flatlandindustries.com)
      IDBot Mozilla/5.0 (compatible; IDBot/1.0; +http://www.id-search.org/bot.html)
      Spam Bot Mozilla/2.0 (compatible; NEWT ActiveX; Win32)
      Greaterera Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.greaterera.com/)
      GEXTEST-00393 gsa-crawler (Enterprise; GEXTEST-00393; gsasymbiosys@gmail.com,xeonbox4@gmail.com)
      Pagebull Pagebull http://www.pagebull.com/
      RSS One Engine RSS One Engine/0.72 (+http://www.rss-one.com)
      Dodgebot dodgebot/experimental
      Bot bot/1.0 (bot; http://; bot@bot.bot)
      Bigsearch Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com)
      FindLinks findlinks/1.1.4-beta1 ( http://wortschatz.uni-leipzig.de/findlinks/)
      ConveraCrawler ConveraCrawler/0.9e ( http://www.authoritativeweb.com/crawl)
      Blaiz-Bee Blaiz-Bee/2.00.5622 ( http://www.blaiz.net)
      KIT_Fireball KIT_Fireball/2.0
      ICC-Crawler ICC-Crawler(Mozilla-compatible;http://kc.nict.go.jp/icc/crawl.html;icc-crawl-contact(at)ml(dot)nict(dot)go(dot)jp)
      Pubblisito info@pubblisito.com- (http://www.pubblisito.com) il Sud dei Motori di Ricerca
      SkreemRBot Mozilla/5.0 (compatible; SkreemRBot +http://skreemr.com)
      WebAlta Crawler WebAlta Crawler/1.3.33 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)
      Pumpkin blogsearchbot-pumpkin-3
      Mail.Ru Mail.Ru/1.0
      Mammoth Mozilla/5.0 (+http://www.eurekster.com/mammoth) Mammoth/0.1
      Attentio Attentio/Nutch-0.9-dev (Attentio\’s beta blog crawler; www.attentio.com; info@attentio.com)
      GurujiBot GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)
      Gigabot Gigabot/3.0 (http://www.gigablast.com/spider.html)
      Jobs.de-Robot Mozilla/5.0 (compatible; jobs.de-Robot http://www.jobs.de; jobsde@jobscout24.de) ( newsexpress e-mail: newsexpress-l@neofonie.de http://www.neofonie.de/loesungen/search/robot.html )
      ArabyBot ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby.com;)
      VWBOT VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu;+vwbot@cs.uiuc.edu
      IWAgent IWAgent/ 1.0 - www.brandprotect.com
      Sirketcebot Sirketcebot/v.01 (http://www.sirketce.com/bot.html)
      Spock Crawler Spock Crawler (http://www.spock.com/crawler)
      Flatlandbot great-plains-web-spider/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; jason@flatlandindustries.com)
      Nebulla Nebullabot/2.2 (http://bot.nebulla.de)
      EasyDL EasyDL/3.04 http://keywen.com/Encyclopedia/Bot
      LapozzBot LapozzBot/1.4 (+http://robot.lapozz.hu)
      WWW.fi crawler www.fi crawler, contact crawler@www.fi
      Uni-koblenz http://www.uni-koblenz.de/~flocke/robot-info.txt
      NimbleCrawler Mozilla/5.0 (Windows;) NimbleCrawler 2.0.1 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com
      YodaoBot Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )
      DAUM RSS Robot ELI/20070402:2.0 (DAUM RSS Robot, Daum Communications Corp.; +http://ws.daum.net/aboutkr.html)
      DAUM Web Robot Mozilla/4.0 (compatible; MSIE enviable; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea; +http://ws.daum.net/aboutkr.html)
      Changedetection Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://www.changedetection.com/bot.html )
      ICC-Crawler ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl-contact(at)ml(dot)nict(dot)go(dot)jp)
      Semager Semager/1.1 (http://www.semager.de/blog/semager-bots/)
      Multicrawler multicrawler ( http://sw.deri.org/2006/04/multicrawler/robots.html)
      NetinfoBot NetinfoBot/1.0 (http://netinfo.bg/netinfobot.html)
      Envolkspider envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html)
      CazoodleBot CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com)
      RutterBot RutterBot(+http://www.aktienbetreuer.de/bot.html)
      Worio bot Mozilla/5.0 (compatible; woriobot heritrix/1.10.0 +http://worio.com)
      Tags2dir tags2dir.com/0.8 (+http://tags2dir.com/directory/)
      Combine Combine/3 http://combine.it.lth.se/
      Lawinfo-crawler lawinfo-crawler/Nutch-0.9-dev (Crawler for lawinfo.com pages; http://www.lawinfo.com; webmaster@lawinfo.com)
      FuseBulb FuseBulb.Com
      Earthcom Mozilla/5.0 (compatible; EARTHCOM/2.2; +http://enter4u.eu)
      Askpeter_bot Mozilla/5.0 (compatible; askpeter_bot/3.2; +http://www.askpeter.info)
      LapozzBot LapozzBot/1.5 (+http://robot.lapozz.hu)
      FAST-WebCrawler FAST Enterprise Crawler/6.4.18 (crawler@fast.no)
      BuiltWith Mozilla/5.0 (compatible; BuiltWith/0.1; +http://builtwith.com/bot.html)
      Hiiglespider Hiiglespider/0.1, Hiigle.com, http://hiigle.com/spider
      Page-store Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com)
      Metacarta Mozilla/5.0 (compatible; heritrix/1.5 +http://www.metacarta.com)
      Multicrawler multicrawler (+http://sw.deri.org/2006/04/multicrawler/robots.html)
      LibertyW LibertyW (+http://www.libertyw.eu)
      BlogRefsBot Mozilla/5.0 (compatible; BlogRefsBot/0.1; http://www.blogrefs.com/about/bloggers)
      Holmes holmes/3.11 (http://morfeo.centrum.cz/bot)
      DataparkSearch DataparkSearch/4.47 (+http://dataparksearch.org/bot)
      ImageWalker ImageWalker/2.0 (www.bdbrandprotect.com)
      SeznamBot SeznamBot/2.0-test (+http://fulltext.sblog.cz/)
      Entireweb Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
      BrightCrawler BrightCrawler (http://www.brightcloud.com/brightcrawler.asp)
      BabalooSpider BabalooSpider/1.2 (BabalooSpider; http://www.babaloo.si; spider@babaloo.si)
      WebRankSpider WebRankSpider/1.37 (+http://ulm191.server4you.de/crawler/)
      Gungho-crawler Gungho/0.08004 (http://code.google.com/p/gungho-crawler/wiki/Index)
      PWeBot Mozilla/5.0 (compatible; PWeBot/3.1; http://www.programacionweb.net/robot.php)
      PWeBot PWeBot/1.2 Inspector (http://www.programacionweb.net/robot.php)
      Exabot Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)
      Bloglines-Images Bloglines-Images/0.1 (http://www.bloglines.com)
      Doubanbot Doubanbot/1.0 (bot@douban.com http://www.douban.com)
      Disco-crawl disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com)
      Disco-crawl disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com)
      BotSeer Mozilla 4.0(compatible; BotSeer/1.0; +http://botseer.ist.psu.edu)
      ForAll.pl-Crawler ForAll.pl-Crawler/1.0
      Podtech Mozilla/5.0 (compatible; MSIE 6.0; Podtech Network; crawler_admin@podtech.net)
      MSRBot MSRBOT (http://research.microsoft.com/research/sv/msrbot/
      Nsyght nsyght.com/Nutch-0.9 (nsyght.com; search.nsyght.com)
      Backlink-Check Backlink-Check.de (+http://www.backlink-check.de/bot.html)
      ASAHA ASAHA Search Engine Turkey V.001 (http://www.asaha.com/)
      Sphsearch FAST Enterprise Crawler 6 used by Singapore Press Holdings (crawler@sphsearch.sg)
      Google-Adsense Mediapartners-Google
      SAIT sait/Nutch-0.9 (SAIT Research; http://www.samsung.com)
      Teemer Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.netseer.com/crawler.html; crawler@netseer.com)
      Euro-spider Euro-Spider Shopping 1.0
      Lovel Lovel as 1.0 ( +http://www.everatom.com)
      Hermits Search Mozilla/5.0 (compatible; Hermit Search. Com; +http://www.hermitsearch.com)
      ScoutAnt ScoutAnt/0.1; +http://www.ant.com/what_is_ant.com/
      Voyager voyager-hc/1.0
      De.com Mozilla/5.0 (compatible; de/1.13.2 +http://www.de.com)
      Yahoo Japan robot DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html)
      LijitSpider LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/; info(a)lijit(d)com)
      • Last Update: February 9, 2008 at 5:36 pm

No Responses to “Kaizeku Crawler Maps”

    • Anonymous's photo Kakkoi
    • RE: Kaizeku Crawler Maps - 'Commenting Guidlines' ↓
      4 months, 3 weeks ago on Tuesday, December 18th, 2007 at 9:12 pm 5 url
      0%

      The following "Code" are designed to protect you and other users of the site.

      • Be relevant: Your comment should be a thoughtful contribution to the subject of the entry. Keep your comments constructive and polite.
      • No advertising or spamming: Do not use the comment feature to promote commercial entities/products, affiliates services or websites. You are allowed to post a link as long as it's relevant to the entry.
      • Keep within the law: Do not link to offensive or illegal content websites. Do not make any defamatory or disparaging comments which might damage the reputation of a person or organisation.
      • Privacy: Do not post any personal information relating to yourself or anyone else - (ie: address, place of employment, telephone or mobile number or email address).

      In order to keep these experiences enjoyable and interesting for all of our users, we ask that you follow the above guidlines. Feel free to engage, ask questions, and tell us what you are thinking! Regular and insightful comments are most welcomed.

      be the first to comment.

    • Anonymous's photoAdSense
    • RE: Kaizeku Crawler Maps - 'Advertisement' ↓
      4 months, 3 weeks ago on Tuesday, December 18th, 2007 at 9:12 pm 5 url
      3

Have your say

  • Hint: Write as if you were talking to a good friend (in front of your mother).

Disclaimer: For any content that you post, you hereby grant to Kakkoi the royalty-free, irrevocable, perpetual, exclusive and fully sublicensable license to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, perform and display such content in whole or in part, world-wide and to incorporate it in other works, in any form, media or technology now known or later developed. Some rights reserved.