Add Reply
Add Comment As A Guest
Close Topic OptionsCrawl-19.cuill.com
Crawl-19 Cuill Com - Computer Issues, Video Gaming - Posted: 22nd Mar, 2008 - 9:06pm
Posts: 4 - Views: 10022
A Terror Bot!
Crawl-19.cuill.com
Crawl-19.cuill.com
I have been watching crawl-19.cuill.com for some time now. It belongs to a server that is part of an unknown Search Engine. Some of you may be experiencing spikes on the Forum because of it. Spikes are periods of high server usage where the pages seem to take longer to load than usual. I read elsewhere that it is a Search Bot that finds and indexes links like the google Bot does, but I have no surety about that. Do you have any information about crawl-19.cuill.com?
Sponsored Links:
Crawl-cuillcom
Here is some more of my findings. Their current site doe NOT show all the IPs that their crawler (twiceler) uses. I recently wrote an Email to them:
| QUOTE |
| Please stop crawling any and all domains under xxx.bordeglobal.com I also notice that you do not display all the IP numbers your crawler uses. Please confirm this request. |
My biggest beef with Cuill is this... crawl my site if you actually have a search database that people can use within the same standing as Google, Yahoo, etc. If not, stay off my server. When you go their site there are no search options.
Crawl-19.cuill.com Gaming Video & Issues Computer
Here is the response I got back:
| QUOTE |
| Dear Web Team, Twiceler is the crawler that we are developing for our new search engine. It is important to us that it obey robots.txt, and that it not crawl sites that do not wish to be crawled. Recently we have seen a number of crawlers masquerading as Twiceler, so please check that the IP address of the crawler in question is one of ours. You can see our IP addresses at https://cuill.com/twiceler/robot.html. I will add www.bordeglobal.com, www.bordeglobalimpactdesigns.com and www.bordeglobal.net to our list of sites to exclude and I apologize for any inconvenience this has caused you. Please let me know if there are other sitenames or IP addresses that you would like me to block. Sincerely, J. A. Operations Engineer Cuill, Inc. |
So, if you see a spider crawling your site that is not in their IP block then its not them. I'm still glad they are not on my site though, unless they are giving some benefit leave my resources alone.
Out of curiosity I replied:
| QUOTE (JB) |
| I am interested to know why you are crawling sites for months but at the same time do not have any real search facility, what is the point in that? |
Crawl-cuillcom
Here was the quick response:
| QUOTE |
| Dear Web Team, Like all startups, we hope to launch sooner rather later, but exactly when that will be, I don't know. Watch our web site (www.cuill.com) for the announcement. Sincerely, J.A. Operations Engineer Cuill, Inc. |
In checking though, they have been doing this for more than months, I think they have been indexing for at least a couple of years now. Your guess is as good as mine about when they will actually make their Search Engine available.
TOPIC: Crawl-19.cuill.com