Also known as a bot or robot: a computer program which runs through the internet collecting data, often for search engines.

An automatic function of some search engines that index a page, and then visit subsequent pages that the initial page links to. As the cycle continues over time, search engine crawlers or “bots”/“spid [..]

Automated programs in search engines that gather web site listings by automatically crawling the web. A search engine's crawler (also called a spider or robot) "reads" page text contents and web page coding, and also follows links to other hyperlinked pages on the web pages it crawls. A crawler makes copies of the web pages found and [..]

 Component of a search engine that gathers

The active first instar of a scale insect.

Synonyms droid, Web crawler, Web spider.

A program used by a search engine to “crawl” links on the Internet to find and index content. Also called a robot or spider. Can be used to identify and differentiate between types of crawlers indexin [..]

Dictionary of Internet Terms See spider

A web crawler is a program, often called a bot or robot, which systematically browses the Web to collect data from webpages.  Typically search engines use crawlers to build indexes. Learn more

A robot software program that searches "intelligently" for information on the World Wide Web, for example, one that looks for new documents and Web sites by following hypertext links from on [..]

A search engine’s crawlers are responsible for discovering new webpages and indexing them accordingly. Ensuring one’s content is visible to search engine crawlers is critical for SEO.

See Spider.

An automated application (also known as a bot, a shortened form of robot) that seeks and visits Web pages, not to view them but to collect information from them. Some crawlers analyze text content to [..]

A web crawler (also known as web spider) is a program which browses the World Wide Web in a methodical, automated manner. A web crawler is one type of bot. Web crawlers not only keep a copy of all the visited pages for later processing - for example by a search engine but also index these pages to make the search narrower.

Web crawler, a computer program that gathers and categorizes information on the Internet.

Automated programs in search engines that find and fetch Web pages, a process called crawling or spidering, and build lists of words and phrases found on each Web page.

See Spider

A crawler (or robot) is a search engine's program that visits (or crawls) many websites for the purpose of indexing these websites in their database.

A system that allows authors, artists, scientists, musicians and others to mark their work with the freedoms they want it to carry — a way of licensing the use of work.

A bot or piece of software that crawls the web to extract and create an index of data.

(n) a person who tries to please someone in order to gain a personal advantage(n) a person who crawls or creeps along the ground(n) terrestrial worm that burrows into and helps aerate soil; often surf [..]

A program that indexes pages on the World Wide Web for search engines.

A program that indexes pages on the World Wide Web for search engines.

  Also known as a Web Crawler, but sometimes described as an Agent, or a Bot. In essence a Crawler is a highly specialised search engine, designed to 'crawl' around the World Wide Web looking for particular pieces of information, addresses, references, etc., while the user is off-line, i.e. not connected to the Internet, and therefore not [..]

 A software program which visits virtually all pages of the Web to create indexes for search engines. They are more interested in text files than graphic files. See also spider, bot, and intelligent agent.  

The active first instar of a scale insect. Cremaster.

generally, the mobile neonate nymph of scale insects prior to settling and becoming sessile

A weak specimen found in most waves, usually considered a nuisance.

Automated programs used by search engines to visit web sites and index their content

(Australia,obsolete) A person who is abused, physically or verbally, and returns to the abuser a supplicant. (UK,Australia|slang) A sycophant.