Apple details its Web crawler, possibly indicating a future push into search

Author

Owen Williams

May 6, 2015

This article originally appeared on The Next Web

Apple has quietly added a new page to its support site that details its first Web crawler.

A crawler is a bot, used by search engines such as Google and Bing, that automatically scans the Web for pages and information to add to an index and provides information when pages change.

Apple’s crawler, which existed previously but was never detailed by the company, identifies itself as “Applebot” and could indicate a future push into full Web search.

Right now, Apple says its crawler is used to add data such as movie information to Siri and Spotlight. You can exclude Applebot from searching your site by updating the robots.txt file on your site.

The company’s deal to use Google as the default search engine on iOS is reportedly due to expire soon and rumors have indicated in the past that Apple may be quietly building a search engine of its own.

Applebot

Image credit: Songquan Deng / Shutterstock.com

 

This article was written by Owen Williams from The Next Web and was legally licensed through the NewsCred publisher network.


Comment this article

Great ! Thanks for your subscription !

You will soon receive the first Content Loop Newsletter