1. SEJ
  2.  ⋅ 
  3. News

Apple Starts Crawling Web – Is Apple Launching a Search Engine in 2015?

Web development consultant Jan Moesen alerted the Twitterverse last week about the fact that Apple is running a web crawler, written in the programming language Go.

This is leading many to wonder whether or not Apple is planning to launch its own search engine in the near future. If so, that would be especially bad news for Bing who currently powers Apple’s Siri.

On his website, Moesen explains in more detail the information he has gathered so far, saying the first instance of Apple’s web crawler dates back to October 15th:

It looks like someone at Apple is running a web crawler written in Go.

The first hit I could find on <> was from on the 15th of October: – – [15/Oct/2014:03:05:42 +0200] “GET /robots.txt HTTP/1.1” 301 185 “-” “Mozilla/5.0 (compatible; Fetcher/0.1)”

All IPs starting with “17.” are Apple’s:

NetRange: –

A Twitter user by the name of Michael Mittelstadt chimed in saying he has been receiving intermittent amounts of traffic from Apple’s crawler, with short bursts of traffic followed by three week gaps.

That kind of pattern makes it look like Apple is running a search-related experiment, perhaps as R&D for an official search product

Apple is reportedly crawling mobile sites for the most part, while Moesen and others claim to have seen evidence that Apple is crawling the sites of those who have apps in the App Store.

Another possible explanation is that it’s tied to iOS 8 and Apple’s new desktop operating system, Yosemite, and their ability to access web search content.

Of course, everything so far is just speculation until we hear official word from Apple. What do you think it could be?

Category News
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Matt G. Southern, Senior News Writer, has been with Search Engine Journal since 2013. With a bachelor’s degree in communications, ...

Apple Starts Crawling Web – Is Apple Launching a Search Engine in 2015?

Subscribe To Our Newsletter.

Conquer your day with daily search marketing news.