There’s been a lot of complaining about Google’s search results lately, especially when it comes to high-stakes subjective queries. But no viable competitor has emerged despite many new search engines launching lately.

At least nothing good enough that people are excitedly sharing “Hey, have you tried X lately, it’s way better than Google.” What follows is an effort to explain what’s going on and a pitch for an open-source project I’ve been working on that tries to solve the problem in a new way.

What makes Google so hard to beat?

Google’s moat is simple but powerful — the more people use the search engine, the better it understand what the best answer to any question is.

They use signals like time spent on page and bounce rates to determine what pages best answer a question. Their ownership of Chrome makes this even more powerful. On pages where the search result page (SERP) has an inline answer, the user’s next query is a powerful signal of success.

With a 92% market share, Google has an order of magnitude more of this kind of data than any of it’s competitors.

If they have all this data, why are the results subjectively bad?

There’s a few reasons I can think of that hurt Google search quality, primarily in complex subjective queries.

  1. Time-on-page and bounce rates are actually not good signals to determine success for complex topics. I may spend 15 minutes reading a page about a complex topic and come away with a weak understanding of the topic, and a good page on the topic would also take 15 minutes to digest. Often, power users will search, command/ctrl + click the top 5 links and read through all of them for queries like this.
  2. Google is too large to take a stand on which publishers are “credible” and which are not. Google can’t just come out and say “hey, WebMD is fear mongering and UpToDate is a credible source for medical queries”, even though that’s what all the insiders in the field already believe to be true. The best is can do is try to organically rank one above the other. But this is difficult when the better source publishes more complex information that is harder to comprehend for most users (and thus results in higher bounce rates).
  3. Google makes most of it’s revenue from people making complex life decisions. Things like healthcare and financial decisions. A cynical suggestion may be that if the organic results were much better than the ads above, then significant revenue will be lost.
  4. It does not matter that results are bad in the “tail” (complex but rare queries) because it makes for a small percent of total queries and most users form search habits based on head queries, which Google is exceedingly good at.

Untitled

For example, Google invests heavily in short answer snippets and takes full advantage of their data superiority to stay ahead of the competition.

Untitled

Powered by Fruition