Ahrefs reveals its new search engine: Yep

Did you forget that SEO toolset provider Ahrefs announced plans to build its own search engine in 2019?

If you answered “yep,” you aren’t alone. 

Ahrefs has been busy in those three years since, investing $60 million of its own money into launching a new search engine, called Yep. 

What is Yep? 

Yep is a general-purpose web search engine. Yep will soon be available in all countries and in most languages. 

Ahrefs is positioning it as a Googe competitor. However, we’ve seen plenty of Google competitors and Google “killers” come and go over the past two decades. So for now, let’s just call it a Google alternative.

So what is Yep banking on to become a true Google alternative? Two things: 

Privacy

Yep will not collect personal information (e.g., geolocation, name, age, gender) by default. Your Yep search history will not be stored anywhere.

What Yep will rely on is aggregated search statistics to improve algorithms, spelling corrections, and search suggestions, the company said.

“In other words, we do save certain data on searches, but never in a personally identifiable way,” said Ahrefs CEO Dmytro Gerasymenko. “For example, we will track how many times a word is searched for and the position of the link getting the most clicks. But we won’t create your profile for targeted advertising.”

What Yep will use is a searcher’s:

• Entered keywords.
• Language preference received from the browser.
• Approximate geographical area at the origin of the search at the scale of a region or a city (deduced from the IP address).

Profit-Sharing

The plan for Ahrefs’ search engine is a 90/10 profit-sharing model, where Ahrefs shares 90% of its advertising profits with publishers. 

The reason: Google displays content in its search results, without the need to actually click through to the website. That means websites are losing traffic. And for many sites, less traffic means less revenue. 

“Creators who make search results possible deserve to receive payments for their work,” Gerasymenko said. “We saw how YouTube’s profit-sharing model made the whole video-making industry thrive. Splitting advertising profits 90/10 with content authors, we want to give a push towards treating talent fairly in the search industry.” 

Here’s what Yep says:

“Let’s say that the biggest search engine in the world makes $100B a year. Now, imagine if they gave $90B to content creators and publishers.

Wikipedia would probably earn a few billion dollars a year from its content. They’d be able to stop asking for donations and start paying the people who polish their articles a decent salary.

There would be no more need for paywalls and affiliate links, so publishers who’ve had to resort to chasing traffic with clickbait articles and filling their pages with ads would be able to get back to doing investigative pieces and quality analysis. A citizen journalist uncovering corruption on the side of a full-time job could get compensated without having to spend time trying to monetize content.

And the best thing? You don’t have to be an expert to benefit.

Let’s say that you love pancakes more than anything else in the world. Now you have an incentive to grow that passion – imagine getting fairly paid to share creative recipes, publish photos of your creations and teach the rest of the world how they, too, can make the fluffiest pancakes ever. Independent creators everywhere will finally be able to flourish.”

All of that sounds nice in theory. But Yep is just launching.

DuckDuckGo, which launched in 2008, gets as many searches per year (~15.7 billion) as Google gets in about two or three days. Even Microsoft Bing – which is owned by Microsoft, the third-largest company on the planet by market cap – has failed to make a significant dent in Google’s search market share since 2009.

How Yep works

What really matters is search quality. That means Yep will have to satisfy the wants and needs of searchers. So how are they going about putting together those search results? 

Crawling

Yep collects website data using AhrefsBot. Ahrefs said it plans to replace AhrefsBot with YepBot in the “near future.”

AhrefsBot visits more than 8 billion webpages every 24 hours, which makes it the second most active crawler on the web, behind only Google, Ahrefs said.

For 12 years, AhrefsBot has been crawling the web. They had just been using the AhrefsBot data to power its link database and SEO insights, 

Indexing

The Yep search index is updated every 15 to 30 minutes. Daily, the company adds 30 million webpages and drops 20 million.

Other technical details

Ahrefs said its Singapore data center is powered by around 1,000 servers that store and process 100 petabytes of web data (webpages, links between them, and the search index). Each server uses at least 2x 100GB connections. Some servers use multiple GPU cards to train big transformer models. Before the end of the year, Ahrefs plans to open a U.S.-based data center.

More to come

I’ve reached out to Ahrefs to answer several more questions about their search engine. I will update as those answers come in. 

---