003 How Search Engines Work, Part One

The Transcript

How Search Engines Work, Part One

Demian Farnworth: Hey this is Demian Farnworth and welcome back to another episode of Rough Draft, your daily podcast that delivers the essential writing advice that you need to succeed online as a writer.

I’m your host Demian Farnworth, Chief Content Writer for Copyblogger Media and thank you for sharing the next four minutes of your life with me.

So this is episode three and we are calling this “How Search Engines Work.”

So we need to back up a little bit. And talk about how people actually find anything on the web to begin with. We have to face that ugly monster called obscurity. While you don’t need to be an expert on this — it helps to understand it.

How do people find things on the web?

The Two Things All Search Engines Must Do Really, Really Well

A search engine. You’ve know doubt heard of Google. The dominate search name. There’s also Bing. Then smaller boutique search engines that specialize in narrow fields. Academic, medical.

The job of a search engine like Google is to find content that matches your query — the question you are basically asking: How far is the earth from the sun? Who is the leader singer of Led Zeppelin? What is a freemason? The better it can match the better match your question to a good answer, the better the experience.

You probably use it on a daily basis. If not an hourly basis. More likely a per minute basis. Right? You and your friends are discussing a subject — someone disputes you — and so you slide out your phone and say, “Let’s end this right now.”

But have you ever wondered how that page which confirms you are right and they are wrong ever gets there? How does Google know this content and not some other?

A Short Introduction to the Cute Little Search Robots Known as Crawlers

This is how it works.

Search engines send out these little robots that travel along links — these robots — called spiders or crawlers crawl through your words on the page and make a judgment about what your page is all about and then stick it in this library card catalog, the librarian (Google) delivers the information to you.

Now this crawling and indexing happens fast. So fast it is mind boggling. Do you remember back in the day when Google used to publish on their home screen how many pages they indexed?

“We have indexed 4,556,797,934,111 pages today. Please. Stop publishing.”

They can’t do that anymore because it would be meaningless to us.

All these pages they gather and then deliver to us is based on a ranking system. They just don’t grab random pages and shove them your way. They are very deliberate about what they give you. This is based on a couple of things and that’s what we are going to talk about on the next episode of the Rough Draft.

Thank you. See you then.

Rough Draft is brought to you by Authority Rainmaker, a carefully designed live educational experience that presents a complete and effective online marketing strategy to help you immediately accelerate your business. Don’t miss the opportunity to see Dan Pink, Sally Hogshead, punk legend Henry Rollins, and many other incredible speakers live … not to mention the secret sauce of it all: building real-world relationships with other attendees. Get all the details right now at rainmaker.fm/event, and we look forward to seeing you in Denver, Colorado this May. That’s rainmaker.fm/event.