The Truth Behind “Doorway Pages” and Why They Matter for SEO, with Duane Forrester of Bing.com

Last week, the Webmaster Central team at Google announced that they will be rolling out an algorithmic change that addresses “Doorway Pages.” Huh?

You may ask yourself “What’s a Doorway Page?” If so, you’re in good company. Many people haven’t heard of them.

Doorway Pages have not been a part of the overall search equation for years, but now Google is redefining what they consider to be a Doorway Page, which is a little vague.

Google does loosely define what a Doorway Page may be based on a particular set of questions:

• Is the purpose to optimize for search engines and funnel visitors into the actual usable or relevant portion of your site, or are they an integral part of your site’s user experience?
• Are the pages intended to rank on generic terms yet the content presented on the page is very specific?
• Do the pages duplicate useful aggregations of items (locations, products, etc.) that already exist on the site for the purpose of capturing more search traffic?
• Are these pages made solely for drawing affiliate traffic and sending users along without creating unique value in content or functionality?
• Do these pages exist as an “island?” Are they difficult or impossible to navigate to from other parts of your site?
• Are links to such pages from other pages within the site or network of sites created just for search engines?

Loren Baker and Duane Forrester of Bing.com discuss the concept of Doorway Pages, what they are, and how some pages could be considered by Google as Doorway Pages.

What kind of pages might you have on your site that Google could consider to be Doorways? Landing pages generated for paid search campaigns … or content mashup pages for an email campaign could be considered Doorway Pages.

And the bottom line is, if you are creating pages for non-SEO efforts, then they should not be indexed in Google or Bing anyway (which really makes the pruning of Doorways somewhat similar to an expansion of Panda).

Duane and Loren talk about these examples and also ways to prune back a site using Robots.txt and different Google & Bing Webmaster Tool options.

Furthermore, the discussion gets even deeper into the reasons why some SEO workarounds exist, especially in environments when SEO changes cannot be made to the CMS due to development restrictions, and the pressure of ranking — both for the in-house team or the agency — lead to implementing bandaids on the site such as optimized pages that exist off of the normal CMS; which could be considered Doorway Pages by Google and targeted in this update.

The Truth Behind “Doorway Pages” and Why They Matter for SEO, with Duane Forrester of Bing.com

Loren Baker: Hello and welcome to another episode of Search & Deploy. This is your host Loren Baker, and on Search & Deploy, a Rainmaker.FM podcast, we discuss news in the world of SEO, SEM, Google, Microsoft, and Yahoo, and kind of cut through a lot of the noise to determine what’s important, what’s real, and how to implement those real happenings into your overall digital strategy.

With me today I have a special guest, Duane Forrester, the senior marketing manager at Microsoft Bing. I’m going to bring Duane on in a moment, but I want to go over some of the topic matter that’s going to be discussed today.

Earlier last week, Google rolled out an announcement on the Google Webmaster Central Blog saying that they are going to be unleashing a new algorithmic update targeting doorway pages. I’ve not heard the word ‘doorway pages’ for about 10 or 15 years, but Google has made the announcement that this is going to be something they’re targeting, and they’re making updates to the Webmaster Guidelines accordingly.

Is the purpose to optimize for search engines and funnel visitors into the actual usable or relevant portion of your site, or are they an integral part of your site’s user experience?

They asked a series of questions for webmasters to think about. The first question is ‘Is the purpose of the page to optimize for search engines and funnel visitors into the actual, visible, or relevant portion of the site?’ Meaning, is that page living outside of the rest of the site only for search purposes, and then bringing the user in to make a transaction? They asked a lot of other similar questions in terms of are the pages using duplicate content, or aggregating content from the main site itself only to appear in search? Are these pages made solely for drawing affiliate traffic, and then sends the user over to a site to make a purchase where the owner of a doorway page makes a commission revenue off of that?

Very interesting news to come from the way of Brian White on the Google Webmaster Team. Also this pretty much has SEO scrambling. Not that setting up SSL on your site was enough, or getting ready for the April 21st mobile update on the Google side, but now webmasters have something else to worry about. Like I said before, I haven’t heard the word ‘doorway pages’ for almost a decade. On that note, I’d like to introduce Duane Forrester the senior marketing manager of Microsoft Bing. Welcome to Search & Deploy, Duane.

Duane Forrester: Thank you very much Loren, it’s a pleasure to spend some time with you guys today.

Loren Baker: Yeah, really excited to have you on for our third episode. We made it to No. 15 in the iTunes podcast marketing category, right?

Duane Forrester: Excellent, well done.

Loren Baker: Yeah, so my goal is to get to No. 10, and then No. 1 maybe one day.

Duane Forrester: I was going to say. You’re setting your goal at No. 10, awesome.

Loren Baker: Yeah, low.

Duane Forrester: Come on, we can all do better than that for you.

Loren Baker: I’m No. 10, we’re No. 10! Like I was saying, I think the last time I actually used the word ‘doorway page’ was when I was optimizing for Alta Vista back in the day. It’s such a broad and vague term — and I don’t want to necessarily get into what’s happening on the Google side of things — but I really want to talk with you about doorway pages as a whole. What do you feel that they are now, and how did you feel that they were back in the day and whether this is a broader thing.

Because it was really interesting what I just brought up — we’re talking about websites that are utilizing content aggregators from maybe their product pictures, maybe it’s multiple blog posts. We’re talking about affiliates. We’re probably talking about commercial sites as well. In your professional opinion, given your years in this industry, what do you consider a doorway page? More frankly, what was the last time you actually heard this term used?

Duane Forrester: I’ll do these in reverse order, Loren. Like you, it’s been a long time since I’ve heard any conversations about doorway pages in a meaningful way. Random large companies will contact me and I’ll sit down with them and have conversations, and you still hear them come up every now and then like, “We have some doorway pages we’re taking down, and we’re just going to focus on optimizing our individual website pages,” and so on.

Loren Baker: “We’ve inherited doorway pages.”

Duane Forrester: Yeah exactly. It’s always mysteriously part of a legacy, which just goes to show you that it’s old. The implementation was older. It came from a time when, at first, we weren’t necessarily calling them doorway pages. It was just that landing experience that was customized for very specific things.

When we started seeing changes in the paid world, when the quality of the landing page started to count a lot more and the algorithms got smart about it, we started having savvy SEOs look at that and say, “Hang on a second. There’s an algorithm that’s able to actually look at a landing page and PPC and say, ‘You know what? This meets a quality quotient.’ Maybe we should start designing individual pages for SEO in the same way: with that very specific focus on one very specific message. Optimize the heck out of it. Get it ranked, and have it drive people into whatever it is we’re trying to do. Convert on a product, sign up for a newsletter, consume pages, advertising, whatever.”

Loren Baker: You’re more or less talking about a landing page, right? That kind of hangs off of the main site architecture on its own little dangle thing.

Are the pages intended to rank on generic terms yet the content presented on the page is very specific?

Duane Forrester: Exactly. That was one of those early stage, doorway development-type things. Then people start realizing, “Hold on a second, we could actually do a lot of pass-through stuff. If my page appears to be super relevant for ski bindings, well then if it’s loaded with content from retailers to do with this it’ll rank well, and I will get that affiliate value from it.” Given the state of the algorithms, given the state of searchers at the time and their desires, it wasn’t a bad approach per say.

As with everything, there were edge cases on both sides where some people could have taken more advantage of it, and other people did take more advantage of it. I think what’s happened is, there’s elasticity to the world of online marketing, right? Where the things that are current and fresh today that we’re all working on seem to kind of emanate from the United States, for better or worse.

As everyone else, and this is even within the United States, there is a small group of people that are very, very intrinsic to the core of the industry. They are experts. They know exactly what they’re doing. They’re always on the cutting edge. They are able to guide clients. They are able to attract clients. They are successful with their own websites. Then you have another layer of people that are newer, and not quite as well-versed in all of the complexities of building a digital business and operating it.

They’re still learning. They’re still picking up on some of these tricks and tips. Unfortunately what happens is, when you go, or when I go out, when all of us go out that have all of the grey hair (or any hair in your case) from all of this time on the Internet, we’re producing all this content. We’re telling people all these things that work today.

Loren Baker: It’s funny that you bring that up. I have had updates pop up in my email from blog posts that I wrote three years ago, seven years ago, about buying paid links and buying blogs and stuff like that where people are like, “How dare you make this recommendation?” Did you look at the date stamp? I mean, come on. I’m not going to change it because it’s archived knowledge.

Duane Forrester: Exactly.

Loren Baker: You can update it but not change it.

Duane Forrester: Yeah, but it still serves a purpose for your business because people find it. It is an entry point. Maybe they disagree with it — they ping you, you have a conversation — there’s still value there right? I think this is at the crux of all this, where there is a latency that happens with knowledge sharing.

I see this when I travel internationally. I go to conferences and I’ll sit on stage with folks, and they’re talking about things and I’m thinking to myself “Hang on a second we were dealing with that six months ago or a year ago, and you’re talking about it like it’s the fresh new thing.” There’s a latency that happens — I believe it happens globally, and it happens vertically within an area as well. You’ve always got some fast movers and early adopters in any given area, and they will have the freshest information. And then they have to disseminate out and people find those old blog posts that they made and so on.

Do the pages duplicate useful aggregations of items (locations, products, etc.) that already exist on the site for the purpose of capturing more search traffic?

What ends up happening is you have people that are discovering these concepts of doorway pages. They make complete sense. Rationally speaking, if you’re looking to grow a business and your goal is to have a retail-based business and actually drive revenue from affiliate programs or direct sales or whatever it is, the concept of the doorway page fits beautifully with that model. You’re looking at it and saying, “Search will only be a portion of my marketing plan. I will also do email list building. I will also do social media.” Whatever it is.

Loren Baker: If you’re setting up those pages for paid, or for email landing pages, it’s also the job of the SEO or the webmaster to prune and control things within robots.txt and the various Webmaster Tool platforms to make sure that’s not being indexed.

Duane Forrester: Exactly.

Loren Baker: What has me kind of thinking about this is this is an extension of previous updates, right?

Duane Forrester: You have to understand too that, for Google, for us, for any engine to take an action on something, there has to be prevalence. It’s not because somebody woke up one day and said, “Do you remember the concept of doorway pages? We should probably protect the Internet from that now.” It’s not like there’s drunk superheroes cruising around.

Loren Baker: “I had this nightmare last night about doorway pages.”

Duane Forrester: Yeah, “Where do I apply my superhero?” Or randomly, “Let’s do this!” It’s not like that, there’s obviously a reason for it because for any work to be done is a significant cost to a business. Developers, programmers, program managers, marketing people — everyone is involved. We’re talking about human beings with salaries and benefits, and a hard cost to that business to make that change. They’re not doing it because they felt like it, they’re doing it because there’s an intrinsic value behind it. That’s how all of our investments move forward.

Now I think from a webmaster’s standpoint, you bring up something that is incredibly crucial. A lot of SEOs have this thinking and it’s this throw it over a wall thinking which is the, “My job is to get traffic here. That’s my job. Better ranking, more traffic, and off we go. I’ve done my job. The conversion stuff, not my thing. That’s not up to me, that’s up to the product manager or whomever, right?”

The savvy among the SEO community will take on the conversion work as well and realize that they can be true heroes by taking a step in that direction, learning and applying. What we tend to have happen is a lot of people will look at it and say, “Okay we’re producing content that’s great,” but there’s grey areas. It’s not like it’s a black and white photo that you’re working with, which is how people tend to look at it. There are 256 shades of grey. You can produce a very rich diagram if you use all those shades of grey.

SEOs tend to think in terms of it’s black or it’s white — we’re going to produce the page and publish it or we’re not. Your point is exactly right. There are completely valid reasons to produce pages for any number of reasons: back up a marketing campaign, it works in social driving scenarios, whatever the reason is fine. But control that stuff so that the search engines are getting the signals.

Are these pages made solely for drawing affiliate traffic and sending users along without creating unique value in content or functionality?

Loren Baker: The thing is, Duane, I would not send a page that I put together for SEO purposes to my email newsletter.

Duane Forrester: Right, exactly.

Loren Baker: I would not use that page — an organic page specifically made for SEO, typical visitors, typical user experience — as part of a PPC campaign.

Duane Forrester: Right.

Loren Baker: I want to make sure that page and PPC or an email, speaks directly to that audience and converts. Then why would you want to do the opposite and open up those kinds of pages to the Google index?

The one thing that’s interesting that you brought up a couple times while you were speaking I realized that I have worked a little bit in terms of doorway pages, but not what I used to consider a doorway page, which was pretty much a page made of spam, a search engine that sent someone over to a transactional page.

More so are the limitations that are sometimes made from a CMS and IT perspective, and how those limitations basically block or minimize the success that organic search or SEOs can implement. If you’re in-house it’s one thing, but if you’re an agency and you’re hired to do an audit, and implement that audit, and part of that relationship is making change and improvements in the rankings.

What happens sometimes is, especially if consultants don’t do their homework, is that audit is going to sit on a desk and gather dust for years, right? Because it’s not going to get implemented by IT. It can’t get implemented by IT. There are other priorities, or maybe marketing does not even converse with IT. Then what happens typically is someone says, “Hey, if we can’t do this, do you mind if you CNAME the sub-domain over to another host so we can set something up? Can you at least give us a sub-directory to work with, and maybe we can throw some content up on there? How about if we set up some landing pages via the WordPress blog?”

What happens is there is this demand and stress sometimes put on internal in-house teams and consultants to get the job done. It means that you’re not going to make your goals or bonuses internally, and it also means that the consultant might be cut. Basically it’s because of you that they couldn’t do their job correctly.

What’s set up then is typically a section of the site that hangs off the rest of the site, and I experienced this in some pretty large global insurance companies, where you can get nothing through from an IT perspective. Also there’s a lot of retailers that still have this on their site, because stuff shuts down in October to get ready for the holidays, but you still have the pressure to get something up, right? You end up changing these pages that are hanging off of these doorway pages or optimizing them. I just never considered them doorways. I guess I considered them more like orphan SEO pages, or the red-headed stepchild of the rest of the website.

Duane Forrester: Yeah, one of the things too I think that’s important for folks to keep in mind is this concept of pass-through, right? I like to think of it in terms of pass-through versus retention. What’s the page’s job? If the page’s job is to answer a question, then a search engine wants that in the index because it’s a good result to show to a searcher. Somebody comes in, they ask their question, the page ranks well. The user goes to that — or the searcher goes to that location — and they’re satisfied. Ultimately that means they’re satisfied with the search experience as well, the search engine. That’s really the metric that the engine is going for: satisfaction of searcher.

If, however, that page is designed to simply get you to click through to something else, well then given we know the totality of the Internet, why would we need the extra click between the searcher and the product on the other side? If you actually start pulling out those types of pages, then the actual product page surfaces.

When the searcher asks for the product, they get the actual product page — one click to what they’re looking for, versus this intermediate click where somebody gets to make a dollar. Now, there’s nothing wrong with making a dollar, but if we’re talking about our job is to satisfy the searcher, adding extra clicks is not a way to drive up satisfaction. It is a way to drive distraction and dissatisfaction. Obviously we would prefer ranking things that are direct to source.

Loren Baker: So if I’m in-house or with some of my clients, before I get into Chicken Little syndrome where I start running around and everything is going to fall apart, I see this as more of an opportunity to be able to make that case internally. To be able to open up — if for some reason the in-house SEO only had limited access to releases, or limited access to the CMS, or no ability to do things like add canonical tags to the site, or take care of duplicate content issues and things like that — I see this as being an incredible opportunity, especially for enterprises, because that’s typically where most of this happens.

For smaller businesses, log into WordPress. Do a change here. Do a change there. It’s all good. Hit publish. But on the enterprise side where a lot of this does happen because of those restrictions, I would see this as an opportunity. Especially if search engines as a whole are focusing more on not serving these pages in the future because it’s making the user jump through more hoops to find what they’re looking for at the end of the day.

Duane Forrester: Ultimately this comes back to that shades of grey conversation right?

Loren Baker: You lost me at shades of grey.

Duane Forrester: I’ll fill in the blank for you. The reality here is that a lot of times the SEO world ends up being black or white. It’s publish, don’t publish. Publish this type of content versus that type of content. Marketing wants one thing, email wants another thing, and PPC demands something else. It puts an SEO in the situation of having a lot of ‘yes/no’ conversations, and then losing. And these things are still out there.

Loren Baker: Right.

Duane Forrester: What the SEO still can control and guide the organization on is the subtleties, the shades of grey on, “Look, that’s okay, go ahead and publish the page, but we’ll block it in robots.txt, or we’ll put something in meta tag that blocks this from being called.” We all know that Google Bot will just crawl it anyhow. That’s not going to stop it. It’s about signaling intent. So if you tell the robot, “Look, our intent is that you don’t index this.” Okay, well then you’ve given all the signal you can that this is not a page that should be surfaced in front of searchers.

Bing Bot has the opposite reaction — you tell it not to crawl, it doesn’t crawl — but it’s the exact same intent. My intent is that this is not in the index and shown to searchers. The semantics on whether something gets crawled or not is beside the point here.

The point is you’re specifically saying, “You know what? My PPC landing page over here, the page that’s designed specifically for my email program, which I am mystified at how you found it to begin with, but you found it. It shouldn’t be in the index, you shouldn’t be looking at that, and therefore you should be looking at my domain as a more thoughtful domain all up, because I am carefully monitoring what I think you should be indexing versus what I know you would be indexing. I’m giving you those signals.” The algorithms respond to those things, and that makes a difference over time.

Loren Baker: Saying that there is no malintent, there is no bad intention with setting this up. Now if people link to it, from the Google perspective, even if it is blocked in robots.txt, and people do link to it, it will rank. There are options within Google Webmaster Tools where you can try to take that out and everything else or block parameters, but the message to Google is that, “Hey we are not setting this up to trick you or to trick users that are searching you to find this information.” Right?

Duane Forrester: Exactly. It’s in the similar vein of the disavow tools. The whole point behind the disavow tools is to give the webmaster the control to say what their intention is. In some cases, there’s always going to be the people who have good intentions who ended up with links pointing at them but have no idea that the tool exists. They don’t know what to do with it.

We still have to make determinations on those things. But the fact of the matter is, if you’re an active webmaster and you believe that you have a problem and you’re watching on these things, there is a facility to tell us that that your intention is that those paid links — those crab links that are pointed at you — those are not things that you would otherwise choose to go and solicit on your own.

Of course all of this falls under the off-page SEO umbrella that you really can’t choose who links to you. That’s just not possible. This helps the engine understand the intent. Over time, that intent starts to matter because the person who’s faking it will give up on faking it and then try to game the system because that’s their intention. The person who’s actually thoughtful about it, is a true SEO, is doing true work, and works for a legitimate company, who is doing the right things is going to look at that and say, “Great this is a good safety net and I should send out those signals.”

That’s the true value of SEO today. Understanding how to help a company balance its portfolio of assets on the web so that the right things are being given to a search engine and the things used for other purposes are actually giving off signals that say, “Don’t bother with me, I’m not part of the search equation. I exist on the web for another purpose.”

Loren Baker: Yeah, it’s funny too because I can’t help while we have this conversation thinking in the back of my head, “Over at Foundation when Boser and I are talking to leads or possible new clients, we really, really try to get as much information from them as possible. What can be done? What can’t be done? How does your IT team work? What are the internal lines of communication? Can we do an onboarding beforehand? Can we get this questionnaire filled out?” So we’re not put in a position to fail basically.

A lot of my friends are in-house, and some have great opportunities where what they say happens: the code on the site is clean. Everything is working, functionable. There are no hiccups. There’s no duplicate content, and there’s a mutual respect between not just IT but also like you brought up before, social, email, analytics, the CMO. The company is working together.

Duane Forrester: There’s true integration. It doesn’t happen everywhere.

Loren Baker: There are situations where that’s not a fact. Where I’m getting texts in the middle of the night, “Hey you know anybody that’s hiring?” It is pretty interesting, and I kind of see this as, if there is a straw that can break some camel’s backs in terms of the quality of life in that SEO position and the ability to be that well-rounded, good-hearted SEO that you’re discussing, this could be one of them.

So, if I have a website and I added on a doorway page Band-Aids in the past along with other Band-Aids to be able to compete with my competition, and I want to get that cleaned up (you had touched on disavows and things like that), how does that work on the Bing side of things?

Duane Forrester: You know what, it’s really straightforward. And this is advice I give to a lot of businesses, startups, and everyone. You truly have to understand what your business model is and what you’re trying to accomplish. You need to have a website built that actually will enable that, puts you on the right path toward that success. If you’re thinking about, “Okay we need to add something,” or “we hadn’t planned for that, let’s just create a subdomain,” or “put something over here in a random folder.” You need to take another step back and ask yourself, “Is this the right platform? Are we going in the right direction?”

So many people try to start businesses with WordPress websites, only to find out later on that selling through affiliate links isn’t what’s going to make money. They want to have actual sales experiences, deeper dives, product pages and so on. They’re trying to do all of that through a plugin in WordPress as opposed to building out a proper e-commerce platform.

Then e-commerce platforms have their challenges with SEO and managing everything, so there’s a lot of big pieces there. But it’s important to understand exactly what the flow of the business is going to be so that you’re on the right path to begin with. Because if you try to go into an e-commerce platform and say, “Alright now we’re going to add a blog to it. Oh, it has a facility to do that.” You know what? That facility to do that is probably not nearly as good as what WordPress can offer you.

If you have to make the choice to use the right tool for the job. This actually falls under this concept of adding these pages to accomplish very specific things. You have to ask yourself, what is the point of the web page I’m on? If the point of the webpage I’m on is to drive traffic and convert for sales, then perhaps you actually need conversion work, and you need to reorganize the page layouts, the verbiage on the them, the positioning of things on your page — the usual conversion optimization work. As opposed to, “Let’s put a link in here and direct them off to another page.”

Hey, you know what? I get it. Sometimes that is the solution you have access to. It’s not as easy as going back and re-architecting something. The solution that you need to move forward with is, ‘We are creating a new section. It is going to have all of these pages.’ You need to get that stuff noted in the robots.txt file that you don’t want that crawled. Keep the crawler out of it, just /folder/, and put it all in there.

Make it easy on yourself for maintenance and forecasting in the future. You may decide to just junk that entire area, no easier way to do that then put it all in a folder and nuke the folder five years from now when you don’t need it anymore. Doing those things — very important. If you actually want to, you can go into disavow tools and start taking away those links essentially that are pointed at those pages.

Do these pages exist as an “island?” Are they difficult or impossible to navigate to from other parts of your site?

Duane Forrester: The search engine is kind of like that person who walks into a shopping mall and sees a crowd of people standing around a store window. We’re going to mosey on over, we want to know what the commotion is. Why is that popular? We want to see what it is. If it’s a consistent flow of popularity, well then we’re probably going to try that in search to see if it resonates with searchers and if it satisfies searchers. That could be because you blocked a page for a new product that you’re offering hoping that it wouldn’t get shown in search results for a week until you were ready.

Okay, well if that’s your plan, the only suggestion I have for you is do not put that page into production until you are ready. Wait the week, because there are no secrets on the Internet today. Long gone are the days of testing something live on the Internet and then releasing it to be crawled. Those days are so far behind us that inadvertently you could be creating these scenarios for yourself, and so now you have to plan actively, “how do I protect myself in that case?” It’s about admitting the signals.

Loren Baker: We also like to call that popping the cork. When you’re setting up a new part of the site — a new page or a new section — and you block it from the Google or perspective, then you start getting some links to it and you start marketing it. You see it kind of crawling up, kind of rising up in the Google rankings with that default URL or whatnot as a title, and the little robots.txt message in the description. More and more people are interested in it, and you see it crawling, climbing, climbing, climbing. And then when it starts to hit that peak of popularity, that’s when we open it up and we just see that sucker cached and indexed, and up in the rankings in a day or less because of all the popular signals going to it.

One thing that you do bring up is that there’s not only an issue with the live testing. Have you ever been in the scenario where you have someone email you or message you and they’ll be like, “Yeah we launched our blog two weeks ago and it’s not ranking yet.” You look at it and you’re saying, “Why isn’t this thing ranking?” You look at the robots.txt and that’s fine. You look at everything else and that’s fine, and then you look at the source code and you see that “no index no follow” is still in the code.

So it’s little things like that that happen, but I do agree. I was looking at a Volusion site the other day, and by default there were a lot of duplicate content and wildcard pages that are open to be indexed. This was a site that someone, a friend, was setting up that was live. I’m looking at it I’m like, “Wow so you have a five page site that has 150-odd pages?” That’s interesting. You’re right. There is no live testing, and there’s no reason to live test. Live test being opening up that section to the search engine.

Are links to such pages from other pages within the site or network of sites created just for search engines?

Duane Forrester: Yeah. I mean today I think there’s enough tools available, there’s enough facilities available to understand what it is you’re trying to accomplish before you launch, that you can legitimately do it. Like the example you’re citing right now, the Volusion site. I have never run on the Volusion platform, but I can almost guarantee you in the community somewhere someone has a standardized robots.txt you can copy and paste to get rid of 90 percent of those problem pages that are just crap. You don’t want them in there.

Loren Baker: Right. It’s like WordPress used to be when it came out.

Duane Forrester: Exactly. I still have, the facility in WordPress, that ping facility where it will ping all these services.

Loren Baker: Yeah.

Duane Forrester: I still laugh at it, but as a part of my workflow every single time I set up a WordPress site, I have my original ping list of about 40 or 50 different services that I’ve been using for the last decade, and I just take them and paste them in there and off it goes. It provides probably zero value at this point. However if anybody’s actually using the other side of that service, why not copy and paste it in there?

Loren Baker: Why not?

Duane Forrester: I have similar things for my robots.txt files when I launch a site. I have standardized things that I put in there specifically to do certain things — get rid of my admin page, block the login page — all those standard things. There’s no point in the search engine having it. It doesn’t do them any value. And I would like to send the signal that I know that, and that you shouldn’t have that.

Ideally the engine will filter it out anyhow. You go look at my personal blog. You shouldn’t be seeing the login page as the second result in there. It just shouldn’t be there. That’s only for me, it’s not for anyone else. Doesn’t need to be surfaced. But hey, I do have a responsibility to kind of maintain my own space here. I think this is where SEOs provide a staggering amount of value to businesses today and will continue to the future, is in these details. In the sculpting of the message that gets sent to the algorithm. It’s no longer just about, “Here’s an optimized page. It should outrank my competition.” It’s more about, “Here’s my optimization mindset and how it reflects on my domain,” and therefore that feeds toward authority and trustworthiness and so on. That’s the value that SEOs provide today.

Loren Baker: The thing is too is, I don’t mean for this to be an ad for Rainmaker or Genesis or what not, but I was thinking when I mentioned WordPress when it was right out of the box when it launched compared to now, at least now you have the ability with the Genesis platform and other tools and plugins — like they host SEO for WordPress plugin –. You have the ability to block the indexing of category pages from Google. You have the capability to control pagination. I don’t think of any of them yet remind you if you forgot to update your About Us page, it still has that same message.

That’s one of my favorite things, running a search for “just another WordPress site” and seeing how many pop up. There are those capabilities now. And with all of the landing page plugins out there — whether it’s Premise which is now part of Rainmaker and some of the other tools — at least now they do give you the capability to block those paid search landing pages from the engine, or even open some up if they make sense from an organic conversion perspective.

Duane Forrester: Absolutely.

Loren Baker: Part of the user journey. It is a little bit easier but that is on a platform by platform basis.

Duane Forrester: I agree. I actually personally use Genesis for my newer blogs that I’m maintaining, and I love the simplicity of the interaction. I’m fine going in and monkeying around the code, but why? I can just copy and paste. It’s super simple. But more important to me is that control, is the ability to turn off the things that I don’t think you need to be indexing. What I want is for you to index the individual page. The category page — absolutely pointless. It’s just a list of links, and there’s no need for you to have it because you should have my individual links is what you should have.

Loren Baker: The worst thing is when you search for something, right? Then you find a category page appear in the search, but you don’t realize it’s a category page and that it was cached a week ago. You’re doing like a Ctrl find on the page when you get there, “Where is this?” It’s not part of it anymore!”

Duane Forrester: Yeah. Part of the equation here that folks need to really keep in their mind too is the context of recency, the expectation of the searcher to be looking for things that are current for answering their questions essentially. That’s kind of like a default state in the human mind. I’m old enough that when I ask you about tying knots, I’m not asking you about how to tie my shoes. It’s a different scenario, and so I want things that are more current. And in fact I welcome responses back to queries like that that are things like, “Why tie your shoes? Here are five shoes that are slip-on.” Because you know what? This is a fashion statement and I like the look of this, and “Yeah, why am I tying my shoe?”

Loren Baker: Adults can wear Velcro, right?

Duane Forrester: Exactly. The ability to control what’s being exposed I think. The challenge that we have today is that people don’t understand it necessarily, so they don’t do it. Those are the nuances that folks need to really understand. If you’re going to hop on to a platform, whatever it is, you need to understand the facilities that it offers and what each one of those buttons and options do for you. They’re in there because they fulfill a specific need.

You have to determine whether you have that specific need or not. If you’ve been doing this for any length of time, you have a default. It’s muscle memory for you. I automatically do this, I do all my settings, and then off you go. But for folks that are newer — and this is where ultimately as we originally started that these concepts tend to perpetuate over time — is you find something that’s three or four years old, it’s from a trusted source. It sounds legitimate. It’s got lots of good positive commentary in the comment section, and you think, “Awesome!” Good resource, bookmark, and I’m going to learn from that.” That’s fine. But take the extra step, ping the resource and say, “Is this thinking still current? Should I still do it this way? Are you still doing it this way?” Because I can tell you right now, when I first learned how to do link building, I called up this guy named Eric Ward. If folks don’t know who Eric is, he’s seen as the godfather of link building.

Loren Baker: Moses, right?

Duane Forrester: Exactly, Link Moses. Eric was the guy who essentially kickstarted the concept of link building for Amazon when they were getting started. Huge, huge resource in terms of digital marketing and understanding these things. Eric very clearly described to me a process for being able to solicit links for a business, and it was very old school. You track down the person, you send them a note, and you ask them permission to engage. You were very polite about it, and very nice and respectful. I found that to be an incredibly useful way to talk to other webmasters and actually exchange complimentary links, or suggest myself in their systems and whatnot.

Today, people come and they talk to me about link building, and I’m like, “You’re already thinking the wrong way. You’re already in the wrong direction. It’s not about link building. The links will come to you. The question is what are you doing to deserve them? You need to put all of your time and effort into answering the question: ‘why do I deserve to be linked to?’ Not, ‘Now that I built something, trust me it’s wonderful, people should point links at it.’ No, we’re way past the if-you-build-it-they-will-come stage.”

Loren Baker: I tell this story a lot, and it’s kind of like one of those old man SEO stories. Basically when I got started, for my first job I was an intern in college in 1998, and I was working at a check printing company. Remember checks? Like personal checks.

Duane Forrester: I remember them well, yes.

Loren Baker: Probably write at least one a year now. They used to be a big deal and people loved the personalization with their checks, because when you went to the supermarket you would write a check, when you went somewhere else you would write a check. There were wrestling checks, there were auto racing checks, there were teddy bear checks, there were model train checks, so my job was not to build links for that company, but to do their on-site SEO, and advertise or market that company.

Part of that marketing was — back in ’98 and ’99 — finding like Geocity pages, fan pages, similar type sites, and building a relationship with them. Sending them an email, and getting to know that person, and telling them, “Hey I’m not sure if you saw this or not, but yada yada check company has just put out a new line of model train checks.” And I would get an email back in about five minutes saying, “Wow! Thank you so much! No one ever writes to my site. I can’t believe you guys are looking at it! I put this link up already. It’s in the navigation,” type thing. It really is like that’s the same kind of mentality that I use on a scale basis now that still works.

Duane Forrester: In some instances it is still a very valid approach. Like so many things that are just good quality marketing, are still good quality marketing, no question about it. You do however, have to think a little more about the long term repercussions of it and the downstream effects of what you’re doing, because you reach out and you establish a relationship with somebody, that’s fantastic. If they’re so excited that they’re constantly dropping links to you, and they’re constantly referencing you, it could be as innocuous as you now you have a personal stalker that you have to deal with. This person’s grateful and they appreciate the engagement. It could be as complex as the person that you’re talking with actually has a network of websites and decides that you’re a good enough resource, they’re going to …

Loren Baker: Drop a link on the sidebar of each one, right?

Duane Forrester: Exactly.Suddenly it’s like, “Hang on a second now. I reached out to do you a solid and to say something nice about you and to cement a business relationship. Now I’ve got to go to a link disavow tool and say ‘look I like you, but I don’t love you.’”

Loren Baker: Right. Or next thing you know they’re getting an email saying please take down this link immediately and being followed up on a daily basis, and they have no idea why.

Duane Forrester: Exactly. You just have to be careful with some of these tactics. You have to understand the reality of what the downstream looks like from them. Again the value of good SEOs. I mean they get these things. They understand the ripple effect. They intrinsically know that when you release something here’s what’s likely to happen and they’re ready with the answers. They’re ready with the direction when things start to go slightly off track.

Loren Baker: Great! Maybe we can do a follow-up episode, Duane, about linking. Bring yourself on and maybe some others.

We’re almost out of time. Thanks again. Duane Forrester from Microsoft Bing. Love you guys. One thing I was going to mention to you was that Microsoft has a store in the Century City mall here in LA, and about six months ago when I walked past the store, there weren’t too many people in there compared to Apple. Now it’s full of kids, you know why? It’s the Xbox and Minecraft store now, right? It’s amazing.

Duane Forrester: Exactly, I’ll tell you. I have Minecraft on my phone, and I’m addicted to it. It brings out the architect in me. I can’t help it.

Loren Baker: Exactly. Maybe my five year old will become an architect one day. He’s not playing yet, but he’s watching all the videos on YouTube of people playing. It’s incredible.

Duane Forrester: I love this, right? Ecosystems get built out behind things to answer questions and engage people, and our industry is like that. It’s always so exciting to watch.

Loren Baker: That’s great. Thanks again. Duane Forrester here on Search & Deploy. We’ll be doing a follow-up blog post on this. Really enjoyed talking to you and a lot of the tips you had on basically what doorway pages may be, and how in the grand scheme of things, making sure you’re not over-polluting the search engines with pages on your site that are not intended to be indexed. We’ll be following up with that and thanks again, really appreciate it.

Duane Forrester: Cool. Thanks for your time, Loren.

Loren Baker: Have a good one, buddy.

Duane Forrester: You too.