The Master Plan Mini Course

Avoid Theme Bleeding - How to

In the last tutorial you learned about a phenomenon called theme bleeding.

In this lesson you will learn how to avoid theme bleeding by using a method that keeps the search engines from following links that may cause this.

User navigation is one of the main ways that theme bleeding occurs.

Say that you have a pet website and you have written a page of content about cat toys. Because you want your visitors to be able to quickly find other information on your site you will have a user navigation menu.

In the user navigation you have a link that leads to dog beds which are not related to cat toys.

The search engine spiders will follow this link from cat toys to dog beds and now your theme of cat toys is bleeding into dog beds. From the perspective of a search engine robot, this could play foul on your rankings.

You may not rank as high as you could if theme bleeding was eliminated.

How can you eliminate theme bleeding?

In The Master Plan I discussed using the nofollow attribute on links that you don't want followed.

This is a normal link:

Dog Beds

This is how the link would look after the nofollow attribute has been added.

rel="nofollow">Dog Beds

The nofollow attribute is shown in bold red above.

The only problem with the nofollow is the fact that search engine spiders do still follow these links.

I went to work looking for a solution to this issue and after collaborating with several Master Plan members we concluded that the Robots.txt method is the best method to use to keep the robots at bay and protect your themes.

I will discuss this method shortly but first I want you to understand a point.

Many people ask me why would you not want a robot to follow all the links in your site.

My answer usually goes something like this...

You do want the robots to follow all the links in your site so they become indexed and ranked in the search engines. However, you must control when and where the robot follows a link to pages in your site.

The robots.txt method gives you power over how the robots spider your site.

You are simply saying, "hey robot, I don't want you to see a link from my cat toys page to dog beds within the context of cat toys".

You will show the robot the dog bed information only when the robot is spidering your dog bed information... not while it is spidering cat toy information.

This is the basis of theme bleeding and you should avoid it to gain a boost in your search engine rankings.

The Robots.txt Method

Robots.txt is a simple text file that sits on your server in the same directory where your home page exists.

It is easy to see examples of robots.txt all over the Internet. To see one simply go to any websites domain and then type "/robots.txt".

For example:

To see the robots.txt file for seo2020.com go to:

Here is the robots.txt file for seo2020:

User-agent: *
Disallow: /seo-articles/tag-and-ping-primer/
Disallow: /cgi-bin/
Disallow: /img/
Disallow: /web-samples/
Disallow: /directory-submissions/
Disallow: /download/
Disallow: /d/
Disallow: /co-op/
Disallow: /seo-video-tutorials/
Disallow: /tmp/

We are basically telling the search engine robots not to, under any circumstance, to ever spider and URL in any of the directories listed.

Now on to the next part of this tutorial...

You see the line I have in bold red above.

I just happen to have a URL redirect script inside my /cgi-bin/ directory.

Say I want to link to a totally irrelevant site from this site without being penalized in any kind of way.

Say I want to link to a cat website that has nothing to do with the theme of this tutorial: fanciers.com

The HTML would look like this:

Cat Fanciers

I would surely get penalized for linking to non related information if I posted the link this way.

So instead I will use my robots.txt and link to a script inside my /cgi-bin/ directory which will redirect the click to fanciers.com.

The HTML looks like this:

To prove my confidence in this method I will post a link to Cat Fanciers website right here: Cat Fanciers

As you can see, the URL in the link points to the /cgi-bin/ directory which the robots are not allowed to follow so no theme bleeding will occur.

You can use this same strategy to control when a robot follows an internal link within your website. Use it in your user navigation links to avoid theme bleeding or any other link in your site that is not directly related to the subject of the page where the link exists.

Here is the cgi code to place a simple redirect script in your cgi-bin. To use this code simply upload this code to your /cgi-bin/ directory on your server. Then any link you want to point to but don't want the robots to follow is easy.

Above I used the following URL:

http://www.seo2020.com/cgi-bin/redirect2.cgi?http://www.fanciers.com

The cgi script will redirect to any URL you place after the ? mark.

Here is the code:

#!/usr/bin/perl @date = localtime(time); $date[4]++; $Time = "$date[4]/$date[3]/$date[5]"; $Query_File = $ENV{QUERY_STRING}; $Query_File =~ s/%([0-9A-F][0-9A-F])/pack("C",oct("0x$1"))/ge; $Query_File =~ tr/+/ /; ($url) = split(/\&/,$Query_File); if ($url =~ /=/) { ($name, $url) = split(/=/, $url); } if ($url =~ /^(ht|f)tp:\/\//) { print "Location: $url\n\n"; } else { &Error("Your URL sould be begining by http:// or ftp://\n"); } sub Error { my($ErrorText) = @_; print "Content-type: text/html\n\n"; print "Error: ".$ErrorText; exit; }

Just copy the code above into notepad and then save your new notepad document as "redirect.cgi" then upload it to your /cgi-bin/ directory.

Another Strategy

Those of you who don't run cgi scripts on your server will want to use a simple meta redirect.

In this case you could create a new directory on your server and call it whatever you want. For this example I will call it "r" for redirect.

Then you want to disallow this new directory in your robots.txt

Just add the line:

Disallow: /r/

Now for each link that you want to redirect you simply create a simple HTML page with a meta refresh or java redirect.

Here is an example of the meta refresh:

 

Place the web page above in your new /r/ directory and replace the http://www.url.com with the site you want to link to.

The only drawback to using this method is that you have to create a new redirect page for every link that you don't want the robots to follow.