Archive

Archive for the ‘Googlebot’ Category

Never let a Search Engine make a decision

February 12th, 2009 ogletree 5 comments

When you build a website you need to completely understand how a search engine looks at you. Never let a search engine make any decisions. When a search engine comes to your website you need to know exactly how it is going to crawl your site.

Pages should only have one url. If you have a database driven website it is very common that the same content can be pulled up with an unlimited number URL’s. You have to enforce your url’s. Your content needs to be tied to only one url. Put logic into your site so that if it detects that content is being pulled up by the wrong url that you 301 redirect to the correct url. All URL’s should be consistent. The best way to do this is to make every url lower case. If you use a windows server you need to be extra careful because IIS will send you to the same content and produce duplicate content.

firefox google bot user agent – How to Impersonate Googlebot using Firefox see what Googlebot sees

January 24th, 2006 ogletree 17 comments

I have spoken about Impersonating Googlebot user agent using Firefox in the past and people have asked me how to do this using Firefox so I thought I would give a little tutorial on how to see what googlebot sees. I will start off with why somebody might want to impersonate Googlebot and the pros and cons of impersonating it. Read more…

Categories: Firefox, Google, Googlebot, Tutorials Tags: