≡ Menu

firefox google bot user agent – How to Impersonate Googlebot using Firefox see what Googlebot sees

I have spoken about Impersonating Googlebot user agent using Firefox in the past and people have asked me how to do this using Firefox so I thought I would give a little tutorial on how to see what googlebot sees. I will start off with why somebody might want to impersonate Googlebot and the pros and cons of impersonating it.
Googlebot is the spider that crawls the internet looking for web pages. Whatever the Googlebot sees is what Google uses to decide how a page will rank in the search engine. Knowing this some people have decided to design a web page exactly how they think Google wants it to look like so it will rank better. Often times this can be a very ugly page so they design another page that looks how they want it to look. They then cloak their page so that the Googlebot user agent sees the page written for it and everybody else sees the page written for humans. This is against the Google rules.

When you surf the net your Internet browser sends out messages telling the website things about itself. Websites keep track of visitors in their log files. One of the bits of information that is sent out is something called a “User Agent”. Each browser has its own user agent string. You can find the user agent that you are sending by clicking here.

So if you want to surf around impersonate googlebot all you have to do is change your user agent. The way that I do it is to use Mozilla Firefox with the User Agent Switcher extension. Both sites have a download or install button just follow the directions it is quite straight forward. One the user agent switcher is installed you go to Tools -> User Agent -> switcher -> options -> options. Then click on “User Agents” and then click on “add”. Put “Googlebot” in the Description. Put

“Googlebot/2.1+(+http://www.googlebot.com/bot.html)”
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

in the User Agent.

firefox google bot
firefox googlebot
[tags]googlebot, firefox, googlebot firefox, spoofing user agent[/tags]

Comments on this entry are closed.

  • Huub February 3, 2007, 5:33 pm

    What`s the use of this?

  • ogletree February 3, 2007, 7:02 pm

    You use it to detect if somebody is serving differnt pages to googlebot that to a normal user.

  • mcpwnalds November 14, 2007, 12:26 pm

    how do i know if it works or not?

  • JSX January 4, 2008, 7:56 am

    How to test it?

  • New Method December 5, 2008, 6:30 am

    It’s interesting, with User Agent switcher set to Googlebot, the Delicious website is still full of nofollow links. I remember seeing on a forum that the nofollow didn’t show if you had useragent set to Googlebot. Maybe everything has changed?!?

  • Sussex Signs December 16, 2008, 3:01 pm

    In response to New Method – The Delicious site has had nofollow in place for quite some time now.

  • ogletree February 26, 2009, 12:53 pm

    No quotes

  • BloodNova March 6, 2009, 4:27 pm
  • Kashif March 15, 2009, 3:39 am

    Current version of User Agent Switcher is not compatible with FireFox 3.0.7. Any workaround to test this tip?

  • ogletree June 5, 2009, 9:28 pm

    I use Firefox 3.0.10 and it works fine for me.

  • Henry Gilbert June 11, 2009, 9:20 pm

    Or just use Konqueror (Linux KDE). It switches to Googlebot easily in the menu. No need to configure or add anything. Just switch and unswitch.

  • Pavlo February 18, 2010, 11:09 am

    @ishi77

    You are seriously kidding me! You don’t know how to install firefox extensions? get a life.