728x90 Ad Space

Saturday 18 January 2014

How To Add Custom Robots File In Blogger?



On : 1/18/2014

Previously, We Discussed About How To Add Meta Description, Now We Are Back With Another Great Tutorial For Search Prefrences Section. Today I Am Going To Tell You About Robots Of Search Engines Who Crawls Your Page And Index Your Blog, It Also Improves Traffic Highly. Robots.txt File Is Used By Many Bloggers But In Past Old Blogger Interface, We Were Unable To Find That Option Of Robots.txt To Enable, But Now It Is Available In Search Prefrences Section. Who Did Not Know What Is Robors.txt, In This Tutorial, We Will Learn About Robots.txt And Will Enable. So Lets Get Started!.

What Is Robots.txt File

This Is A Simple File Which The Website Owner Writes Commands For The Search Engine's Crawler. It Tells The Robots To Crawl The Index Of The Command Given. That Commands Are Written In Different Codes Which Is Only For Search Engine's Crawler. You Can See Your Robots.txt File Just By Following The Below URL:-

http://www.yourdomain.com/robots.txt

Which Areas Should You Disallow To Crawl?

So, This Question May Come In Your Mind That Which Areas Should You Disallow For Robots To No Crawl. You Can Disallow Any Area You Want For That, Your Blog Will More Search Engine Friendly And That Areas Are Search Results Pages, Archive Pages etc. Now Below, I'am Going To Tell You How To Enable Robots.txt And Make Your Blog Search Engine Friendly. Just Follow The Instructions Below:-

Enabling The Robots.txt File

So, The Process Of Enabling Is Very Easy, You Have To Just Copy A Simple Code And Paste It.Be Careful When Copying And Pasting Because If You Missed 1 Letter, The Whole Code Will Not Work. Follow The Instructions Below:-
  • Go To Blogger.com >> Your Blog >> Settings >> Search Prefrences.
  • Look At The "Crawlers And Indexing" Section.
  • Click On Edit On "Custom Robots.txt".
  • Now Click On Yes.
  • Now A Box Will Appear Asking You To Write The Code.
  • Than Copy The Code Below And Paste It In The Box-

User-agent: Mediapartners-Google
User-agent: *
Disallow: /search?q=*
Disallow: /*?updated-max=*
Allow: /
Sitemap: http://www.yourdomain.com/feeds/posts/default?orderby=updated

  • Important Note! : The Frist Line "User-agent: Mediapartners-Google" Is Used For Google Adsense Enabled Blogs. So If You Don't Have Google Adsense, Then Remove It Otherwise It Will Not Work.
  • Click On "Save Changes".
  • Done!.

Now After Adding The Code, You Must Learn That For What Reason The Each Line Of Code Is.

Explanations

  • User-agent: Mediapartners-Google : In This Command, You Are Telling The Adsense Robot To Crawl All Pages Which Are Including Or Displaying Adsense Ads.
  • User-agent: * : In This Command, You Are Calling The Robots Of Different Search Engines Example Google, Yahoo, Bing etc.
  • Disallow: /search?q=* : This Line Tells The Crawlers To Not To Crawl The Search Pages.
  • Disallow: /*?updated-max=* : This Line Disallows The Search Engine's Crawler To Not Crawl Label Or Navigation Pages.
  • Allow: / : This Line Allows To Crawl Or Index The Homepage Of  Your Blog.
  • Sitemap : This Line Tells The Search Engine's Crawler To Crawl The Every New Or Updated Post. Replace "yourdomain" With Your Domain Name.
Now I Think You've Go It!

How Can I Add Commands To The Code?

In This Part, I Will Tell You How Can You Add A Command For Disallowing Any Web Page, So If You Are Interested Adding Your Own Command, You Can Add It By Just Few Simple Steps. Here Is An Example For Disallowing Any Page Disallow: /p/sitemap.html. So, First You Need To Replace "/p/sitemap.html" Which Is The Other Part Of Domain Name With Another. It Means You Just Need To Exclude The Main Domain And Just Type Remain Part Of The Page. If You Still Not Got What To Do, Ask In Comments Freely

Final Words!

So Friends, That Was Our Robots.txt File Integration For Search Engine Crawlers. We Will Try To Give Each And Every Post With Great Tutorials Of SEO With Explanation So You Can Understand Easily. So Keep Visiting Us For More Tutorials. Please Share This Post With Your Friends And On Social Networks And Also Give Your Feedback. I Hope This Tutorial Will Help You Improve Your Traffic On Your Blog.  If You Can't Understand This Tutorial, Ask In Comments Becaus Comments Are Always Appreciated. Take A Lot Of Care Of Your Family And You And Happy Blogging!.

No comments:

Helping You To Know The Knowledge Of Blogger Blogging Since 2014™.