Add Robots.txt File In Blogger Blog

As we discuss the important of Robots.txt file in old post blogger SEO tips. So that, it is important to create and add robots.txt file in blogger blog. Recently blogger added a new option which called search preferences into blogger that allow you to add custom robots header tags and custom robots.txt file. Every blogger must know about to robots.txt file before changing the robots.txt file because the wrong robot.txt file effect on your search engine results. 

What is Robots.txt File?

Robots.txt file contains a few lines of code. That interact with the search engines crawlers.  A robots.txt file restricts access to your blog by search engine robots that crawl the blog. The search engine sends their crawler to crawl new page on the blog. If crawlers find the new page and they will like to index at that time crawlers checks the robots.txt file. If you disallow a page in robots.txt file then crawlers will not index that page. 

Create And Add Robots.txt File 

Step 1: - Log in to blogger account.

Step 2: - Now navigate to Setting >> Search Preference. Click on edit in Custom Robots.txt.

Step 3: - Add bellow code into Custom Robots.txt  and Click on Save Change. 

User-agent: Mediapartners-Google
User-agent: *
Disallow: /search/
Disallow: /search?q=* Allow: /
Sitemap: http://YOURBLOGADDRESS.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

Note:-
1) If you have less than 500 posts then the red code replace with below code.

Sitemap: http://YOURBLOG.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

2) If you have more than 500 posts then the red code replace with below code.

Sitemap: http://YOURBLOG.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
Sitemap: http://YOURBLOG.blogspot.com/atom.xml?redirect=false&start-index=500&max-results=1000

#Explanation Of Robots.txt File

• User-agent: Mediapartners-Google :- This line indicates that you are using the Google Adsense if you are not using Google Adsense then remove it. 
• User-agent:* :-  Here user-agent calling the robot and * is for all search engine robots.
• Disallow: /search/ :- This command tells search engine crawlers/bots to not crawl label pages.
• Disallow: /search?q=* :- This command tells search engine crawlers/bots to not crawl search pages.
• Allow: / :- This code allows to index your blog.
• Sitemap :- This lines tells the search engine crawler to index the every new or updated pages and post. 

How To Check Robots.txt File?

You can check your robots.txt file by using below URL.

http://www.YOURBLOGADDRESS.BLOGSPOT.com/robots.txt

Once you visit the robots.txt URL you will see the entire code which adding in your blogger blog. See the below image.

Hope now you have a good knowledge of  robots.txt file. If you have any query then feel free to mention it in comment box.