What is a robots txt file? How to add to blog – Full Guide

If you are a blogger or you have just started blogging. In such a situation, you must have heard about robots txt file in blogger or WordPress, you must have known about robots.

This robot is a machine of spring. It follows our command. This is the same thing as robots.txt file. Whatever command we give to this file, it sends this command directly to the search engine. As you know

The search engine works. To see your post according to the search of the people, and to index your post, let us know about it well.

Best Domain Offers
Best Hosting Offers

 

What is a robots txt file?

As you know from the name itself, .txt is an extension of a text file. In which we can only write text. Similarly, robots.txt is a file. In which we write some text or message.

robots-txt-file

In this message, we write about the parts of our blog. Which part is to be shown in the search engine and which part is not.

Robots txt file example

For example, if you do not want to show pages from your blog in search engines, or you want to not index the category or tags in your blog. So you can write all these things inside the robots txt file.

robots-txt-file

After that whenever a search engine like google, yahoo, bing will start indexing your blog post. So, on behalf of robots.txt file, all these search engines will get a message that the blog will index all these things.

Do not index all these things. It benefits from this, which makes your blog SEO friendly. This makes the search engine easier to an index which part and which not.

How to add a Robot txt file to Your Blog

Hello friends, welcome to your own blog called Digital SEO Life. Today, you will know how to add a robot txt file to your blog. Whether you run your blog in WordPress, or you run your blog in Google Blogger. Both of these will teach you to add it.

robots-txt-file

 

Recommended Tool

How to Add Robots txt File to Blogger.

In Blogger, you will find the option of robots.txt file under search preferences by going to a setting.

When we make our Blogspot blog SEO friendly. Then we get the option of this robots.txt file.

Which most people leave like this. By the way, if you leave this option in Blogger, then the blogger automatically makes the best robots.txt file for you.

But if you want. If some parts of your blog are hidden in the search engine, then you can do it. But if you are taking blogging seriously, and you later migrated your Blogspot blog to WordPress then you must use this feature.

Let us now learn, “how to add robotic txt file to Blogspot blog“. Before that, let us know about some things that you will use inside this file.

Allow: – If you allow something inside the text file, it means that you want this part to be visible in the search engine.

Disallow: – If you disallow a post, it means that you do not want this part to appear in the search engine.

User-agent: – By using this option, you can show your visitors what you want to show on your site, as if the source of the image is the file where it is saved, etc.

User-agent: – Mediapartners-Google: By its use, you can hide the AdSense code from the user, if you disallow it, Google Adsense will not be able to show ads in your site, so you should always allow it.

Go to Blogger’s settings, click on search preferences, then click on robots.txt, then click on edit.

robots-txt-file
 

Now you have to copy the code given below and paste it inside the box and save it

robots-txt-file

 

User-agent: Mediapartners-Google
Disallow:
User-agent:
Disallow: / search
Allow: /
Sitemap: http://www.yoursite.com/sitemap.xml

Note: Put your site map in place of the site map and save the setting.

codes for robots.txt on blogger

In this way, you can heighten and show some posts of your blog by using robots.txt file in your Blogspot blog. If you are using blogger, then it will be better if you do not use this feature if you are using WordPress, then you must use this feature.

What is a sitemap? How to Submit Your Sitemap In Google.

How to make Blogspot blog SEO Friendly

Seo Friendly Content Writing Tips 2020

How to add Robots.txt file to WordPress

1. If you use WordPress. So you must have installed Yoast SEO. So you have to go to Yoast SEO and click on Tools then click on File editor.

robots-txt-file

2. Now copy the code below and paste it inside the robots.txt file. Then click on save changes to Robots.txt and save it.

User-agent: *
Disallow: /wp-admin/
Disallow: /cgi-bin/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /xmlrpc.php
Allow: /wp-admin/admin-ajax.php
User-agent: Mediapartners-Google*
Allow: /
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Mobile
Allow: /
Sitemap: https://catchhow.com/sitemap.xml

Note: Put your sitemap in place of a sitemap.

robots-txt-file

Tips:- If you use the Blogspot blog. If you do not add robots.txt file, then Google automatically creates the best robots file for you. If you use WordPress. So you must use this robot file. If you still have to ask anything, you can ask in the comment box below.

If you want to get a backlink on this blog then contact us.

If you like our article, then do share it with it.

Leave a Comment