How to add robots.txt in Your Blog Complete Guide

If you are a blogger or you have just started blogging. In such a situation, you must have heard about robots txt file in blogger or WordPress, you must have known about robots.

This robot is a machine of spring. It follows our command. This is the same thing as robots.txt file. Whatever command we give to this file, it sends this command directly to the search engine. As you know.

The search engine works. To see your post according to the search of the people, and to index your post, let us know about it well.

How to add robots.txt in Your Blog Complete Guide

What is a robots txt file?

As you know from the name itself, .txt is an extension of a text file. In which we can only write text. Similarly, robots.txt is a file. In which we write some text or message.

In this message, we write about the parts of our blog. Which part is to be shown in the search engine and which part is not.

Robots txt file example

For example, if you do not want to show pages from your blog in search engines, or you want to not index the category or tags in your blog. So you can write all these things inside the robots txt file.

After that whenever a search engine like google, yahoo, bing will start indexing your blog post. So, on behalf of robots.txt file, all these search engines will get a message that the blog will index all these things.

Do not index all these things. It benefits from this, which makes your blog SEO friendly. This makes the search engine easier to an index which part and which not.

Hello friends, welcome to your own blog called Digital SEO Life. Today, you will know how to add a robot txt file to your blog. Whether you run your blog in WordPress, or you run your blog in Google Blogger. Both of these will teach you to add it.

How to Add Robots txt File to Blogger.

In Blogger, you will find the option of robots.txt file under search preferences by going to a setting.

robot txt generator for blogger
robot txt generator for blogger

When we make our Blogspot blog SEO friendly. Then we get the option of this robots.txt file.

Which most people leave like this. By the way, if you leave this option in Blogger, then the blogger automatically makes the best robots.txt file for you.

How to add a Robot txt file to Your Blog

But if you want. If some parts of your blog are hidden in the search engine, then you can do it. But if you are taking blogging seriously, and you later migrated your Blogspot blog to WordPress then you must use this feature.

Let us now learn, “how to add robotic txt file to Blogspot blog“. Before that, let us know about some things that you will use inside this file.

Allow: – If you allow something inside the text file, it means that you want this part to be visible in the search engine.

Disallow: – If you disallow a post, it means that you do not want this part to appear in the search engine.

User-agent: – By using this option, you can show your visitors what you want to show on your site, as if the source of the image is the file where it is saved, etc.

User-agent: – Mediapartners-Google: By its use, you can hide the AdSense code from the user, if you disallow it, Google Adsense will not be able to show ads in your site, so you should always allow it.

Go to Blogger’s settings, click on search preferences, then click on robots.txt, then click on edit.

Now you have to use Sitemap Generator For Blogger and Generat your robots.txt file copy and paste it inside Blogger robots txt box and save it.

Note: Put your site map in place of the site map and save the setting. 

In this way, you can heighten and show some posts of your blog by using robots.txt file in your Blogspot blog. If you are using blogger, then it will be better if you do not use this feature if you are using WordPress, then you must use this feature.

How to add Robots.txt file in WordPress

1. If you use WordPress. So you must have installed Yoast SEO. So you have to go to Yoast SEO, And RankMath SEO and click on Tools then click on File editor.

2. Now copy the code below and paste it inside the robots.txt file. Then click on save changes to Robots.txt and save it.

User-agent: *
Disallow: /wp-admin/
Disallow: /cgi-bin/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /xmlrpc.php
Allow: /wp-admin/admin-ajax.php
User-agent: Mediapartners-Google*
Allow: /
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Mobile
Allow: /


Note: Put your sitemap in place of a sitemap.

Tips:- If you use the Blogspot blog. If you do not add robots.txt file, then Google automatically creates the best robots file for you. If you use WordPress. So you must use this robot file. If you still have to ask anything, you can ask in the comment box below.

If you want to get a backlink on this blog then contact us.

If you like our article, then do share it with it. 

More Artical


Ad Blocker Detected!

Ad Blocker Detected!

How to disable? Refresh