How to Use Robots.Txt to Block Subdomains
As a webmaster, you encounter times when you may wish to block search engine access from your website to prevent indexing. This is often the case if you have private content or duplicate content on two different domains. Normally you can upload a robots.txt file to the root directory of your domain and specify search engine access to all sub-directories on the domain. If you have a subdomain on the site, however, you must create a new robots.txt file specifically for the subdomain and upload it to the root directory of the subdomain.
Other People Are Reading
Instructions
-
-
1
Launch your computer's text editor and paste the following code in the document:
User-agent: * Disallow: /
-
2
Save the file as a text file (.txt) and name it "robots.txt".
-
-
3
Upload the file to the root (top level) directory of your subdomain using your preferred FTP server. For example, if your subdomain is subdomain.domain.com, then you should be able to access the robots.txt file at subdomain.domain.com/robots.txt.
-
1
References
More Like This
Comments
-
Thanks for this. But I have 1 problem. I have a subdomain redirected to a new domain URL. If I do this it won't index my subdomain but will also not index to my redirected domain name.
-
example: user-agent: * Disallow: /images here folder named images will be blocked by all search engine spiders or crawlers.
-
Thanks, very useful.
