Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor. Lorem ipsum dolor sit amet, consectetur.
Contact us now +40 799 219500
[email protected]

Instagram Feed


Home  /  Online marketing   /  How to create a Robot.txt | what is a Robots.txt | Robots exclusion standard – Part 8

How to create a Robot.txt | what is a Robots.txt | Robots exclusion standard – Part 8


We should use robots.txt file to restrict any search engine for crawling. In this video WsCube Tech explain you how to create robots.txt.

The Robot Exclusion Standard = Robots Exclusion Protocol = robots.txt Protocol

1. You need a robots.txt file only if your site includes content that you don’t want search engines to index like Your site Admin Panel.

2. User-agent: *Disallow: /Admin Panel/

3. less then bracket meta name=”robots” content=” noindex,nofollow“ greater then bracket

4. less then bracket meta name=”googlebot“ content=”noindex” greater then bracket

5. X-Robots-Tag: noindex

Link – www.domain-name.com/robots.txt

Learn Digital Marketing : Download APP


This app is a complete guide for learning digital marketing step by step.

Follow Us On :
Facebook : https://www.facebook.com/wscubetech.india
Linked in : https://www.linkedin.com/company/wscube-tech
Google+ : https://plus.google.com/+wscubetechjodhpur
YouTube : https://www.youtube.com/c/wscubetechjodhpur
Twitter : https://twitter.com/wscube

This is author biographical info, that can be used to tell more about you, your iterests, background and experience. You can change it on <a href="/wp-admin/profile.php">Admin > Users > Your Profile > Biographical Info</a> page."

No Comments
Post a comment

12 − five =