How to create a Robot.txt | what is a Robots.txt | Robots exclusion standard – Part 8

18 views
%
0

http://www.wscubetech.com/

We should use robots.txt file to restrict any search engine for crawling. In this video WsCube Tech explain you how to create robots.txt.

The Robot Exclusion Standard = Robots Exclusion Protocol = robots.txt Protocol

1. You need a robots.txt file only if your site includes content that you don’t want search engines to index like Your site Admin Panel.

2. User-agent: *Disallow: /Admin Panel/

3. less then bracket meta name=”robots” content=” noindex,nofollow“ greater then bracket

4. less then bracket meta name=”googlebot“ content=”noindex” greater then bracket

5. X-Robots-Tag: noindex

Link – www.domain-name.com/robots.txt

Learn Digital Marketing : Download APP

https://play.google.com/store/apps/details?id=com.wscubetech.seovideotutorials

This app is a complete guide for learning digital marketing step by step.

Follow Us On :
Facebook : https://www.facebook.com/wscubetech.india
Linked in : https://www.linkedin.com/company/wscube-tech
Google+ : https://plus.google.com/+wscubetechjodhpur
YouTube : https://www.youtube.com/c/wscubetechjodhpur
Twitter : https://twitter.com/wscube

Date: October 10, 2017