Skip to main content
Help Center
The GoDaddy Community will undergo maintenance starting on Tuesday, August 3rd at 3pm PST / 6pm EST. Learn more
  • GoDaddy Community
  • How to Market Your Business
  • How to Market Your Business

    cancel
    Showing results for 
    Show  only  | Search instead for 
    Did you mean: 
    stas
    Getting Started

    Search engine optimization ignore rbots.txt & sitemap

    Hello 

    I'm use search engine optimization, but I can't understand why this is tools analysis pages which close for index. For example tools ask add meta descriptions for https://escort-byuro.net/wp-login.php

    but I was close this is URL in file robots.txt 

     

     

     

     

    Also I'm add "no index" for this is pages. But not add "no follow", I was read "no follow" not good crawls. 

    2 REPLIES 2
    jpablo
    Helper VI

    @stas does the tool have access to your files? I mean, can it open them?

    You should also know that the robots.txt may be not be followed by some crawlers. It is a directive, but some may do not listen to it, like the DNT http header.

     

    Hope it helps!

    If you find my replies helpful, give me a like or kudo 🙂
    And if I solved your issue, mark it as solution 😄
    stas
    Getting Started

    also what is mean error?

     

     

    If im click on alexa (check rating my site (from goddady panel) I will see error((

     

     

    But really site indexed in alexa.com ypu can check 

     

    http://www.alexa.com/siteinfo/escort-byuro.net#?sites=escort-byuro.net

     

    Why tools show wrong information?