Dynamic robots-txt

Last Updated : |  

Dynamically robots.txt - generate, in PHP for it one can use a small function and can be expanded and adapted arbitrarily. You can also create them with any text editor.

Use the robots.txt because it also has a part in the web meaning and you can save unnecessary things even if not all bots listen to them and follow. There are a lot of bots that also cause unnecessary things on your web page and thus influence your statistics. You ignore any request in the robots.tx, go to any folder on your website and rummage through everything. Bots are automated, they are programmed to do certain tasks and never get tired;).


Do not leave important files unencrypted on your server and certainly not free Accessible if you have important files on the server. Try to protect your environment as much as possible, the reason why a website is unsafe is in most cases the programmer.

     0  /**  1   * Dynamic robots.txt  2   */  3  function createRobotsTXT() {  4    5          $return = false;  6          $default = '# robots.txt for tnado.com ' . PHP_EOL;  7          $default .= 'Sitemap: https://tnado.com/sitemap.xml' . PHP_EOL;  8          if (!file_exists('yourpath')) {  9              $return = file_put_contents('yourpath', $default);  10          }  11          return $return;  12      } 

    In the Search Console of Google is a tab under crawling robots.txt-tester and it can also be there robots.txt be created and also tested, but this is not necessary and takes a little longer. The other ways are faster and it will take until Google displays them in the Search Console.

    Among other things, you can specify your sitemap in robots.txt or exclude it with certain bots (if you listen to it and do not ignore it).

     Further information to robots.txt from Google.

    One can with the .htaccess, e.g. do more and really make it possible to exclude mastering proficiency with regulars.

    Your Comment

    * This fields are required, email don't publish
    You can use this tags: <p>, <em>, <strong>, <u>, <s>, <pre>, <blockquote>

    This field is optional
    Fill this field link your name to your website.

    Data entered in this contact form will be stored in our system to ensure the output of the comment. Your e-mail address will be saved to determine the number of comments and for registration in the future

    I have read the Privacy policy and Terms. I Confirm the submission of the form and the submission of my data.
    tnado © 2019 | All Rights Reserved
    In cooperation with Hyperly