Select your language

What is the robots.txt file?

The file called robots.txt is a file that can be used to control which files on your website so-called crawlers should access.

The robots.txt file is always located in the document root of a website, i.e. in the directory that contains all the files of a website. For the website www.your-own-domain.ch, the path to the robots.txt file would be as follows: www.your-own-domain.ch/robots.txt.

The robots.txt file is a plain text file without formatting that contains the “Robots Exclusion Standard” (for more on this: https://en.wikipedia.org/wiki/Robots_exclusion_standard). Rules are defined in this file which allow or block a specific crawler from accessing the document root of the domain or subdomain. So if nothing is specified in the robots.txt file, crawlers can access all files of the website.

What comes with the standard version of this file? 

If you have not defined your own version of the robots.txt file for your website, the Hostpoint standard version is delivered (default version). This defines a crawl delay (interval between calls in seconds) so that the bots cannot send their queries directly one after the other, but have to wait between each query. The Hostpoint standard also allows access to all files on a website.

The standard version of the file therefore contains the following parameters:

robots.txt
 User-agent: *
 Crawl-delay: 3 

Please note that not all bots pay attention to the crawl delay.

How can I overwrite the standard version of the file?

If you want to create your own robots.txt file, you can place it in your website's directory and write as many rules as you wish inside. You can find more information on robots.txt files at the following link: https://moz.com/learn/seo/robotstxt

Please use this form only to provide feedback on the above guide.
For support requests please use this form instead.

 

 

© 2001 - Hostpoint AG
Cookie

Wir verwenden Cookies  🍪

We use Cookies  🍪

Nous utilisons des cookies  🍪

Utilizziamo dei cookie  🍪

Die digitalen Auftritte von Hostpoint (Website, Control Panel, Support Center etc.) verwenden Cookies. Diese werden dazu verwendet, um Daten über Besucherinteraktionen zu sammeln. Wenn Sie auf «Akzeptieren» klicken, stimmen Sie der Verwendung dieser Cookies für Werbezwecke, Website-Analyse und Support zu. Gewisse essenzielle Cookies sind jedoch für eine ordnungsgemässe Funktion dieser Seiten unerlässlich und können deshalb nicht deaktiviert werden. Auch ohne Ihre Zustimmung können gewisse Daten in anonymisierter Form für statistische Zwecke und zur Verbesserung unserer Websites verwendet werden. Bitte beachten Sie unsere Datenschutzerklärung.

Hostpoint's digital presences (website, Control Panel, Support Center, etc.) use cookies. These are used to collect data on visitor interactions. If you click “Accept”, you agree to the use of these cookies for advertising purposes, website analysis and support. However, certain cookies are essential for the proper functioning of these pages and therefore cannot be disabled. Even without your consent, certain data may be used in anonymized form for statistical purposes and to improve our websites. Please note our Privacy policy.

Le sites Web de Hostpoint (site Web, Control Panel, Centre d'assistance, etc.) utilisent des cookies. Ces cookies servent à collecter des données sur les interactions des visiteurs. En cliquant sur «Accepter», vous consentez à l’utilisation de ces cookies à des fins de publicité, d’analyse du site Web et d’assistance. Certains cookies essentiels sont cependant indispensables au bon fonctionnement de notre sites Web et ne peuvent donc pas être désactivés. Même sans votre consentement, certaines données peuvent être utilisées sous forme anonymisée à des fins statistiques et pour améliorer notre sites Web. Veuillez prendre connaissance de notre Déclaration de protection des données.

Le presenze digitali di Hostpoint (sito web, Pannello di controllo, Support Center, ecc.) utilizzano i cookie. Questi vengono utilizzati per raccogliere dati sulle interazioni dei visitatori. Facendo clic su «Accetta», acconsente all’utilizzo di questi cookie per scopi pubblicitari, di analisi del sito web e di supporto. Alcuni cookie essenziali sono tuttavia indispensabili per il corretto funzionamento di questi siti web e pertanto non possono essere disattivati. Anche senza il Suo consenso, determinati dati potrebbero essere utilizzati in forma anonima per fini statistici e per l’ottimizzazione dei nostri siti web. Si prega di tenere conto della nostra Dichiarazione per la pivacy.

Ablehnen
Decline
Refuser
Rifiuta
Akzeptieren
Accept
Accepter
Accetta