aboutsummaryrefslogtreecommitdiff
path: root/src/SemanticScuttle/header.php
diff options
context:
space:
mode:
authorSilvio Rhatto <rhatto@riseup.net>2020-12-05 17:04:13 -0300
committerSilvio Rhatto <rhatto@riseup.net>2020-12-05 17:04:13 -0300
commitf9b94e3ba677fd19c311fdde7cd9da8709953000 (patch)
tree6d95d3bebd6c908368c105af3bc279f340ea3b68 /src/SemanticScuttle/header.php
parent37f10bd9d3f955fd648424d3908d23c33bea39fc (diff)
downloadsemanticscuttle-f9b94e3ba677fd19c311fdde7cd9da8709953000.tar.gz
semanticscuttle-f9b94e3ba677fd19c311fdde7cd9da8709953000.tar.bz2
Feat: adds a robots.txt
Robots tend to crawl all tag combinations, which make a huge performance impact in larger datasets, specially regarding database queries. This robots.txt file will prevent that regular search engines to index the application, which is already a search index.
Diffstat (limited to 'src/SemanticScuttle/header.php')
0 files changed, 0 insertions, 0 deletions