Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-22 15:37:45 +00:00
Code Issues Projects Releases Wiki Activity
c996f1ee74
mitmproxy/docs/bucketassets/robots.txt

3 lines
52 B
Plaintext
Raw Normal View History

docs: add robots.txt and error.html - robots.txt prevents crawling of everything but /stable - error.html has a simple meta refresh to the latest stable docs. This will also help crawlers find the right path.
2018-03-04 21:57:50 +00:00
User-agent: *
Disallow: /archive/
Disallow: /master/
Reference in New Issue Copy Permalink