Update robots.txt to remove crawling of all but root
This commit is contained in:
parent
65c81c1110
commit
78ab3d2622
|
@ -1,3 +1,3 @@
|
|||
User-agent: *
|
||||
Disallow: /*/snapshot/*
|
||||
Allow: /
|
||||
user-agent: *
|
||||
Disallow: /
|
||||
Allow: /$
|
||||
|
|
Loading…
Reference in New Issue