External source for robots.txt
-
Hi!
I would love to be able to enter a git repository or an URL to a file which are pulled regularly to manage my robots.txt
It is a little bit pain to update every running app when there is a new crawler which need to be blocked. I maintain a repository for myself but need to copypaste manually into cloudron.
Bonus would be if you could merge two external files to one robots.txt
Thanks for considering this
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login