Karan Kumar

May be putting it behind authentication would stop bots from crawling it I guess..

Karan Kumar

@theruss Unfortunately its publicly available. So I'd want to put it behind authentication and stop bots from crawling it.

Karan Kumar

Heh..Using Deploy script by Sysadmins from hosting provider team (I don't have that privileges) ...so the issue I'm having is.. I'd like to have different robots.txt for test/UAT and prod env. I can easily create a file and deploy it with git but can't edit it on specific env.

Karan Kumar

@dorsetdigital I agree, it is an overkill, I've done the same thing with my previous projects. But for this particular project, the hosting platform we are using doesn't provide ability to upload files (which is annoying) and are bit slow in replying as well so I just wanted to avoid that. Anyway Thanks a lot for your help 🙂

Karan Kumar

Hi Team, could you please suggest a good SS4 module for robots.txt generation, there are quite a few options available, just wanted to have some feedback. Thanks 🙂

Karan Kumar

@Jose Not sure the purpose of it but did you have a look at this module: https://github.com/silverstripe/sspak

Show 1 attachment(s)
GitHub  
silverstripe/sspak

Tool for managing bundles of db/assets from SilverStripe environments - silverstripe/sspak

Hide attachment content
Karan Kumar

@StefGuev You can do that with UploadField but don't think there's core method to do that with HTMLEditorField, may be because HTMLEditorField UI does present you with an option to select your destination folder first prior to uploading.