Hi Team, has anyone encountered this issue here: https://github.com/silverstripe/sspak/issues/29 and got any suggestions
i have a large website (asset folder is about 5GB).
When i am creating a sspak file by
sspak save backup.sspak /path/to/website
the archive seems to be created
When i try to extract te same file afterwards:
sspak extract backup.sspak
The 2 containing archives are extracted:
When i am trying to unzip the assets.tar.gz the following error occurs:
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
Any ideas ?Hide attachment content
Yes, sspak is limited by the underlying PHP library, and can therefore not handle more than 4GB of assets
I've made a Python based application that basically creates an SSPAK file using Python instead of PHP
It can handle up to 200GB of assets (or more, that's the biggest I've tested)
Note, there's no built package yet. So you'le have to manually run it using
python main.py, after installing the requirements
for anything over the limit the Manual access portion here works: https://silverstripe.github.io/sspak/
I need to look at the py repo though, hate not having the option for larger sites
@muskie9 Thanks for that. Yes there's comment in Sspak.php about doing it manually as well but it was still throwing same error for me.
@Firesphere Thank you very much 🙂 your app is a life savour. Just one thing I'm having confusion about, To create the sspak file does this command look ok to you
python main.py create --db --file=test.sspak
that's odd... tar isn't specific to sspak, so that's likely something else
That would create a DB only export, yes
It's been a while since I worked on that project, so I'm digging in to my memory as to what I exactly did
Ah okay, yes I left it for db only as its fast. For some reason running this is throwing an error IOError: [Errno 2] No such file or directory: '--file=test.sspak'
python create db --file=test.sspak
no dashes before db or assets
Ahh, right, That worked 🙂
Yeah, that's tarring running out of memory
Ow, wait, no, I've had that once, I'm not sure how/where/what I ended up doing to fix that
Ah so you mean my problem is not tar running out of memory?
Initially I thought that error was caused due to corrupt images. So I removed those and then created sspak (with db and assets) but on extracting the same sspak, new images started showing up as corrupt ones. So I thought it might be linked to sspak save/saveexisting
@Firesphere Thank you 🙂 Your app worked for me
May be putting it behind authentication would stop bots from crawling it I guess..
@theruss Unfortunately its publicly available. So I'd want to put it behind authentication and stop bots from crawling it.
Heh..Using Deploy script by Sysadmins from hosting provider team (I don't have that privileges) ...so the issue I'm having is.. I'd like to have different robots.txt for test/UAT and prod env. I can easily create a file and deploy it with git but can't edit it on specific env.
@dorsetdigital I agree, it is an overkill, I've done the same thing with my previous projects. But for this particular project, the hosting platform we are using doesn't provide ability to upload files (which is annoying) and are bit slow in replying as well so I just wanted to avoid that. Anyway Thanks a lot for your help 🙂
Hi Team, could you please suggest a good SS4 module for robots.txt generation, there are quite a few options available, just wanted to have some feedback. Thanks 🙂
@StefGuev You can do that with UploadField but don't think there's core method to do that with HTMLEditorField, may be because HTMLEditorField UI does present you with an option to select your destination folder first prior to uploading.