Karan Kumar

Hi Team, has anyone encountered this issue here: https://github.com/silverstripe/sspak/issues/29 and got any suggestions

Show 1 attachment(s)
derRobert

Hi,
i have a large website (asset folder is about 5GB).
When i am creating a sspak file by

sspak save backup.sspak /path/to/website

the archive seems to be created

When i try to extract te same file afterwards:

sspak extract backup.sspak

The 2 containing archives are extracted:
assets.tar.gz
database.sql.gz

When i am trying to unzip the assets.tar.gz the following error occurs:
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

Any ideas ?

Hide attachment content
Firesphere

Yes, sspak is limited by the underlying PHP library, and can therefore not handle more than 4GB of assets

Firesphere

I've made a Python based application that basically creates an SSPAK file using Python instead of PHP

Firesphere

It can handle up to 200GB of assets (or more, that's the biggest I've tested)

Firesphere

Note, there's no built package yet. So you'le have to manually run it using python main.py, after installing the requirements

muskie9

I need to look at the py repo though, hate not having the option for larger sites

Karan Kumar

@muskie9 Thanks for that. Yes there's comment in Sspak.php about doing it manually as well but it was still throwing same error for me.

Karan Kumar

@Firesphere Thank you very much 🙂 your app is a life savour. Just one thing I'm having confusion about, To create the sspak file does this command look ok to youpython main.py create --db --file=test.sspak

muskie9

that's odd... tar isn't specific to sspak, so that's likely something else

Firesphere

It's been a while since I worked on that project, so I'm digging in to my memory as to what I exactly did

Karan Kumar

Ah okay, yes I left it for db only as its fast. For some reason running this is throwing an error IOError: [Errno 2] No such file or directory: '--file=test.sspak'

Karan Kumar

@Firesphere Did you try it manually using tar as mentioned here https://silverstripe.github.io/sspak/ and by @muskie9 for some reason it was still throwing an error tar: Unexpected EOF in archive for me.

Firesphere

Ow, wait, no, I've had that once, I'm not sure how/where/what I ended up doing to fix that

Karan Kumar

Ah so you mean my problem is not tar running out of memory?

Karan Kumar

Initially I thought that error was caused due to corrupt images. So I removed those and then created sspak (with db and assets) but on extracting the same sspak, new images started showing up as corrupt ones. So I thought it might be linked to sspak save/saveexisting


Show less replies
Karan Kumar

May be putting it behind authentication would stop bots from crawling it I guess..

Karan Kumar

@theruss Unfortunately its publicly available. So I'd want to put it behind authentication and stop bots from crawling it.

Karan Kumar

Heh..Using Deploy script by Sysadmins from hosting provider team (I don't have that privileges) ...so the issue I'm having is.. I'd like to have different robots.txt for test/UAT and prod env. I can easily create a file and deploy it with git but can't edit it on specific env.

Karan Kumar

@dorsetdigital I agree, it is an overkill, I've done the same thing with my previous projects. But for this particular project, the hosting platform we are using doesn't provide ability to upload files (which is annoying) and are bit slow in replying as well so I just wanted to avoid that. Anyway Thanks a lot for your help 🙂

Karan Kumar

Hi Team, could you please suggest a good SS4 module for robots.txt generation, there are quite a few options available, just wanted to have some feedback. Thanks 🙂

Karan Kumar

@Jose Not sure the purpose of it but did you have a look at this module: https://github.com/silverstripe/sspak

Show 1 attachment(s)
GitHub  
silverstripe/sspak

Tool for managing bundles of db/assets from SilverStripe environments - silverstripe/sspak

Hide attachment content
Karan Kumar

@StefGuev You can do that with UploadField but don't think there's core method to do that with HTMLEditorField, may be because HTMLEditorField UI does present you with an option to select your destination folder first prior to uploading.