tractorcow

there's a lot of internal api in the flysystem backend you can ignore

tractorcow

as long as you fill in the AssetStore interface properly it should work

tractorcow

I wrote one that writes to the local filesystem

tractorcow

You can capture that in your user signup / update email forms though

tractorcow

ping me in PM if either of you need to discuss 😛

tractorcow

https://github.com/silverstripe/silverstripe-assets/pull/223

Show 1 attachment(s)
maxime-rainville

Change to how paths works

This PR allows the protected and public file store to use different file format. The two principal file format are:

• Natural path (actual/path/shown/in/the/cms.txt) which is equivalent to the path used when legacy file names is enabled. • Hash path (some/folder/abcdf7890/myfile.txt) which is equivalent to the default path used in SilverStripe 4.0 to 4.3.

Hash paths provide a way for multiple versions of the same file to co-exist which is needed for protected files. Natural paths are easier to understand and persistent across versions which pleases search engines and smelly humans.

The public store now defaults to using natural path, while the protected store still uses hash paths.

Redirection

Just like the recent change to SS4.3, each store will try to resolve paths that are in a different format to a matching file in a different format. This will allow sites that are using the old format to upgrade to the newer one progressively, without the need to immediately run a file migration task.

How is this magic handled

Three new constructs have been added FileIDHelper, FileResolutionStrategy and ParsedFileID

In this context, a file ID is a string pointing to a file relative to the root of a file store. It's a file path by another name.

FileIDHelper

Each class implementing the FileIDHelper interface represent a file format and provide logic to decode and encode file IDs. FileIDHelpers are glorified regular expression boxes. They don't look directly at files ... only at FileID strings.

3 FileIDHelpers are povided:

NaturalFileIDHelper which defines natural paths ; • HashFileIDHelper which defines hash paths ; • LegacyFileIDHelper which is very similar NaturalFileIDHelper, but can understand SS3 variant fileID (e.g. my-folder/_resampled/ResizeXYZ/image.jpg)

In theory, a developer who wanted to use their own custom file ID format could create their own FileIDHelper. Or someone who wanted to keep using the SS3 format could define LegacyFileIDHelper as their default. Neither of those scenarios have been tested however and are not officially supported.

FileResolutionStrategy

A FileResolutionStrategy is responsible for trying to understand a FileID provided in an unknown format and find an existing file in a potentially different format. This bridges the gap between FileIDHelpers and the file system.

A FileResolutionStrategy isn't associated to a specific Filesystem. Instead most method on the FileResolutionStrategy expect to be provided a Filesystem reference.

There's only one implementation of FileResolutionStrategy at this moment: FileIDHelperResolutionStrategy. This helper expects to be provided a default FileIDHelper which will be used to generate new FileIDs for new files and a list of resolution FileIDHelpers which will be used to try to resolve request for existing files.

FileIDHelperResolutionStrategy will also attempt to read information from the DB to find information about files and redirect users to newer version of past files. FileIDHelperResolutionStrategy accepts a Versioned stage to control on what stages we'll look for files.

The Protected store and public store will both have their own unique FileResolutionStrategy.

ParsedFileID

A ParseFileID is a more sophisticated equivalent of a file tuple array that gets passed around between AssetStore, the FileResolutionStrategy and the FileIDHelper. It has the following properties:

• Filename • Hash • Variant • FileID

It's just a convenient way to encapsulate some File identification data. ParseFileID is immutable but it has some setter methods. When calling those setter methods, you will get a slightly a new copy with some slightly altered data.

What else has change

The bulk of the other change have occurred on FlysystemAssetStore. Most of the changes involves figuring out on what store an operation should occur and calling the right combination of Filesystem and ResolutionStrategy.

That magic occurs in applyToFileOnFilesystem(). This method receives some file information and will be looking for a matching physical file with these steps:

• an exact match on the public store, • an exact match on the protected store, • a resolution match on the public store, • a resolution match on the protected store.

Once applyToFileOnFilesystem has manage to resolve a file, it will call a closure with a matching ParsedFileID, a Filesystem, a FileResolutionStrategy and a visibility flag.

Other changes

• There's 3 methods that can be use to write files on the AssetStore. They use to be completely separate. setFromLocalFile and setFromString now wrap their data around a stream resource and call setFromStream. • A bunch of methods have been marked as protected methods on FlysystemAssetStore have been marked as deprecated. They usually have an equivalent private method. I don't want to remove them because some people may be relying on them in their own custom asset store, but it's annoying as hell that we have to keep them around. • There's a new swapPublish method on FlysystemAssetStore. I didn't want to make it part of the interface to avoid breaking people code. That methods stashes existing publish files to avoid overriding them. • Following discussion with https://github.com/chillu|@chillu, legacy_filenames is now compeltly ignored and doesn't do anything. • FlysystemAssetStore was throwing League\Flysystem\Exception in many places. This was a bit confusing because people could have read this to mean that we were throwing native PHP \Exception. I've aliased League\Flysystem\Exception to FlysystemException for clarity.

Testing this crazy thing

When testing this locally, you probably want to remove this line from the .htaccess file in your assets folder: RewriteCond %{REQUEST_FILENAME} !-f. Otherwise a big chunk of your request will never it PHP.

Another thing to keep in mind is that his new approach does a lot more file existence check and file reads (to build hashes) than the previous ones. There's not really a way around it because of the nature of the change, but we do want to make sure we don't affect serving time to much.

Things to pay special attention to

• The deprecated protected methods on FlysystemAssetStore don't have full unit test coverage and they are not indirectly call by other methods any more. • The order in which FileIDHelpers are added to a FileIDHelperResolution strategy is very important. NaturalFileIDHelper adn LegacyFileIDHelper will interpret the hash in a Hash path as folder name. NaturalFileIDHelper adn LegacyFileIDHelper are only different when looking at variants.

Important things left to do

• Update the file migration script • Run the asset-admin unit test and behat test on this crazy thing • Run the kitchen sink test against this crazy thing

Parent issue

https://github.com/silverstripe/silverstripe-versioned/issues/177|silverstripe/silverstripe-versioned#177

Hide attachment content
(1)