Problem/Motivation
Flysystem 3.x is now stable (current stable release version is 3.19), and is receiving all feature enhancements and support. The new version has a much simpler and straightforward API that is more closely aligned with the StreamWrapper interface that is part of Drupal Core.
Proposed resolution
Let's refactor the Flysystem main and supporting module ecosystem to update the modules to be based on Flysystem 3.x.
Remaining tasks
* Create Drupal Flysystem 3.x branch.
* Assess Flysystem 3.0+ to determine the best path forward to integrate with Drupal 10:
* Develop a Roadmap for implementation, and break down tasks into an orderly implementation.
Initial assessment of the Flysystem 3.x library indicates that a roadmap that includes the following is probably the most straightforward implementation with the current Flysystem library:
- Leverage the existing Flysystem Filesystem Class to create an integration point that implements Drupal's StreamWrapper interface (Flysystem's Filesystem class has a 90% compliant implementation with StreamWrapper interface currently)
- Create an Adapter interface within the Drupal Flysystem module that adopts the Flysystem Adapter Interface, and implements any additional Drupal intergration points needed for implementation
- Write an Adapter for a Local filesystem stream wrapper, which is part of the base Flysystem 3.x library
- Develop an Adapter implementation plan for supplemental Drupal Flysystem modules, such as Flysystem S3, Flysystem FTP, etc.
User interface changes
* It appears that these would be minimal; however implementation of an admin UI to allow for configuration options to be managed through the UI would be a nice feature enhancement. Currently, most configuration for Flysystem stream wrappers takes place in the settings.php file directly.
API changes
* Since the API for Flysystem 3.x has significant changes from the 1.x version, documentation of API changes (which would be significant breaking changes) would be in order.
* It's worth noting that, while there are significant breaking changes it is likely that a "migration" would not be in order for content (file assets), but a minimal migration of configuration may be needed.
Data model changes
* Data model changes are not in order here; storage of data related to assets managed with the stream wrappers would leverage existing Drupal data structures, as does the current module versions.
League\Flysystem 3.x Overview
External Dependencies
league/mime-type-detectionUsed to detect nimetypes:
* based on file names and file contents (MimetypeDetection\MimeTypeDetector)
* based on file extension (MimetypeDetection\ExtensionLookup)
Interfaces in League/Flysystem v3
The two key interfaces here are FilesystemOperator and FilesystemAdapter.
FilesystemWriter
Contract interface for CUD (Create / Update / Delete) methods
FilesystemReader
Contract interface for R (Read) methods
FilesystemOperator extends FilesystemReader, FilesystemWriter
Main interface for the library. Implemented by Filesystem, which is the primary class through which interaction happens, including managing the adapters for different supported file systems.
FilesystemException extends Throwable
Main interface for the libraries Exception identification.
FilesystemOperationFailed extends FilesystemException
FilesystemAdapter
Contract interface for supported adapters. Class instances are injected into Filesystem instances to register various file systems interaction support.
PathNormalizer
Contract for path normalization methods. Class instances implementing this interface are injected into Filesystem instances.
ChecksumProvider
Contract for methods to provide ad-hoc chucksum calculations (defined as using MD5 hash for checksums)
StorageAttributes extends JsonSerializable, ArrayAccess
Contract for methods performing operations on File, Directory, and Mount Attributes
UnixVisibility/VisibilityConverter
Contract for methods managing permissions for Files and Directories.
UrlGeneration/TemporaryUrlGenerator
Contract for methods related to temporary file URLs
UrlGeneration/PublicUrlGenerator
Contract for methods related to public file URLs
| Comment | File | Size | Author |
|---|---|---|---|
| #10 | interdiff-9-10.txt | 1.02 KB | jan.stoeckler |
| #10 | flysystem-3195832-10.patch | 36.07 KB | jan.stoeckler |
| #9 | flysystem_asyncawss3.zip | 4.15 KB | jan.stoeckler |
Comments
Comment #2
lisa.rae commentedComment #3
lisa.rae commentedComment #4
lisa.rae commentedComment #5
BioALIEN commentedI recommend you use the 2.0.x branch rather than 3.0.x as no stable release has been cut on the 2.x branch.
Comment #6
bradjones1@mglaman is working on this on a few related issues.
Comment #7
mxr576... And Flysystem 3 is out...
https://packagist.org/packages/league/flysystem#3.0.0
https://blog.frankdejonge.nl/flysystem-3-0-0-is-released/
Probably it is time to reevaluate plans
Comment #8
bradjones1No real significant changes in 3.x, but yeah, this is mostly blocked on bandwidth.
Comment #9
jan.stoecklerHey folks,
this is an attempt to port the flysystem.module to Flysystem v2/3 API with as little architectural changes to the module as possible, so basically foregoing the refactoring mentioned in the issue summary as a first step to get something working. Local and FTP adapters are working on my end, as well as the DrupalCacheAdapter. I have not touched any tests yet, so those are very likely to fail.
In case anyone wants to try this out, and/or needs a working S3 integration, there's also an example provider module using the AsyncAws S3 Adapter attached: https://www.drupal.org/files/issues/2023-06-22/flysystem_asyncawss3.zip.
Would appreciate any feedback and/or guidance on if and how to proceed further. Thanks!
(Patch is rolled against 2.1.x, can reroll/change that if appropriate.)
Comment #10
jan.stoecklerAdded a small fix to
__flysystem_check_dependenciesto account for updated dependencies.Comment #13
lisa.rae commentedComment #16
lisa.rae commentedComment #17
lisa.rae commentedComment #22
lisa.rae commentedComment #23
lisa.rae commentedComment #24
lisa.rae commentedComment #25
lisa.rae commentedComment #26
lisa.rae commentedComment #27
lisa.rae commentedComment #28
lisa.rae commentedComment #33
lisa.rae commentedComment #34
lisa.rae commentedComment #35
lisa.rae commented