silverstripe-framework/docs/en/02_Developer_Guides/14_Files/05_File_Migration.md

title: File migration summary: Manage migration of legacy files to the new database structure

File migration

This section describes how to upgrade existing filesystems from earlier versions of SilverStripe.

Running migration

Since the structure of File dataobjects has changed between 3.x and 4.x, a new task MigrateFileTask has been added to assist in migration of legacy files.

You can run this task on the command line:

$ ./vendor/bin/sake dev/tasks/MigrateFileTask

This task will also support migration of existing File objects to file versioning. Any pre-existing File objects will be automatically published to the live stage, to ensure that previously visible assets remain visible to the public site.

If additional security or visibility rules should be applied to File dataobjects, then make sure to correctly extend canView via extensions.

Automatic migration

Migration can be invoked by either this task, or can be configured to automatically run during dev build by setting the File.migrate_legacy_file config to true. However, it's recommended that this task is run manually during an explicit migration process, as this process could potentially consume large amounts of memory and run for an extended time.

SilverStripe\Assets\File:
  migrate_legacy_file: true

You can also run this task without CLI access through the queuedjobs module.

Migration of thumbnails

If you have the asset admin module installed this will also ensure that thumbnails for these images are generated when running 'MigrateFileTask'. Existing thumbnails will not be migrated however, and must be re-generated for use in the CMS.

Note: Thumbnails can be regenerated on a one-by-one basis within the CMS by re-saving it within the file edit details form.

Discarded files during migration

Note that any File object which is not in the File.allowed_extensions config will be deleted from the database during migration. Any invalid file on the filesystem will not be deleted, but will no longer be attached to a dataobject anymore, and should be cleaned up manually.

To disable this, set the following config:

SilverStripe\Assets\FileMigrationHelper:
  delete_invalid_files: false

Pre-existing file security solutions for 3.x (such as secure assets module) are likely incompatible with core file security. You should check the module README for potential upgrade paths.

Keeping archived assets

By default, "archived" assets (deleted from draft and live stage) retain their historical database entries with the file metadata, but the actual file contents are removed from the filesystem in order to avoid bloat. If you need to retain file contents (e.g. for auditing purposes), you can opt-in to this behaviour:

SilverStripe\Assets\Flysystem\FlysystemAssetStore:
  keep_archived_assets: true

Migrating substantial number of files

The time it takes to run the file migration will depend on the number of files and their size. The generation of thumbnails will depend on the number and dimension of your images.

If you are migrating a substantial number of files, you should run file migration task either as a queued job or on the command line. If the migration task fails or times out, you can start it again and it will pick up where it left off.

If your environement supports the Imagick PHP library, you may want to use that library instead of GD. Imagick is considerably faster when resizing images. You can switch back to GD after running the file migration task.

Changing the image manipulation driver to Imagick

If your project hosts big images (e.g. 4K images), this can also affect the amount of memory used to generate the thumbnails. The file migration task assumes that it will have at least 512MB of memory available.

By default the file migration task will not generate thumbnails for files greater than 9MB to avoid exhausting the available memory. To increase this limit, add the following code to your YML configuration:

SilverStripe\Core\Injector\Injector:
  SilverStripe\AssetAdmin\Helper\ImageThumbnailHelper:
    constructor:
      0: '100MB'

You can also set this to 0 to disable the limit.

System Requirements

The approach to running your file migration depends on your system and how many files you are migrating.

Use the following estimates to decide how you will run your file migration:

Number of files	Method	Expected Execution Time	Approximate Memory Usage
< 150	Web Request	30 seconds	6 MB
< 500	Queued Job	120 seconds	8 MB
< 10000	Command Line	10000 seconds	950 MB
10000+	Command Line or contact support	n/a	n/a

Your exact experience will vary based on your host server, the size of your files and other conditions. If your site is hosted on a managed environement (e.g.: Common Web Platform or SilverStripe Platform), you may not have access to the command line to manually run the migration task. Contact your hosting provider's helpdesk if that's your case.