silverstripe-framework/docs/en/04_Changelogs/4.4.0.md

4.4.0 (Unreleased)

Overview

• Optional migration to hash-less public asset URLs
• Optional migration of legacy thumbnail locations
• Correct PHP types are now returned from database queries
• Server Requirements have been refined: MySQL 5.5 end of life reached in December 2018, thus SilverStripe 4.4 requires MySQL 5.6+.
  SilverStripe 4.3 and prior still support MySQL 5.5 for their own lifetime.
• The name of the directory where vendor module resources are exposed can now be configured by defining a extra.resources-dir key in your composer.json file. If the key is not set, it will automatically default to resources. New projects will be preset to _resources. This will avoid potential conflict with SiteTree URL Segments.
• dev/build is now non-destructive for all Enums, not just ClassNames. This means your data won't be lost if you're switching between versions, but watch out for code that breaks when it sees an unrecognised value!
• Removed File.migrate_legacy_file config option. Migration tasks now need to run via dev/tasks/, running them as part of dev/build is no longer supported

Upgrading

Adopting to new _resources directory

The name of the directory where vendor module resources are exposed can now be configured by defining a extra.resources-dir key in your composer.json file. If the key is not set, it will automatically default to resources. New projects will be preset to _resources. This will avoid potential conflict with SiteTree URL Segments.

1. Update your .gitignore file to ignore the new _resources directory. This file is typically located in the root of your project or in the public folder.
2. Add a new extra.resources-dir key to your composer file.

{
    // ...
    "extra": {
        // ...
        "resources-dir": "_resources"
    }
}

3. Expose your vendor assets by running composer vendor-expose.
4. Remove the old resources folder. This folder will be located in the public folder if you have adopted the public web root, or in the root of your project if you haven't.

You may also need to update your server configuration if you have applied special conditions to the resources path.

Optional migration tasks: Overview

SilverStripe 4.x introduced an asset abstraction system which required a file migration task.

We have ironed out some edge cases since then:

• Hash-less public asset URLs
• Legacy thumbnail locations
• TODO Shortcode migration

You can opt-in to performing these migration tasks on your already upgraded SilverStripe 4.x project. Projects which are upgraded from 3.x to 4.4 or newer will run these tasks by default. You can perform these tasks in the background without CLI access by installing and configuring the queuedjobs module.

Optional migration tasks: Hash-less public asset URLs

It allowed files to be access protected, have a separate draft stage, and track changes to file metadata and their contents through the Versioned system.

This change defaulted to adding a content "hash" to file paths, unless the migration was performed with legacy_filenames=true. The hash would be updated when file contents change, and any link generated through SilverStripe (e.g. through HTMLText or $Image template placeholders) would automatically adjust. However, any direct links from search engines, bookmarks, etc would break. This limitation was pointed out in the upgrading advice to developers, but the impact wasn’t sufficiently highlighted.

SilverStripe 4.3.2 introduced a redirect to fix those broken links. Dynamic redirects are more resource intensive than serving static files. With SilverStripe 4.4.0, we're providing an optional migration script to move public files into their "hash-less" locations, removing the need for most redirects. The original "hashed" file paths continue to work through redirects. In order to opt-in to moving these files, run the following command:

vendor/bin/sake dev/tasks/MigrateFileTask

Further information is provided in the Hash-less Public Asset URLs FAQ below, as well as the Github issue.

Optional migration tasks: Legacy thumbnail locations

Alongside the new asset abstraction, we've also changed where thumbnails are stored. The file format is now generalised as a "variant", and no longer lives in a _resampled folder. Prior to this release, these thumbnails were left in place. For thumbnails generated through PHP or template code, these are generated on demand in their new location, causing duplication and unnecessary storage space.

You can opt-in to moving these legacy thumbnails to their new locations with a subtask of dev/tasks/MigrateFileTask. If newer thumbnails have been generated, it'll keep those. New migrations from 3.x to 4.x will include this migration step by default.

vendor/bin/sake dev/tasks/MigrateFileTask only=move-thumbnails

Note that as part of the hash-less public asset URLs introduced in this release, requests to these legacy thumbnails will automatically redirect to their new locations. Review the Github issue for more details.

Optional migration tasks: Remove artefacts from "secureassets" module

The silverstripe/secureassets module allowed opt-in file security on SilverStripe 3.x sites. It is included by default in the Common Web Platform (CWP) 1.x recipe as well. The module uses the same metadata as the asset abstraction built-in to SilverStripe 4.x (File.CanViewType), so no migration is required for existing data.

The module worked by placing .htaccess files in the specific folders it protected. The 4.x asset abstraction places files in a separate assets/.protected store instead. Those leftover .htaccess files can cause issues with accessing public files. In order to remove them, run the following task.

vendor/bin/sake dev/tasks/MigrateFileTask only=fix-secureassets

The task will auto-detect if you have any custom .htaccess files in folders, and retain those. Note that you'll need to run your own migration scripts if you've used the module with IIS and web.config files instead.

Hash-less Public Asset URLs FAQ

How are files named and renamed?

Here's an example of how file names changed during an initial 4.x migration, and when upgrading to 4.4:

• Original file created under SS 3.x: assets/myfile.pdf
• File migrated under SS 4.x with legacy_filenames=false: assets/[content-hash]/myfile.pdf (Regression: links to assets/myfile.pdf no longer work)
• File migrated under SS 4.x with legacy_filenames=true: assets/myfile.pdf
• File with updated file content under SS 4.x: assets/[new-content-hash]/myfile.pdf (Regression: links to assets/[content-hash]/myfile.pdf no longer work)
• File with hotfix applied (SS 4.2.3): assets/myfile.pdf and assets/[old-content-hash]/myfile.pdf redirects to assets/[new-content-hash]/myfile.pdf
• File with full fix applied (SS 4.3.x) : assets/myfile.pdf and assets/[old-content-hash]/myfile.pdf redirects to assets/[new-content-hash]/myfile.pdf
• Newly uploaded file with full fix applied (SS 4.4.0): assets/my-other-file.pdf (no redirect required)

More details on how files are stored can be found in the "File Storage" guide.

How are redirects handled?

Redirects from (now legacy) hashed public URLs default to 301 Permanent Redirect. By opting into the file migration to hash-less public URLs, you can minimise these redirects. SilverStripe will automatically generate hash-less public URLs regardless, but external links might still point to legacy hashed public URLs. If you have a high traffic site with lots of direct references to asset URLs (e.g. search engines indexing popular PDF documents), we recommend that you configure HTTP caching for these redirects in your webserver (e.g. through .htaccess).

The redirect code can be configured via FlysystemAssetStore.permanent_redirect_response_code.

If you upgrade an older SilverStripe 4 project to SilverStripe 4.4 and choose not to run the file migration task, your files will still be stored under an "hash" folder. Browsers who try to access this file via the "hashless" url will be redirected to the "hash" URL with a 302 Temporary Redirect.

The temporary redirect code can be configured via FlysystemAssetStore.redirect_response_code.

Do I need to regenerate HTML with existing links to assets?

Pages and other views can contain links to asset locations, e.g. as HTML content rendered with <img> and <a> tags. These might point to old locations of files (prior to running the optional migration). While SilverStripe will automatically fix these references the next time the view is rendered, this content is often cached (e.g. in a CDN).

We have implemented an automatic redirect for URLs pointing to asset locations prior to running the optional migration script, so there is no need to regenerate any content.

Is there any data loss or data integrity issue?

There are no known issues around data integrity. All files that were available on a SS 3.x site are still available after upgrading to SS 4.x.

Will this change affect my search engine ranking?

As long as your files are still linked on your website, search engines will pick up the new links on any projects which have already been migrated. The 4.3.2 bugfix will redirect links, which passes on any SEO rankings to the new link location. Since links to files should be permanent after the bugfix has been applied, this can lead to improved search engine rankings (since existing files under new links don’t need to be re-ranked by search engines).

Can I migrate away from the legacy_filenames=true option?

Yes. Simply disable legacy_filenames in your YML configuration, then run the MigrateFileTask. This will normalise the path of all existing assets.

Can I still choose legacy_filenames=true when starting new upgrades?

Technically yes, but you need to be aware of the tradeoffs. Once the regression has been fixed, we don’t see the need for people to choose this option any more.

SilverStripe 4.4, ignores the legacy_filenames in most situation. Enabling legacy_filenames on a fresh SilverStripe 4.4 installation will have no affect.

How do I test that the fix has worked on my site?

Before applying the upgrade and migration script: Find an existing published and draft file. Get the link by clicking on the preview in admin/assets. It should link to assets/[hash]/myfile.pdf.

After applying the upgrade and migration task, you can test that it applied correctly. Please perform these tests on a test or UAT environment, since your local environment might have different routing conditions.

• Check that the file still routes correctly with the hash in place (should redirect)
• Remove [hash]/, and check that it still routes correctly (should not redirect)

Will this patch redirect URLs for previous file versions?

Yes, it will attempt to find the most recent public "hash-less" URL for this file and redirect to it.

Changes to internal APIs

• PDOQuery::__construct() now has a 2nd argument. If you have subclassed PDOQuery and overridden __construct() you may see an E_STRICT error
• The name of the directory where vendor module resources are exposed can now be configured by adding a extra.resources-dir key to your composer file. The new default in silverstripe/installer has been changed to _resources rather than resources. This allows you to use resources as a URL segment or a route.