silverstripe-framework/docs/en/04_Changelogs/4.0.0.md

4.0.0 (unreleased)

Overview

Framework

• Minimum PHP version raised to 5.5.0
• Deprecate SQLQuery in favour SQLSelect
• DataList::filter by null now internally generates "IS NULL" or "IS NOT NULL" conditions appropriately on queries
• DataObject::database_fields now returns all fields on that table.
• DataObject::db now returns composite fields.
• DataObject::ClassName field has been refactored into a DBClassName type field.
• Image manipulations have been moved into a new [api:ImageManipulation] trait.
• CMSFileAddController removed.
• UploadField::setAllowedFileCategories('image') now excludes non-resizeable images. 'unresizeable_image' is can be used to validate these types.
• Image_Backend API now loads and saves from AssetContainer instances rather than local files.
• The following File categories have been renamed: 'zip' to 'archive', 'doc' to 'document', and 'mov' to 'video'
• File::updateLinks no longer takes urls as parameters. All file links are now identified either by the DataObject::ID in a data-fileid property, or via shortcodes. This is necessary because file urls are no longer able to identify assets.
• Extension point HtmlEditorField::processImage has been removed, and moved to Image::regenerateImageHTML
• Upload::load now stores assets directly without saving into a File dataobject.
• Protected file storage is now a core Framework API. See [/developer_guides/files/file_security] for more information.
• Object::useCustomClass has been removed. You should use the config API with Injector instead.
• Upgrade of TinyMCE to version 4.
• File is now versioned, and should be published before they can be used on the frontend. See section on Migrating File DataObject from 3.x to 4.0 below for upgrade notes.
• Removed dev/tests/ controller in favour of standard vendor/bin/phpunit command
• Updated PHPUnit from 3.7 to 4.8 (upgrade notes). Please remove any PHPUnit related require_once() calls (e.g. in FeatureContext definitions of the behat-extension module). Run `composer require --dev 'phpunit/phpunit:~4.8' on existing projects to pull in the new dependency.

New API

• New filesystem abstraction including new DBFile database field to hold file references.
• ShortcodeHandler interface to help generate standard handlers for HTML shortcodes in the editor.
• AssetNameGenerator interface, including a DefaultAssetNameGenerator implementation, which is used to generate renaming suggestions based on an original given filename in order to resolve file duplication issues.
• GeneratedAssetHandler API now used to store and manage generated files (such as those used for error page cache or combined files).
• Requirements_Minifier API can be used to declare any new mechanism for minifying combined required files. By default this api is provided by the JSMinifier class, but user code can substitute their own.
• AssetField formfield to provide an UploadField style uploader for the new DBFile database field.
• AssetControlExtension is applied by default to all DataObjects, in order to support the management of linked assets and file protection.
• ProtectedFileController class is used to serve up protected assets.
• Object has been broken up into various traits, each of which can be added to other objects independently:
  • Configurable Provides Config API helper methods
  • Injectable Provides Injector API helper methods
  • Extensible Allows extensions to be applied
• Removed ability to run tests via web requests (http://mydomain.com/dev/tests), use the standard CLI command instead (vendor/bin/phpunit)
• Removed dev/jstests/ controller (no replacement)
• Moved test database cleanup task from sake dev/tests/cleanupdb to sake dev/tasks/CleanupTestDatabasesTask
• Removed TestRunner and JSTestRunner APIs
• Removed PhpUnitWrapper, PhpUnitWrapper_3_4, PhpUnitWrapper_3_5, PhpUnitWrapper_Generic, SapphireTestSuite APIs
• Removed SapphireTest->skipTest, use markTestSkipped() in a setUp() method instead
• HtmlEditorConfig is now an abstract class, with a default implementation TinyMCEConfig for the built in TinyMCE editor.
• HtmlEditorField::setEditorConfig may now take an instance of a HtmlEditorConfig class, as well as a standard config identifier name.
• A lot of standard versioned API has been refactored from SiteTree into Versioned extension. Now all versioned DataObjects have canPublish(), canArchive(), canUnpublish(), doPublish(), doArchive() doUnpublish(), isPublished() and isonDraft() out of the box. However, do*() methods will no longer automatically check can*() permissions, and must be done by usercode before invocation.
• GridField edit form now has improved support for versioned DataObjects, with basic publishing actions available when editing records.

Front-end build tooling for CMS interface

• Management of CMS JavaScript dependencies via npm
• ES6 is now available in all core JavaScript files via Babel
• Bundling of core JavaScript via Browserify
• CMS build tasks managed by Gulp

For more details see the docs on working with client-side dependencies. Note that you don't need any of these tools for running SilverStripe or developing your own website. These improvements are mainly geared at CMS core development.

Added new front-end UI libraries

• Added the following npm libraries:
  • react
  • redux
  • react-redux
  • react-dom
  • isomorphic-fetch
  • redux-thunk
  • react-addons-test-utils
• Added SilverStripeComponent base class for building React UIs

Compiling CSS

Compass has been removed and css/sprite compilation is now handled with gulp (see Markup and Style Conventions).

Bootstrap CSS

Bootstrap 4 Alpha.2 CSS is available in admin/css/bootstrap.

Deprecated classes/methods

Core

• debugmethods querystring argument has been removed from debugging.

ORM

• DataList::getRelation is removed, as it was mutable. Use DataList::applyRelation instead, which is immutable.

ErrorPage

• ErrorPage.static_filepath config has been removed.
• ErrorPage::get_filepath_for_errorcode has been removed
• ErrorPage::alternateFilepathForErrorcode extension point has been removed

See notes below on upgrading extensions to the ErrorPage class

Member

• Member Field 'RememberLoginToken' removed, replaced with 'RememberLoginHashes' has_many relationship

Assets and Filesystem

The following image manipulations previously deprecated has been removed:

• Image::SetRatioSize superceded by Fit
• Image::SetWidth superceded by ScaleWidth
• Image::SetHeight superceded by ScaleHeight
• Image::SetSize superceded by Pad
• Image::PaddedImage superceded by Pad
• Image::CroppedImage superceded by Fill
• Image::AssetLibraryPreview superceded by PreviewThumbnail
• Image::AssetLibraryThumbnail superceded by CMSThumbnail

The following File methods have been removed. Since there is no longer any assumed local path for any file, methods which dealt with such paths may no longer be relied on.

• File::deletedatabaseOnly
• File::link_shortcode_handler renamed to handle_shortcode
• File::setParentID
• File::getFullPath
• File::getRelativePath
• File::Content database field is removed

Image manipulations have been moved out of Image.php and now available to any File or DBFile which has the appropriate mime types. The following file manipulations classes and methods have been removed:

• CleanImageManipulationCache class
• Image_Cached class
• Image::regenerateFormattedImages
• Image::getGeneratedImages
• Image::deleteFormattedImages
• AssetAdmin::deleteunusedthumbnails
• AssetAdmin::getUnusedThumbnails

Many Folder methods have also been removed:

• Folder::syncChildren
• Folder::constructChild
• Folder::addUploadToFolder

The following filesystem synchronisation methods are also removed

• Filesystem::sync
• AssetAdmin::doSync

The Spyc YAML library has been removed from /thirdparty. Please load it yourself, or use the Symfony YAML component that’s automatically installed by composer.

Requirements

The following methods and properties on Requirements_Backend have been renamed:

• Requirements_Backund::$combine_files made protected and renamed $combinedFiles
• Requirements_Backend::$combine_js_with_min made protected and renamed $minifyCombinedFiles
• Requirements_Backend::$write_header_comments made protected and renamed $writeHeaderComment
• Requirements_Backend::$write_js_to_body made protected and renamed to $writeJavascriptToBody
• Requirements_Backend::$force_js_to_bottom renamed to $forceJSToBottom
• get_combined_files_enabled renamed to getCombinedFilesEnabled
• set_combined_files_enabled renamed to setCombinedFilesEnabled
• get_suffix_requirements renamed to getSuffixRequirements
• set_suffix_requirements renamed to setSuffixRequirements
• get_custom_scripts renamed to getCustomScripts
• unblock_all renamed to unblockAll
• include_in_response renamed to includeInResponse
• combine_files renamed to combineFiles
• get_combine_files renamed to getCombinedFiles
• clear_combined_files renamed to clearCombinedFiles
• process_combined_files renamed to processCombinedFiles
• set_write_js_to_body renamed to setWriteJavascriptToBody
• set_force_js_to_bottom renamed to setForceJSToBottom

New methods on Requirements are added to access these:

• get_minify_combined_js_files
• set_minify_combined_js_files
• get_force_js_to_bottom
• get_write_js_to_body

And some methods on Requirements and Requirements_Backend have been removed as they are obsolete.

• delete_combined_files (both classes)

A new config Requirements_Backend.combine_in_dev has been added in order to allow combined files to be forced on during development. If this is off, combined files is only enabled in live environments.

In addition, a new API for javascript files has been added to support front-end tools such as browserify that pre-combine scripts. You can specify a 'provides' option to the second parameter of Requirements::javascript to declare included files.

E.g.

:::php
Requirements::javascript('mysite/js/dist/bundle.js', ['provides' => [
    'mysite/js/jquery.js'
    'mysite/js/src/main.js',
    'mysite/js/src/functions.js'
]]);

Upgrading

Update code that uses SQLQuery

Where your code once used SQLQuery you should now use SQLSelect in all cases, as this has been removed.

See the 3.2.0 upgrading notes for details on how existing code should be upgraded.

New asset storage mechanism

File system has been abstracted into an abstract interface. By default, the out of the box filesystem uses Flysystem with a local storage mechanism (under the assets directory).

Because the filesystem now uses the sha1 of file contents in order to version multiple versions under the same filename, the default storage paths in 4.0 will not be the same as in 3.

In order to retain existing file paths in line with framework version 3 you should set the \SilverStripe\Filesystem\Flysystem\FlysystemAssetStore.legacy_paths config to true. Note that this will not allow you to utilise certain file versioning features in 4.0.

:::yaml
\SilverStripe\Filesystem\Flysystem\FlysystemAssetStore:
  legacy_paths: true

See [/developer_guides/files/file_management] for more information on how the new system works.

Migrating File DataObject from 3.x to 4.0

Since the structure of File dataobjects has changed, a new task MigrateFileTask has been added to assist in migration of legacy files. Migration can be invoked by either this task, or can be configured to automatically run during dev build by setting the File.migrate_legacy_file config to true. However, it's recommended that this task is run manually during an explicit migration process, as this process could potentially consume large amounts of memory and run for an extended time.

:::yaml
File:
  migrate_legacy_file: true

This task will also support migration of existing File DataObjects to file versioning. Any pre-existing File DataObjects will be automatically published to the live stage, to ensure that previously visible assets remain visible to the public site.

If additional security or visibility rules should be applied to File dataobjects, then make sure to correctly extend canView via extensions.

Upgrade code which acts on Image

As all image-specific manipulations has been refactored from Image into an ImageManipulations trait, which is applied to both File and DBFile. These both implement a common interface AssetContainer, which has the getIsImage() method. In some cases, it may be preferable to invoke this method to detect if the asset is an image or not, rather than checking the subclass, as the asset may also be a DBFile with an image filter applied, rather than an instance of the Image dataobject.

In addition, a new file category image/supported has been added, which is a subset of the image category. This is the subset of all image types which may be assigned to the [api:Image] dataobject, and may have manipulations applied to it. This should be used as the file type restriction on any [api:UploadField] which is intended to upload images for manipulation.

Before:

:::php
if($file instanceof Image) {
	$upload = new UploadField();
	$upload->setAllowedFileCategories('image');
}

After:

:::php
if($file->getIsImage()) {
	$upload = new UploadField();
	$upload->setAllowedFileCategories('image/supported');
}

In cases where image-only assets may be assigned to relationships then your datamodel should specify explicitly an Image datatype, or refer to DBFile('image/supported').

E.g.

:::php
class MyObject extends DataObject {
	private static $has_one = array(
		"ImageObject" => "Image"
	);
	private static $db = array(
		"ImageField" => "DBFile('image/supported')"
	);
}

Upgrading code that writes to File dataobjects, or writes files to the 'assets' folder

In the past all that was necessary to write a File dataobject to the database was to ensure a physical file existed in the assets folder, and that the Filename of the dataobject was set to the same location.

Since the storage of physical files is no longer a standard location, it's necessary to delegate the writing of such files to the asset persistence layer. As a wrapper for an individual file, you can use any of the setFrom* methods to assign content from a local (e.g. temporary) file, a stream, or a string of content.

You would need to upgrade your code as below.

Before:

:::php
function importTempFile($tmp) {
	copy($tmp, ASSETS_PATH . '/imported/' . basename($tmp));
	$file = new File();
	$file->setFilename('assets/imported/'.basename($tmp));
	$file->write();
}

After:

:::php
function importTempFile($tmp) {
    Versioned::reading_stage('Stage');
	$file = new File();
	$file->setFromLocalFile($tmp, 'imported/'.basename($tmp));
	$file->write();
	$file->doPublish();
}

Note that 'assets' is no longer present in the new code, and the path beneath what was once assets is now used to generate the 'filename' value. This is because there is no longer an assumption that files are stored in the assets folder.

There are other important considerations in working with File dataobjects which differ from legacy:

• File synchronisation is no longer automatic. This is due to the fact that there is no longer a 1-to-1 relationship between physical files and File dataobjects.
• Folder dataobjects are now purely logical dataobjects, and perform no actual filesystem folder creation on write.
• All Files are versioned, which means that by default, new File records will not be visibile to the public site. You will need to make sure to invoke ->doPublish() on any File dataobject you wish visitors to be able to see.

You can disable File versioning by adding the following to your _config.php

:::php
File::remove_extension('Versioned');

Upgrading code performs custom image manipulations

As file storage and handling has been refactored into the abstract interface, many other components which were once specific to Image.php have now been moved into a shared ImageManipulation trait. Manipulations of file content, which are used to generate what are now called "variants" of assets, is now a generic api available to both File and DBFile classes through this trait.

Custom manipulations, applied via extensions, must be modified to use the new API.

For instance, code which sizes images to a fixed width should be updated as below:

Before:

:::php
// in MyImageExtension.php
class MyImageExtension extends DataExtension {
	public function GalleryThumbnail($height) {
		return $this->getFormattedImage('GalleryThumbnail', $height);
	}
	public function generateGalleryThumbnail(Image_Backend $backend, $height) {
		return $backend->paddedResize(300, $height);
	}
}

// in _config.php
Image::add_extension('MyImageExtension');

Now image manipulations are implemented with a single method via a callback generator.

After:

:::php
// in MyImageExtension.php
class MyImageExtension extends Extension {
	public function GalleryThumbnail($height) {
		// Generates the manipulation key
		$variant = $this->owner->variantName(__FUNCTION__, $height);

		// Instruct the backend to search for an existing variant with this key,
		// and include a callback used to generate this image if it doesn't exist
		return $this->owner->manipulateImage($variant, function(Image_Backend $backend) use ($height) {
			return $backend->paddedResize(300, $height);
		});
	}
}

// in _config.php
File::add_extension('MyImageExtension');
DBFile::add_extension('MyImageExtension');

There are a few differences in this new API:

• The extension is no longer specific to dataobjects, so it uses the generic 'Extension' class instead of 'DataExtension'
• This extension is added to both DBFile and File, or order to make this manipulation available to non-dataobject file references as well, but it could be applied to either independently.
• A helper method variantName is invoked in order to help generate a unique variant key. Custom code may use another generation mechanism.
• Non-image files may also have manipulations, however the specific manipulateImage should not be used in this case. A generic manipulate method may be used, although the callback for this method both is given, and should return, an AssetStore instance and file tuple (Filename, Hash, and Variant) rather than an Image_Backend.

Upgrading code that uses composite db fields.

CompositeDBField is now an abstract class, not an interface. In many cases, custom code that handled saving of content into composite fields can be removed, as it is now handled by the base class.

The below describes the minimum amount of effort required to implement a composite DB field.

:::php
<?
class MyAddressField extends CompositeDBField {

	private static $composite_db = array(
		'Street' => 'Varchar(200)',
		'Suburb' => 'Varchar(100)',
		'City' => 'Varchar(100)',
		'Country' => 'Varchar(100)'
	);

	public function scaffoldFormField($title = null) {
		new AddressFormField($this->getName(), $title);
	}
}

Upgrading code that references DataObject::database_fields or DataObject::db

These methods have been updated to include base fields (such as ID, ClassName, Created, and LastEdited), as well as composite DB fields.

DataObject::database_fields does not have a second parameter anymore, and can be called directly on an object or class. E.g. Member::database_fields()

If user code requires the list of fields excluding base fields, then use custom_database_fields instead, or make sure to call unset($fields['ID']); if this field should be excluded.

DataObject:db() will return all logical fields, including foreign key ids and composite DB Fields, alongside any child fields of these composites. This method can now take a second parameter $includesTable, which when set to true (with a field name as the first parameter), will also include the table prefix in Table.ClassName(args) format.

Update code that uses SQLQuery

SQLQuery is still implemented, but now extends the new SQLSelect class and has some methods deprecated. Previously this class was used for both selecting and deleting, but these have been superceded by the specialised SQLSelect and SQLDelete classes.

Take care for any code or functions which expect an object of type SQLQuery, as these references should be replaced with SQLSelect. Legacy code which generates SQLQuery can still communicate with new code that expects SQLSelect as it is a subclass of SQLSelect, but the inverse is not true.

Update implementations of augmentSQL

Since this method now takes a SQLSelect as a first parameter, existing code referencing the deprecated SQLQuery type will raise a PHP error.

E.g.

Before:

:::php
function augmentSQL(SQLQuery &$query, DataQuery &$dataQuery = null) {
	$locale = Translatable::get_current_locale();
	if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhere())))  {
		$qry = sprintf('"Locale" = \'%s\'', Convert::raw2sql($locale));
		$query->addWhere($qry); 
	}
}

After:

:::php
function augmentSQL(SQLSelect $query, DataQuery $dataQuery = null) {
	$locale = Translatable::get_current_locale();
	if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhereParameterised($parameters))))  {
		$query->addWhere(array(
			'"Locale"' => $locale
		));
	}
}

Update code that modifies the behaviour of ErrorPage

ErrorPage has been updated to use a configurable asset backend, similar to the AssetStore described above. This replaces the ErrorPage.static_filepath config that was used to write local files.

As a result, error pages may be cached either to a local filesystem, or an external Flysystem store (which is configured via setting a new Flysystem backend with YAML).

ErrorPage::get_filepath_for_errorcode() has been removed, because the local path for a specific code is no longer assumed. Instead you should use ErrorPage::get_content_for_errorcode which retrieves the appropriate content for that error using one of the methods above.

In order to retrieve the actual filename (which is used to identify an error page regardless of base path), you can use ErrorPage::get_error_filename() instead. Unlike the old get_filepath_for_errorcode method, there is no $locale parameter.

In case that user code must customise this filename, such as for extensions which provide a locale value for any error page, the extension point updateErrorFilename can be used. This extension point should also be used to replace any alternateFilepathForErrorcode used.

E.g.

:::php
class MyErrorPageExtension extends SiteTreeExtension {
	public function updateErrorFilename(&$name, &$statuscode) {
		if($this->owner->exists()) {
			$locale = $this->Locale;
		} else {
			$locale = Translatable::get_current_locale();
		}
		$name = "error-{$statusCode}-{$locale}.html";
	}
}


:::yaml
ErrorPage:
  extensions:
    - MyErrorPageExtension

Upgrading asset web.config, .htaccess, or other server configuration

Server configuration files for /assets are no longer static, and are regenerated via a set of standard silverstripe templates on flush. These templates include:

• Assets_HTAccess.ss: Template for public permissions on the Apache server.
• Assets_WebConfig.ss: Template for public permissions on the IIS server.
• Protected_HTAccess.ss: Template for the protected store on the Apache server (should deny all requests).
• Protected_WebConfig.ss: Template for the protected store on the IIS server (should deny all requests).

You will need to make sure that these files are writable via the web server, and that any necessary configuration customisation is done via overriding these templates.

If upgrading from an existing installation, make sure to invoke ?flush=all at least once.

See [/developer_guides/files/file_security] for more information.

ListboxField is now multiple-only

Previously, this field would operate as either a single select (default) or multi-select by setting setMultiple to either true or false.

Now this field should only be used for multi-selection. Single-selection should be done using a regular DropdownField.

GroupedDropdownField::setDisabled now only accepts a list of values.

Where previously you could specify a list of grouped values in the same way as setSource, this method now only accepts either a non-associative array of values (not titles) or an SS_List of items to disable.

Upgrading Requirements file paths

Requirements now throws an exception then a file is not found, rather than failing silently, so check your Requirements are pointing to files that exist.

Core JavaScript files paths have changed, so if you're referencing these, you'll have to update your file paths. Files previously in admin/javascript are now located in admin/javascript/dist and files previously located in javascript are now locatied in javascript/dist.

If you're not doing this already, we suggest looking into a JavaScript bundler like Browserify, to combine JavaScript files. SilverStripe core is moving away from Requirements::combine_files in favour of Browserify as of 4.0 and Requirements::combine_files is being considered for deprecation in future versions.

Upgrading TinyMCE to 4.0

Please see the tinymce upgrading guide to assist with upgrades to customisations to tinymce 3.

In Framework 4.0 the user interface for TinyMCE has been trimmed down considerably, with certain toolbar buttons removed from the default cms configuration. These include:

• Strikethrough
• Styles dropdown
• Block quotes
• Horizontal Rule
• Undo / Redo
• Cut / Paste as word
• Select all
• Fullscreen

However, these function may be enabled on a case by case basis through modifification of the default tinymce config, or by creating custom configurations.

The optional ss_macron plugin for inserting Māori diacritical marks has been removed from core. You can configure the built-in charmap plugin instead:

:::php
$editor = HtmlEditorConfig::get('cms');
$editor->enablePlugins('charmap');
$editor->addButtonsToLine(1, 'charmap');
$editor->setOption('charmap_append', [
    ['256','A - macron'],
    ['274','E - macron'],
    ['298','I - macron'], 
    ['332','O - macron'], 
    ['362','U - macron'], 
    ['257','a - macron'], 
    ['275','e - macron'], 
    ['299','i - macron'], 
    ['333','o - macron'], 
    ['363','u - macron']
]);

For more information on available options and plugins please refer to the tinymce documentation