silverstripe-framework/docs/en/04_Changelogs/4.0.0.md

# 4.0.0 (unreleased)

## Overview

### Framework

 * Deprecate `SQLQuery` in favour `SQLSelect`
 * `DataList::filter` by null now internally generates "IS NULL" or "IS NOT NULL" conditions appropriately on queries
 * `DataObject::database_fields` now returns all fields on that table.
 * `DataObject::db` now returns composite fields.
 * `DataObject::ClassName` field has been refactored into a `DBClassName` type field.
 * Image manipulations have been moved into a new `[api:ImageManipulation]` trait.
 * `CMSFileAddController` removed.
 * UploadField::setAllowedFileCategories('image') now excludes non-resizeable images. 'unresizeable_image' is
   can be used to validate these types.
 * `Image_Backend` API now loads and saves from `AssetContainer` instances rather than local files.
 * The following File categories have been renamed: 'zip' to 'archive', 'doc' to 'document', and 'mov' to 'video'
 * `File::updateLinks` no longer takes urls as parameters. All file links are now identified either by
   the `DataObject::ID` in a `data-fileid` property, or via shortcodes. This is necessary because file
   urls are no longer able to identify assets.
 * Extension point `HtmlEditorField::processImage` has been removed, and moved to `Image::regenerateImageHTML`

## New API

 * New filesystem abstraction including new `DBFile` database field to hold file references.
 * `ShortcodeHandler` interface to help generate standard handlers for HTML shortcodes in the editor.
 * `AssetNameGenerator` interface, including a `DefaultAssetNameGenerator` implementation, which is used to generate
   renaming suggestions based on an original given filename in order to resolve file duplication issues.

## Deprecated classes/methods

The following image manipulations previously deprecated has been removed:

 * `Image::SetRatioSize` superceded by `Fit`
 * `Image::SetWidth` superceded by `ScaleWidth`
 * `Image::SetHeight` superceded by `ScaleHeight`
 * `Image::SetSize` superceded by `Pad`
 * `Image::PaddedImage` superceded by `Pad`
 * `Image::CroppedImage` superceded by `Fill`
 * `Image::AssetLibraryPreview` superceded by `PreviewThumbnail`
 * `Image::AssetLibraryThumbnail` superceded by `CMSThumbnail`

The following `File` methods have been removed. Since there is no longer any assumed local path for any file,
methods which dealt with such paths may no longer be relied on.

 * `File::deletedatabaseOnly`
 * `File::link_shortcode_handler` renamed to `handle_shortcode`
 * `File::setParentID`
 * `File::getFullPath`
 * `File::getRelativePath`
 * `File::Content` database field is removed

Image manipulations have been moved out of Image.php and now available to any File or DBFile which has the
appropriate mime types. The following file manipulations classes and methods have been removed:

 * `CleanImageManipulationCache` class
 * `Image_Cached` class
 * `Image::regenerateFormattedImages`
 * `Image::getGeneratedImages`
 * `Image::deleteFormattedImages`
 * `AssetAdmin::deleteunusedthumbnails`
 * `AssetAdmin::getUnusedThumbnails`

Many `Folder` methods have also been removed:

 * `Folder::syncChildren`
 * `Folder::constructChild`
 * `Folder::addUploadToFolder`

The following filesystem synchronisation methods are also removed

 * `Filesystem::sync`
 * `AssetAdmin::doSync`

## Upgrading

### New asset storage mechanism

File system has been abstracted into an abstract interface. By default, the out of the box filesystem
uses [Flysystem](http://flysystem.thephpleague.com/) with a local storage mechanism (under the assets directory).

Because the filesystem now uses the sha1 of file contents in order to version multiple versions under the same
filename, the default storage paths in 4.0 will not be the same as in 3.

In order to retain existing file paths in line with framework version 3 you should set the
`\SilverStripe\Filesystem\Flysystem\FlysystemAssetStore.legacy_paths` config to true.
Note that this will not allow you to utilise certain file versioning features in 4.0.


	:::yaml
	\SilverStripe\Filesystem\Flysystem\FlysystemAssetStore:
	  legacy_paths: true


See [/developer_guides/files/file_management] for more information on how the new system works.

### Migrating File DataObject from 3.x to 4.0

Since the structure of `File` dataobjects has changed, a new task `MigrateFileTask` has been added to assist
in migration of legacy files. Migration can be invoked by either this task, or can be configured to automatically
run during dev build by setting the `File.migrate_legacy_file` config to true. However, it's recommended that
this task is run manually during an explicit migration process, as this process could potentially consume
large amounts of memory and run for an extended time.


	:::yaml
	File:
	  migrate_legacy_file: true


### Upgrade code which acts on `Image`

As all image-specific manipulations has been refactored from `Image` into an `ImageManipulations` trait, which 
is applied to both `File` and `DBFile`. These both implement a common interface `AssetContainer`, which
has the `getIsImage()` method. In some cases, it may be preferable to invoke this method to detect
if the asset is an image or not, rather than checking the subclass, as the asset may also be a `DBFile` with
an image filter applied, rather than an instance of the `Image` dataobject.

In addition, a new file category `image/supported` has been added, which is a subset of the `image` category.
This is the subset of all image types which may be assigned to the `[api:Image]` dataobject, and may have
manipulations applied to it. This should be used as the file type restriction on any `[api:UploadField]` which
is intended to upload images for manipulation.

Before:


	:::php
	if($file instanceof Image) {
		$upload = new UploadField();
		$upload->setAllowedFileCategories('image');
	}


After:


	:::php
	if($file->getIsImage()) {
		$upload = new UploadField();
		$upload->setAllowedFileCategories('image/supported');
	}


In cases where image-only assets may be assigned to relationships then your datamodel should specify explicitly
an `Image` datatype, or refer to `DBFile('image/supported')`.

E.g.


	:::php
	class MyObject extends DataObject {
		private static $has_one = array(
			"ImageObject" => "Image"
		);
		private static $db = array(
			"ImageField" => "DBFile('image/supported')"
		);
	}


### Upgrading code that writes to `File` dataobjects, or writes files to the 'assets' folder

In the past all that was necessary to write a `File` dataobject to the database was to ensure a physical file
existed in the assets folder, and that the Filename of the dataobject was set to the same location.

Since the storage of physical files is no longer a standard location, it's necessary to delegate the writing of such
files to the asset persistence layer. As a wrapper for an individual file, you can use any of the `setFrom*`
methods to assign content from a local (e.g. temporary) file, a stream, or a string of content.

You would need to upgrade your code as below.

Before:


	:::php
	function importTempFile($tmp) {
		copy($tmp, ASSETS_PATH . '/imported/' . basename($tmp));
		$file = new File();
		$file->setFilename('assets/imported/'.basename($tmp));
		$file->write();
	}


After:


	:::php
	function importTempFile($tmp) {
		$file = new File();
		$file->setFromLocalFile($tmp, 'imported/'.basename($tmp));
		$file->write();
	}


Note that 'assets' is no longer present in the new code, and the path beneath what was once assets is now
used to generate the 'filename' value. This is because there is no longer an assumption that files are
stored in the assets folder.

There are other important considerations in working with File dataobjects which differ from legacy:

* Deleting File dataobjects no longer removes the physical file directly. This is because any file could be referenced
  from DBFile fields, and deleting these could be a potentially unsafe operation.
* File synchronisation is no longer automatic. This is due to the fact that there is no longer a 1-to-1 relationship
  between physical files and File dataobjects.
* Moving files now performs a file copy rather than moving the underlying file, although only a single DataObject
  will exist, and will reference the destination path.
* Folder dataobjects are now purely logical dataobjects, and perform no actual filesystem folder creation on write.

### Upgrading code performs custom image manipulations

As file storage and handling has been refactored into the abstract interface, many other components which were
once specific to Image.php have now been moved into a shared `ImageManipulation` trait. Manipulations of file content,
which are used to generate what are now called "variants" of assets, is now a generic api available to both `File`
and `DBFile` classes through this trait.

Custom manipulations, applied via extensions, must be modified to use the new API.

For instance, code which sizes images to a fixed width should be updated as below:

Before:


	:::php
	// in MyImageExtension.php
	class MyImageExtension extends DataExtension {
		public function GalleryThumbnail($height) {
			return $this->getFormattedImage('GalleryThumbnail', $height);
		}
		public function generateGalleryThumbnail(Image_Backend $backend, $height) {
			return $backend->paddedResize(300, $height);
		}
	}

	// in _config.php
	Image::add_extension('MyImageExtension');


Now image manipulations are implemented with a single method via a callback generator.

After:


	:::php
	// in MyImageExtension.php
	class MyImageExtension extends Extension {
		public function GalleryThumbnail($height) {
			// Generates the manipulation key
			$variant = $this->owner->variantName(__FUNCTION__, $height);

			// Instruct the backend to search for an existing variant with this key,
			// and include a callback used to generate this image if it doesn't exist
			return $this->owner->manipulateImage($variant, function(Image_Backend $backend) use ($height) {
				return $backend->paddedResize(300, $height);
			});
		}
	}

	// in _config.php
	File::add_extension('MyImageExtension');
	DBFile::add_extension('MyImageExtension');


There are a few differences in this new API:

* The extension is no longer specific to dataobjects, so it uses the generic 'Extension' class instead of 'DataExtension'
* This extension is added to both `DBFile` and `File`, or order to make this manipulation available to non-dataobject
  file references as well, but it could be applied to either independently.
* A helper method `variantName` is invoked in order to help generate a unique variant key. Custom code may use another
  generation mechanism.
* Non-image files may also have manipulations, however the specific `manipulateImage` should not be used in this case.
  A generic `manipulate` method may be used, although the callback for this method both is given, and should return,
  an `AssetStore` instance and file tuple (Filename, Hash, and Variant) rather than an Image_Backend.

### Upgrading code that uses composite db fields.

`CompositeDBField` is now an abstract class, not an interface. In many cases, custom code that handled
saving of content into composite fields can be removed, as it is now handled by the base class.

The below describes the minimum amount of effort required to implement a composite DB field.

	:::php
	<?
	class MyAddressField extends CompositeDBField {

		private static $composite_db = array(
			'Street' => 'Varchar(200)',
			'Suburb' => 'Varchar(100)',
			'City' => 'Varchar(100)',
			'Country' => 'Varchar(100)'
		);

		public function scaffoldFormField($title = null) {
			new AddressFormField($this->getName(), $title);
		}
	}


### Upgrading code that references `DataObject::database_fields` or `DataObject::db`

These methods have been updated to include base fields (such as ID, ClassName, Created, and LastEdited), as
well as composite DB fields.

`DataObject::database_fields` does not have a second parameter anymore, and can be called directly on an object
or class. E.g. `Member::database_fields()`

If user code requires the list of fields excluding base fields, then use custom_database_fields instead, or
make sure to call `unset($fields['ID']);` if this field should be excluded.

`DataObject:db()` will return all logical fields, including foreign key ids and composite DB Fields, alongside
any child fields of these composites. This method can now take a second parameter $includesTable, which 
when set to true (with a field name as the first parameter), will also include the table prefix in
`Table.ClassName(args)` format.


### Update code that uses SQLQuery

SQLQuery is still implemented, but now extends the new SQLSelect class and has some methods
deprecated. Previously this class was used for both selecting and deleting, but these
have been superceded by the specialised SQLSelect and SQLDelete classes.

Take care for any code or functions which expect an object of type `SQLQuery`, as
these references should be replaced with `SQLSelect`. Legacy code which generates
`SQLQuery` can still communicate with new code that expects `SQLSelect` as it is a
subclass of `SQLSelect`, but the inverse is not true.

### Update implementations of augmentSQL

Since this method now takes a `SQLSelect` as a first parameter, existing code referencing the deprecated `SQLQuery`
type will raise a PHP error.

E.g.

Before:

	:::php
	function augmentSQL(SQLQuery &$query, DataQuery &$dataQuery = null) {
		$locale = Translatable::get_current_locale();
		if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhere())))  {
			$qry = sprintf('"Locale" = \'%s\'', Convert::raw2sql($locale));
			$query->addWhere($qry); 
		}
	}

After:

	:::php
	function augmentSQL(SQLSelect $query, DataQuery $dataQuery = null) {
		$locale = Translatable::get_current_locale();
		if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhereParameterised($parameters))))  {
			$query->addWhere(array(
				'"Locale"' => $locale
			));
		}
	}