silverstripe-framework/docs/en/04_Changelogs/3.2.0.md

3.2.0

Contents

• Major Changes
• Removed API
• New API
• Bugfixes
• Upgrading Notes

Major changes

• Minimum PHP version raised to 5.3.3
• Introduction of new parameterised ORM
• Default support for PDO
• Moved SS_Report and ReportAdmin out to a separate module. If you're using composer or downloading a release, this module should be included for you. Otherwise, you'll need to include the module yourself (https://github.com/silverstripe-labs/silverstripe-reports)
• Moved SiteConfig also out to its own module. This will be included by default if you include the CMS module. (https://github.com/silverstripe/silverstripe-siteconfig)
• Implementation of new "Archive" concept for page removal, which supercedes "delete from draft". Where deletion removed pages only from draft, archiving removes from both draft and live simultaneously.
• Most of the Image manipulation methods have been renamed

Deprecated classes/methods removed

• ToggleField was deprecated in 3.1, and has been removed. Use custom Javascript with ReadonlyField instead.
• ExactMatchMultiFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter instead.
• NegationFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter:not instead.
• StartsWithMultiFilter was deprecated in 3.1, and has been removed. Use StartsWithFilter instead.
• ScheduledTask and subclasses like DailyTask were deprecated in 3.1, and have been removed. Use custom code instead, or a module like silverstripe-crontask: https://github.com/silverstripe-labs/silverstripe-crontask
• Cookie::forceExpiry() was removed. Use Cookie::force_expiry() instead
• Object statics removal: get_static(), set_static(), uninherited_static(), combined_static(), addStaticVars() and add_static_var() removed. Use the Config methods instead.
• GD methods removed: setGD(), getGD(), hasGD(). Use setImageResource(), getImageResource(), and hasImageResource() instead
• DataExtension::get_extra_config() removed, no longer supports extraStatics or extraDBFields. Define your statics on the class directly.
• DataList::getRange() removed. Use limit() instead.
• SQLMap removed. Call map() on a DataList or use SS_Map directly instead.
• Profiler removed. Use xhprof or xdebug for profiling instead.
• Aggregate removed. Call aggregate methods on a DataList instead e.g. Member::get()->max('LastEdited')
• MySQLDatabase::set_connection_charset() removed. Use MySQLDatabase.connection_charset config setting instead
• SQLConditionalExpression/SQLQuery select(), limit(), orderby(), groupby(), having(), from(), leftjoin(), innerjoin(), where() and whereAny() removed. Use set*() and add*() methods instead.
• Template <% control $MyList %> syntax removed. Use <% loop $MyList %> instead.
• Removed Member.LastVisited and Member.NumVisits properties, see Howto: Track Member Logins to restore functionality as custom code

New and changed API

• Implementation of a parameterised query framework eliminating the need to manually escape variables for use in SQL queries. This has been integrated into nearly every level of the database ORM.
• Refactor of database connectivity classes into separate components linked together through dependency injection
• Refactor of SQLQuery into separate objects for each query type: SQLSelect, SQLDelete, SQLUpdate and SQLInsert
• PDO is now a standard connector, and is available for all database interfaces
• DataObject::doValidate() method visibility added to access DataObject::validate externally
• NumericField now uses HTML5 "number" type instead of "text"
• UploadField "Select from files" shows files in all folders by default
• UploadField won't display an overwrite warning unless Upload::replaceFile is true
• HtmlEditorField no longer substitutes <blockquote /> for indented text
• ClassInfo::dataClassesFor now returns classes which should have tables, regardless of whether those tables actually exist.
• SS_Filterable, SS_Limitable and SS_Sortable now explicitly extend SS_List
• Convert::html2raw no longer wraps text by default and can decode single quotes.
• Mailer no longer calls xml2raw on all email subject line, and now must be passed in via plain text.
• ErrorControlChain now supports reload on exceptions
• FormField::validate now requires an instance of Validator
• API: Removed URL routing by controller name
• Security: The multiple authenticator login page should now be styled manually - i.e. without the default jQuery UI layout. A new template, Security_MultiAuthenticatorLogin.ss is available.
• Security: This controller's templates can be customised by overriding the getTemplatesFor function.
• Deprecation::set_enabled() or SS_DEPRECATION_ENABLED can now be used to enable or disable deprecation notices. Deprecation notices are no longer displayed on test.
• API: Form and FormField ID attributes rewritten.
• SearchForm::getSearchQuery no longer pre-escapes search keywords and must be cast in your template
• Helper function DB::placeholders can be used to generate a comma separated list of placeholders useful for creating "WHERE ... IN (?,...)" SQL fragments
• Implemented Convert::symbol2sql to safely encode database and table names and identifiers. E.g. Convert::symbol2sql('table.column') => '"table"."column"';
• Convert::raw2sql may now quote the escaped value, as well as safely escape it, according to the current database adaptor's preference.
• DB class has been updated and many static methods have been renamed to conform to coding convention.
  • Renamed API:
    • affectedRows -> affected_rows
    • checkAndRepairTable -> check_and_repair_table
    • createDatabase -> create_database
    • createField -> create_field
    • createTable -> create_table
    • dontRequireField -> dont_require_field
    • dontRequireTable -> dont_require_table
    • fieldList -> field_list
    • getConn -> get_conn
    • getGeneratedID -> get_generated_id
    • isActive -> is_active
    • requireField -> require_field
    • requireIndex -> require_index
    • requireTable -> require_table
    • setConn -> set_conn
    • tableList -> table_list
  • Deprecated API:
    • getConnect (Was placeholder for PDO connection string building code, but is made redundant after the PDOConnector being fully abstracted)
  • New API:
    • build_sql - Hook into new SQL generation code
    • get_connector (Nothing to do with getConnect)
    • get_schema
    • placeholders
    • prepared_query
• SS_Database class has been updated and many functions have been deprecated, or refactored into the various other database classes. Most of the database management classes remain in the database controller, due to individual databases (changing, creating of, etc) varying quite a lot from API to API, but schema updates within a database itself is managed by an attached DBSchemaManager
  • Refactored into DBSchemaManager:
    • createTable
    • alterTable
    • renameTable
    • createField
    • renameField
    • fieldList
    • tableList
    • hasTable
    • enumValuesForField
    • beginSchemaUpdate and endSchemaUpdate -> Use schemaUpdate with a callback
    • cancelSchemaUpdate
    • isSchemaUpdating
    • doesSchemaNeedUpdating
    • transCreateTable
    • transAlterTable
    • transCreateField
    • transCreateField
    • transCreateIndex
    • transAlterField
    • transAlterIndex
    • requireTable
    • dontRequireTable
    • requireIndex
    • hasField
    • requireField
    • dontRequireField
  • Refactored into DBQueryBuilder
    • sqlQueryToString
  • Deprecated:
    • getConnect - Was intended for use with PDO, but was never implemented, and is now redundant, now that there is a stand-alone PDOConnector
    • prepStringForDB - Use quoteString instead
    • dropDatabase - Use dropSelectedDatabase
    • createDatabase - Use selectDatabase with the second parameter set to true instead
    • allDatabaseNames - Use databaseList instead
    • currentDatabase - Use getSelectedDatabase instead
    • addslashes - Use escapeString instead
• LogErrorEmailFormatter now better displays SQL queries in errors by respecting line breaks
• Installer has been majorly upgraded to handle the new database configuration options and additional PDO functionality.
• Created SS_DatabaseException to emit database errors. Query information such as SQL and any relevant parameters may be used by error handling user code that catches this exception.
• The SQLConditionGroup interface has been created to represent dynamically evaluated SQL conditions. This may be used to wrap a class that generates a custom SQL clause(s) to be evaluated at the time of execution.
• DataObject constants CHANGE_NONE, CHANGE_STRICT, and CHANGE_VALUE have been created to provide more verbosity to field modification detection. This replaces the use of various magic numbers with the same meaning.
• create_table_options now uses constants as API specific filters rather than strings. This is in order to promote better referencing of elements across the codebase. See FulltextSearchable->enable for example.
• $FromEnd iterator variable now available in templates.
• Support for multiple HtmlEditorConfigs on the same page.
• Object::singleton() method for better type-friendly singleton generation
• New Image methods CropWidth and CropHeight added
• 'Max' versions of Image methods introduced to prevent up-sampling
• Update Image method names in PHP code and templates
  • SetRatioSize -> Fit
  • CroppedImage -> Fill
  • PaddedImage -> Pad
  • SetSize -> Pad
  • SetWidth -> ScaleWidth
  • SetHeight -> ScaleHeight

Bugfixes

• Reduced database regeneration chances on subsequent rebuilds after the initial dev/build
• Elimination of various SQL injection vulnerability points
• DataObject::writeComponents() now called correctly during DataObject::write()
• Fixed missing theme declaration in installer
• Fixed incorrect use of non-existing exception classes (e.g. HTTPResponse_exception)
• GridState fixed to distinguish between check for missing values, and creation of nested state values, in order to prevent non-empty values being returned for missing keys. This was breaking DataObject::get_by_id by passing in an object for the ID.
• Fixed order of File fulltext searchable fields to use same order as actual fields. This is required to prevent unnecessary rebuild of MS SQL databases when fulltext searching is enabled.
• In the past E_RECOVERABLE_ERROR would be ignored, and now correctly appear as warnings.

Upgrading Notes

UploadField "Select from files" shows files in all folders by default

In order to list files in a single folder by default (previous default behaviour), use setDisplayFolderName() with a folder path relative to assets/:

:::php
UploadField::create('MyField')->setDisplayFolderName('Uploads');

UploadField won't display an overwrite warning unless Upload:replaceFile is true

The configuration setting UploadField:overwriteWarning is dependent on Upload:replaceFile which is set to false by default.

To display a warning before overwriting a file:

Via config:

::yaml
Upload:
  # Replace an existing file rather than renaming the new one.
  replaceFile: true
UploadField:
  # Warning before overwriting existing file (only relevant when Upload: replaceFile is true)
  overwriteWarning: true

Or per instance:

::php
$uploadField->getUpload()->setReplaceFile(true);
$uploadField->setOverwriteWarning(true);

File.allowed_extensions restrictions

Certain file types such as swf, html, htm, xhtml and xml have been removed from the list of allowable file uploads. If your application requires the ability to upload these, you will need to append these to the File.allowed_extensions config as necessary. Also if uploading other file types, it's necessary to ensure that File.allowed_extensions includes that extension, as extensions passed to [api:UploadField] will be filtered against this list.

Removed format detection in i18n::$date_format and i18n::$time_format

Localized dates cause inconsistencies in client-side vs. server-side formatting and validation, particularly in abbreviated month names. The default date format has been changed to "yyyy-MM-dd" (e.g. 2014-12-31). New users will continue to have the option for a localized date format in their profile (based on their chosen locale). If you have existing users with Member.DateFormat set to a format including "MMM" or "MMMM", consider deleting those formats to fall back to the global (and more stable) default.

Cookies set via Cookie::set() are now HTTP only by default

Cookies set through Cookie::set() now default to "HTTP only". This means that scripting languages like JavaScript won't be able to read them.

To set it back to be non-HTTP only, you need to set the $httpOnly argument to false when calling Cookie::set().

API: Removed URL routing by controller name

The auto-routing of controller class names to URL endpoints has been removed (rule: '$Controller//$Action/$ID/$OtherID': '*'). This increases clarity in routing since it makes URL entpoints explicit, and thereby simplifies system and security reviews.

Please access any custom controllers exclusively through self-defined routes. For controllers extending Page_Controller, simply use the provided page URLs.

:::php
class MyController extends Controller {
	static $allowed_actions = array('myaction');
	public function myaction($request) {
		// ...
	}
}

Create a new file mysite/_config/routes.yml (read more about the config format). Your controller is now available on http://yourdomain.com/my-controller-endpoint, after refreshing the configuration cache through ?flush=all.

:::yaml
---
Name: my-routes
After: framework/routes#coreroutes
---
Director:
  rules:
    'my-controller-endpoint//$Action' : 'MyController'

The auto-routing is still in place for unit tests, since its a frequently used feature there. Although we advise against it, you can reinstate the old behaviour through a director rule:

:::yaml
---
Name: my-routes
After: framework/routes#coreroutes
---
Director:
  rules:
    '$Controller//$Action/$ID/$OtherID':  '*'

API: Default Form and FormField ID attributes rewritten.

Previously the automatic generation of ID attributes throughout the Form API could generate invalid ID values such as Password[ConfirmedPassword] as well as duplicate ID values between forms on the same page. For example, if you created a field called Email on more than one form on the page, the resulting HTML would have multiple instances of #Email. ID should be a unique identifier for a single element within the document.

This rewrite has several angles, each of which is described below. If you rely on ID values in your CSS files, Javascript code or application unit tests you will need to update your code.

Conversion of invalid form ID values

ID attributes on Form and Form Fields will now follow the HTML specification. Generating ID attributes is now handled by the new FormTemplateHelper class.

Please test each of your existing site forms to ensure that they work correctly in particular, javascript and css styles which rely on specific ID values.

Invalid ID attributes stripped

ID attributes will now be run through Convert::raw2htmlid. Invalid characters are replaced with a single underscore character. Duplicate, leading and trailing underscores are removed. Custom ID attributes (set through setHTMLID) will not be altered.

Before:

:::html
<form id="MyForm[Form]"
	<div id="MyForm[Form][ID]">

Now:

:::html
<form id="MyForm_Form">
	<div id="MyForm_Form_ID">

Namespaced FormField ID's

Form Field ID values will now be namespaced with the parent form ID.

Before:

:::html
<div id="Email">

Now:

:::html
<div id="MyForm_Email">

FormField wrapper containers suffixed with _Holder

Previously both the container div and FormField tag shared the same ID in certain cases. Now, the wrapper div in the default FormField template will be suffixed with _Holder.

Before:

:::html
<div id="Email">
	<input id="Email" />

After:

:::html
<div id="MyForm_Email_Holder"
	<input id="MyForm_Email" />

Reverting to the old specification

If upgrading existing forms is not feasible, developers can opt out of the new specifications by using the FormTemplateHelper_Pre32 class rules instead of the default ones.

mysite/config/_config.yml:

:::yaml
Injector:
  FormTemplateHelper:
    class: FormTemplateHelper_Pre32

Update code that uses SQLQuery

SQLQuery has been changed. Previously this class was used for both selecting and deleting, but deletion is now handled by the new SQLDelete class.

Additionally, 3.2 now provides SQLUpdate and SQLInsert to generate parameterised query friendly data updates.

SQLQuery, SQLDelete and SQLUpdate all inherit from SQLConditionalExpression, which implements toSelect, toDelete, and toUpdate to generate basic transformations between query types.

In the past SQLQuery->setDelete(true) would be used to turn a select into a delete, although now a new SQLDelete object should be created from the original SQLQuery.

Before:

:::php
<?php
$query = new SQLQuery('*');
$query->setFrom('"SiteTree"');
$query->setWhere('"SiteTree"."ShowInMenus" = 0');
$query->setDelete(true);
$query->execute();

After:

:::php
<?php
$query = SQLDelete::create()
	->setFrom('"SiteTree"')
	->setWhere(array('"SiteTree"."ShowInMenus"' => 0));
$query->execute();

When working with SQLQuery passed into user code, it is advisable to strictly cast it into either a SQLSelect or SQLDelete. This can be done by using the new SQLQuery::toAppropriateExpression() method, which will automatically convert to the correct type based on whether the SQLQuery is set to delete or not.

If a SQLQuery is not converted, then the result of getWhere will not be parameterised. This is because user code written for 3.1 expects this list to be a flat array of strings. This format is inherently unsafe, and should be avoided where possible.

:::php
<?php
public function augmentSQL(SQLQuery &$query) {
	$query->getWhere(); // Will be flattened (unsafe 3.1 compatible format)
	$expression = $query->toAppropriateExpression(); // Either SQLSelect or SQLDelete
	$expression->getWhere(); // Will be parameterised (preferred 3.2 compatible format)
}

Alternatively:

:::php
<?php
$query = SQLQuery::create()
	->setFrom('"SiteTree"')
	->setWhere(array('"SiteTree"."ShowInMenus"' => 0))
	->setDelete(true)
	->toAppropriateExpression();
$query->execute();

Update code that interacts with SQL strings to use parameters

The Silverstripe ORM (object relation model) has moved from using escaped SQL strings to query the database, to a combination of parameterised SQL expressions alongside a related list of parameter values. As a result of this, it is necessary to assume that any SQLQuery object may, and will usually, have un-injected parameters.

All database queries performed through DataList, DataQuery and SQLQuery will continue to work, as will those through DataObject::get() (which returns a filterable DataList). However, any conditional expression that includes values escaped with Convert::raw2sql() should use the new standard syntax. This new querying standard method enforces a much higher level of security than was previously available, and all code using manual escaping should be upgraded.

See the security topic for details on why this is necessary, or the databamodel topic for more information.

As a result of this upgrade there are now very few cases where Convert::raw2sql needs to be used.

Examples of areas where queries should be upgraded are below:

1. Querying the database directly through DB, including non-SELECT queries

Before:

:::php
<?php

// Note: No deprecation notices will be caused here
DB::query("UPDATE \"SiteTree\" SET \"Title\" LIKE '%" . Convert::raw2sql($myTitle) . "%' WHERE \"ID\" = 1");
$myPages = DB::query(sprintf('SELECT "ID" FROM "MyObject" WHERE "Title" = \'%s\'', Convert::raw2sql($parentTitle)));

After:

:::php
<?php

DB::prepared_query(
	'UPDATE "SiteTree" SET "Title" LIKE ? WHERE "ID" = ?',
	array("%{$myTitle}%", 1)
);
$myPages = DB::prepared_query(
	'SELECT "ID" FROM "MyObject" WHERE "Title" = ?',
	array($parentTitle)
);

2. Querying the database through DataList, DataQuery, SQLQuery, and DataObject

Before:

:::php
<?php

$items = DataObject::get_one('MyObject', '"Details" = \''.Convert::raw2sql($details).'\'');
$things = MyObject::get()->where('"Name" = \''.Convert::raw2sql($name).'\'');
$list = DataList::create('Banner')->where(array(
	'"ParentID" IS NOT NULL',
	'"Title" = \'' . Convert::raw2sql($title) . '\''
);

After:

:::php
<?php

$items = DataObject::get_one('MyObject', array('"MyObject"."Details"' => $details));
$things = MyObject::get()->where(array('"MyObject"."Name" = ?' => $name));
$list = DataList::create('Banner')->where(array(
	'"ParentID" IS NOT NULL',
	'"Title" = ?', $title
);

3. Interaction with DataList::sql(), DataQuery::sql(), SQLQuery::sql(), or SQLQuery::getJoins() methods

The place where legacy code would almost certainly fail is any code that calls DataList::sql, DataQuery::sql, SQLQuery::sqlorSQLQuery::getJoins()`, as the api requires that user code passes in an argument here to retrieve SQL parameters by value.

User code that assumes parameterless queries will likely fail, and need to be updated to handle this case properly.

Before:

:::php
<?php

// Generate query
$argument = 'whatever';
$query = SQLQuery::create()
	->setFrom('"SiteTree"')
	->setWhere(array("\"SiteTree\".\"Title\" LIKE '" . Convert::raw2sql($argument) . "'"));

// Inspect elements of the query
$sql = $query->sql();
$sql = preg_replace('/LIKE \'(.+)\'/', 'LIKE \'%${1}%\'', $sql); // Adds %% around the argument

// Pass new query to database connector
DB::query($sql);

After:

:::php
<?php

// Generate query
$argument = 'whatever';
$query = SQLQuery::create()
	->setFrom('"SiteTree"')
	->setWhere(array('"SiteTree"."Title" LIKE ?' => $argument));

// Inspect elements of the query
$sql = $query->sql($parameters);
foreach($parameters as $key => $value) {
	// Adds %% around arguments
	$parameters[$key] = "%{$value}%";
}

// Pass new query to database connector
// Note that DB::query($sql) would fail, as it would contain ? with missing parameters
DB::prepared_query($sql, $parameters);

Also note that the parameters may not be a single level array, as certain values may be forced to be cast as a certain type (where supported by the current API).

E.g.

:::php
<?php

$parameters = array(
	10,
	array('value' => 0, 'type' => 'boolean') // May also contain other database API specific options
)
DB::prepared_query('DELETE FROM "MyObject" WHERE ParentID = ? OR IsValid = ?', $parameters);

4. Interaction with SQLSelect::getWhere() method

The SQLSelect class supercedes the old SQLQuery object for performing select queries. Although both implement the getWhere() method, the results returned by SQLSelect::getWhere() will be parameterised while SQLQuery::getWhere() will be a flattened array of strings.

SQLSelect::getWhere() returns a list of conditions, each of which is an associative array mapping the condition string to a list of parameters provided.

Before:

:::php
<?php

// Increment value of a single condition
$query = new SQLQuery(/*...*/);
$conditions = $query->getWhere();
$new = array();
foreach($conditions as $condition) {
	if(preg_match('/\"Count\" = (?<count>\d+)/', $condition, $matches)) {
		$condition = '"Count" = '.($matches['count'] + 1);
	}
	$new[] = $condition;
}
$query->setWhere($new);

After:

:::php
// Increment value of a single condition
$query = new SQLSelect(/*...*/);
$conditions = $query->getWhere();
$new = array();
foreach($conditions as $condition) {
	// $condition will be a single length array
	foreach($condition as $predicate => $parameters) {
		if('"Count" = ?' ===  $predicate) {
			$parameters[0] = $parameters[0] + 1;
		}
		$new[] = array($predicate => $parameters);
	}
}
$query->setWhere($new);

In cases where user code will manipulate the results of this value, it may be useful to replace this method call with the new getWhereParameterised($parameters) method, where applicable.

This method returns a manipulated form of the where conditions stored by the query, so that it matches the list of strings similar to the old 3.1 SQLQuery::getWhere() behaviour. Additionally, the list of parameters is safely extracted, flattened, and can be passed out through the $parameters argument which is passed by reference.

Before:

:::php
public function filtersOnColumn($query, $column) {
	$regexp = '/^(.*\.)?("|`)?' . $column . ' ("|`)?\s?=/';
	foreach($this->getWhere() as $predicate) {
		if(preg_match($regexp, $predicate)) return true;
	}
	return false;
}

After:

:::php
public function filtersOnColumn($query, $column) {
	$regexp = '/^(.*\.)?("|`)?' . $column . ' ("|`)?\s?=/';
	foreach($this->getWhereParameterised($parameters) as $predicate) {
		if(preg_match($regexp, $predicate)) return true;
	}
	return false;
}

5. Update code that interacts with the DB schema

Updating database schema is now done by updateSchema with a callback, rather than relying on user code to call beginSchemaUpdate and endSchemaUpdate around the call.

Since the schema management object is separate from the database controller you interact with it via DB::get_schema instead of DB::get_conn (previously named DB::getConn)

Before:

:::php
<?php
$conn = DB::getConn();
$conn->beginSchemaUpdate();
foreach($dataClasses as $dataClass) {
	singleton($dataClass)->requireTable();
}
$conn->endSchemaUpdate();

After:

:::php
<?php
$schema = DB::get_schema();
$schema->schemaUpdate(function() use($dataClasses){
	foreach($dataClasses as $dataClass) {
		singleton($dataClass)->requireTable();
	}
});

Also should be noted is that many functions have been renamed to conform better with coding conventions. E.g. DB::requireTable is now DB::require_table

Revert to legacy CMS page actions

By default "delete from live" and "delete" actions are deprecated in favour of "unpublish" and "archive". "unpublish" is an existing action which is functionally equivalent to "delete from live", and "archive" is a new feature which performs both unpublish and deletion simultaneously.

To restore "delete from live" add the following config to your site's config.yml.

:::yml
CMSMain:
  enabled_legacy_actions:
    - CMSBatchAction_DeleteFromLive

In order to remove the new "archive" action and restore the old "delete" action you can use the following config

:::yml
CMSMain:
  enabled_legacy_actions:
    - CMSBatchAction_Delete