silverstripe-framework/docs/en/changelogs/3.2.0.md

3.2.0 (unreleased)

Overview

Framework

• Minimum PHP version raised to 5.3.3
• DataObject::validate() method visibility changed to public
• NumericField now uses HTML5 "number" type instead of "text"
• UploadField "Select from files" shows files in all folders by default
• UploadField won't display an overwrite warning unless Upload:replaceFile is true
• HtmlEditorField no longer substitutes <blockquote /> for indented text
• ClassInfo::dataClassesFor now returns classes which should have tables, regardless of whether those tables actually exist.
• SS_Filterable, SS_Limitable and SS_Sortable now explicitly extend SS_List
• ToggleField was deprecated in 3.1, and has been removed. Use custom Javascript with ReadonlyField instead.
• ExactMatchMultiFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter instead.
• NegationFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter:not instead.
• StartsWithMultiFilter was deprecated in 3.1, and has been removed. Use StartsWithFilter instead.
• ScheduledTask and subclasses like DailyTask were deprecated in 3.1, and have been removed. Use custom code instead, or a module like silverstripe-crontask: https://github.com/silverstripe-labs/silverstripe-crontask

CMS

• SearchForm::getSearchQuery no longer pre-escapes search keywords and must be cast in your template

Changelog

CMS

DataObject::validate() method visibility changed to public

The visibility of DataObject::validate() has been changed from protected to public.

Any existing classes that currently set this as protected should be changed like in this example:

::php
class MyDataClass extends DataObject {
	...
	public function validate() {
		...
	}
	...
}

UploadField "Select from files" shows files in all folders by default

In order to list files in a single folder by default (previous default behaviour), use setDisplayFolderName() with a folder path relative to assets/:

UploadField::create('MyField')->setDisplayFolderName('Uploads');

UploadField won't display an overwrite warning unless Upload:replaceFile is true

The configuration setting UploadField:overwriteWarning is dependent on Upload:replaceFile which is set to false by default.

To display a warning before overwriting a file:

Via config:

::yaml
Upload:
  # Replace an existing file rather than renaming the new one.
  replaceFile: true
UploadField:
  # Warning before overwriting existing file (only relevant when Upload: replaceFile is true)
  overwriteWarning: true

Or per instance:

::php
$uploadField->getUpload()->setReplaceFile(true);
$uploadField->setOverwriteWarning(true);

File.allowed_extensions restrictions

Certain file types such as swf, html, htm, xhtml and xml have been removed from the list of allowable file uploads. If your application requires the ability to upload these, you will need to append these to the File.allowed_extensions config as necessary. Also if uploading other file types, it's necessary to ensure that File.allowed_extensions includes that extension, as extensions passed to [api:UploadField] will be filtered against this list.

Removed format detection in i18n::$date_format and i18n::$time_format

Localized dates cause inconsistencies in client-side vs. server-side formatting and validation, particularly in abbreviated month names. The default date format has been changed to "yyyy-MM-dd" (e.g. 2014-12-31). New users will continue to have the option for a localized date format in their profile (based on their chosen locale). If you have existing users with Member.DateFormat set to a format including "MMM" or "MMMM", consider deleting those formats to fall back to the global (and more stable) default.

Bugfixes

• Migration of code to use new parameterised framework

Framework

• Implementation of a parameterised query framework eliminating the need to manually escape variables for use in SQL queries. This has been integrated into nearly every level of the database ORM.
• Refactor of database connectivity classes into separate components linked together through dependency injection
• Refactor of SQLQuery into separate objects for each query type: SQLSelect, SQLDelete, SQLUpdate and SQLInsert
• Rename of API methods to conform to coding conventions
• PDO is now a standard connector, and is available for all database interfaces
• Additional database and query generation tools

Bugfixes

• Reduced database regeneration chances on subsequent rebuilds after the initial dev/build
• Elimination of various SQL injection vulnerability points
• DataObject::writeComponents() now called correctly during DataObject::write()
• Fixed missing theme declaration in installer
• Fixed incorrect use of non-existing exception classes (e.g. HTTPResponse_exception)
• GridState fixed to distinguish between check for missing values, and creation of nested state values, in order to prevent non-empty values being returned for missing keys. This was breaking DataObject::get_by_id by passing in an object for the ID.
• Fixed order of File fulltext searchable fields to use same order as actual fields. This is required to prevent unnecessary rebuild of MS SQL databases when fulltext searching is enabled.

Upgrading

Update code that uses SQLQuery

SQLQuery is still implemented, but now extends the new SQLSelect class and has some methods deprecated. Previously this class was used for both selecting and deleting, but these have been superceded by the specialised SQLSelect and SQLDelete classes. Additionally, 3.2 now provides SQLUpdate and SQLInsert to generate parameterised query friendly data updates.

SQLSelect, SQLDelete and SQLUpdate all inherit from SQLConditionalExpression, which implements toSelect, toDelete, and toUpdate to generate basic transformations between query types.

In the past SQLQuery->setDelete(true) would be used to turn a select into a delete, although now a new SQLDelete object should be created from a separate SQLSelect.

Before:

:::php
<?php
$query = new SQLQuery('*');
$query->setFrom('"SiteTree"');
$query->setWhere('"SiteTree"."ShowInMenus" = 0');
$query->setDelete(true);
$query->execute();

After:

:::php
<?php
$query = SQLDelete::create()
	->setFrom('"SiteTree"')
	->setWhere(array('"SiteTree"."ShowInMenus"' => 0));
$query->execute();

Alternatively:

:::php
<?php
$query = SQLSelect::create()
	->setFrom('"SiteTree"')
	->setWhere(array('"SiteTree"."ShowInMenus"' => 0))
	->toDelete();
$query->execute();

Also, take care for any code or functions which expect an object of type SQLQuery, as these references should be replaced with SQLSelect. Legacy code which generates SQLQuery can still communicate with new code that expects SQLSelect as it is a subclass of SQLSelect, but the inverse is not true.

Update code that interacts with SQL strings to use parameters

The Silverstripe ORM (object relation model) has moved from using escaped SQL strings to query the database, to a combination of parameterised SQL expressions alongside a related list of parameter values. As a result of this, it is necessary to assume that any SQLSelect object (previously SQLQuery) may, and will usually, have un-injected parameters.

All database queries performed through DataList, DataQuery and SQLQuery will continue to work, as will those through DataObject::get() (which returns a filterable DataList). However, any conditional expression that includes values escaped with Convert::raw2sql() should use the new standard syntax. This new querying standard method enforces a much higher level of security than was previously available, and all code using manual escaping should be upgraded.

See the security topic for details on why this is necessary, or the databamodel topic for more information.

As a result of this upgrade there are now very few cases where Convert::raw2sql needs to be used.

Examples of areas where queries should be upgraded are below:

1. Querying the database directly through DB, including non-SELECT queries

   Before:

    :::php
    <?php
   
    // Note: No deprecation notices will be caused here
    DB::query("UPDATE \"SiteTree\" SET \"Title\" LIKE '%" . Convert::raw2sql($myTitle) . "%' WHERE \"ID\" = 1");
    $myPages = DB::query(sprintf('SELECT "ID" FROM "MyObject" WHERE "Title" = \'%s\'', Convert::raw2sql($parentTitle)));
   

   After:

    :::php
    <?php
   
    DB::prepared_query(
    	'UPDATE "SiteTree" SET "Title" LIKE ? WHERE "ID" = ?',
    	array("%{$myTitle}%", 1)
    );
    $myPages = DB::prepared_query(
    	'SELECT "ID" FROM "MyObject" WHERE "Title" = ?',
    	array($parentTitle)
    );
   

2. Querying the database through SQLQuery (deprecated)

   Before:

   Note: Use of SQLQuery would generate a deprecation notice if left un-upgraded.

    :::php
    <?php
   
    $query = new SQLQuery('*', '"SiteTree"', "\"URLSegment\" = '".Convert::raw2sql($testURL)."'");
   
    $query->addWhere(array(
    	'"ParentID" = \''.intval($parentID).'\'',
    	'"ID" IN (SELECT "PageID" FROM "MyObject")'
    ));
   
    $query->addWhere("\"Title\" LIKE '%".Convert::raw2sql($myText)."' OR \"Title\" LIKE '".Convert::raw2sql($myText)."%'");
   

   After, substituting SQLSelect for the deprecated SQLQuery:

   Note: The inclusion of properly ANSI quoted symbols with the table name included, as per best coding practices.

    :::php
    <?php
   
    $query = SQLSelect::create('*', '"SiteTree"', array('"SiteTree"."URLSegment" = ?' => $testURL));
   
    $query->addWhere(array(
    	'"SiteTree"."ParentID"' =>  // Note that the " = ?" is optional for simple comparison
    		array( // Syntax for parameter casting for supporting databases
    			'value' => $parentID,
    			'type' => 'integer'
    		),
    	'"SiteTree"."ID" IN (SELECT "MyObject"."PageID" FROM "MyObject")' // Raw SQL condition with no parameters
    ));
   
    // Multiple parameters may be assigned for a single query (this should not be associative)
    $query->addWhere(array(
    	'"SiteTree"."Title" LIKE %? OR "SiteTree"."Title" LIKE %?' => array($myText, $myText)
    ));
   

3. Querying the database through DataList, DataQuery, and DataObject

   Before:

    :::php
    <?php
   
    $items = DataObject::get_one('MyObject', '"Details" = \''.Convert::raw2sql($details).'\'');
    $things = MyObject::get()->where('"Name" = \''.Convert::raw2sql($name).'\'');
    $list = DataList::create('Banner')->where(array(
    	'"ParentID" IS NOT NULL',
    	'"Title" = \'' . Convert::raw2sql($title) . '\''
    );
   

   After:

    :::php
    <?php
   
    $items = DataObject::get_one('MyObject', array('"MyObject"."Details"' => $details));
    $things = MyObject::get()->where(array('"MyObject"."Name" = ?' => $name));
    $list = DataList::create('Banner')->where(array(
    	'"ParentID" IS NOT NULL',
    	'"Title" = ?', $title
    );
   

4. Interaction with the DataList::sql(), DataQuery::sql() or SQLSelect::sql() methods

   The place where legacy code would almost certainly fail is any code that calls SQLQuery::sql, DataList::sql or DataQuery::sql, as the api requires that user code passes in an argument here to retrieve SQL parameters by value.

   User code that assumes parameterless queries will likely fail, and need to be updated to handle this case properly.

   Before:

    :::php
    <?php
   
    // Generate query
    $argument = 'whatever';
    $query = SQLSelect::create()
    	->setFrom('"SiteTree"')
    	->setWhere(array("\"SiteTree\".\"Title\" LIKE '" . Convert::raw2sql($argument) . "'"));
   
    // Inspect elements of the query
    $sql = $query->sql();
    $sql = preg_replace('/LIKE \'(.+)\'/', 'LIKE \'%${1}%\'', $sql); // Adds %% around the argument
   
    // Pass new query to database connector
    DB::query($sql);
   

   After:

    :::php
    <?php
   
    // Generate query
    $argument = 'whatever';
    $query = SQLSelect::create()
    	->setFrom('"SiteTree"')
    	->setWhere(array('"SiteTree"."Title" LIKE ?' => $argument));
   
    // Inspect elements of the query
    $sql = $query->sql($parameters);
    foreach($parameters as $key => $value) {
    	// Adds %% around arguments
    	$parameters[$key] = "%{$value}%";
    }
   
    // Pass new query to database connector
    // Note that DB::query($sql) would fail, as it would contain ? with missing parameters
    DB::prepared_query($sql, $parameters);
   

   Also note that the parameters may not be a single level array, as certain values may be forced to be cast as a certain type (where supported by the current API).

   E.g.

    :::php
    <?php
   
    $parameters = array(
    	10,
    	array('value' => 0, 'type' => 'boolean') // May also contain other database API specific options
    )
    DB::prepared_query('DELETE FROM "MyObject" WHERE ParentID = ? OR IsValid = ?', $parameters);
   

5. Update implementations of augmentSQL

Since this method now takes a SQLSelect as a first parameter, existing code referencing the deprecated SQLQuery type will raise a PHP error.

Furthermore, it's important to note that even though the signature of SQLSelect::getWhere is similar to the old SQLQuery::getWhere the result will actually be an associative array of SQL fragments mapped to arrays of parameters, and any transformation of these values will require parameters to be maintained.

If your code doesn't modify the parameters then SQLSelect::getWhereParameterised can be used in order to return these SQL statements as a simple array of strings. The resulting parameters are still maintained, but are instead be returned by referenced through the first parameter to this method.

E.g.

Before:

:::php
function augmentSQL(SQLQuery $query, DataQuery $dataQuery = null) {
	$locale = Translatable::get_current_locale();
	if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhere())))  {
		$qry = sprintf('"Locale" = \'%s\'', Convert::raw2sql($locale));
		$query->addWhere($qry); 
	}
}

After:

:::php
function augmentSQL(SQLSelect $query, DataQuery $dataQuery = null) {
	$locale = Translatable::get_current_locale();
	if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhereParameterised($parameters))))  {
		$query->addWhere(array(
			'"Locale"' => $locale
		)); 
	}
}

Update code that interacts with the DB schema

Updating database schema is now done by updateSchema with a callback, rather than relying on user code to call beginSchemaUpdate and endSchemaUpdate around the call.

Since the schema management object is separate from the database controller you interact with it via DB::get_schema instead of DB::get_conn (previously named DB::getConn)

Before:

:::php
<?php
$conn = DB::getConn();
$conn->beginSchemaUpdate();
foreach($dataClasses as $dataClass) {
	singleton($dataClass)->requireTable();
}
$conn->endSchemaUpdate();

After:

:::php
<?php
$schema = DB::get_schema();
$schema->schemaUpdate(function() use($dataClasses){
	foreach($dataClasses as $dataClass) {
		singleton($dataClass)->requireTable();
	}
});

Also should be noted is that many functions have been renamed to conform better with coding conventions. E.g. DB::requireTable is now DB::require_table

Other

• Helper function DB::placeholders can be used to generate a comma separated list of placeholders useful for creating "WHERE ... IN (?,...)" SQL fragments
• Implemented Convert::symbol2sql to safely encode database and table names and identifiers. E.g. Convert::symbol2sql('table.column') => '"table"."column"';
• Convert::raw2sql may now quote the escaped value, as well as safely escape it, according to the current database adaptor's preference.
• DB class has been updated and many static methods have been renamed to conform to coding convention.
  • Renamed API:
    • affectedRows -> affected_rows
    • checkAndRepairTable -> check_and_repair_table
    • createDatabase -> create_database
    • createField -> createField
    • createTable -> createTable
    • dontRequireField -> dont_require_field
    • dontRequireTable -> dont_require_table
    • fieldList -> field_list
    • getConn -> get_conn
    • getGeneratedID -> get_generated_id
    • isActive -> is_active
    • requireField -> require_field
    • requireIndex -> require_index
    • requireTable -> require_table
    • setConn -> set_conn
    • tableList -> table_list
  • Deprecated API:
    • getConnect (Was placeholder for PDO connection string building code, but is made redundant after the PDOConnector being fully abstracted)
  • New API:
    • build_sql - Hook into new SQL generation code
    • get_connector (Nothing to do with getConnect)
    • get_schema
    • placeholders
    • prepared_query
• SS_Database class has been updated and many functions have been deprecated, or refactored into the various other database classes. Most of the database management classes remain in the database controller, due to individual databases (changing, creating of, etc) varying quite a lot from API to API, but schema updates within a database itself is managed by an attached DBSchemaManager
  • Refactored into DBSchemaManager:
    • createTable
    • alterTable
    • renameTable
    • createField
    • renameField
    • fieldList
    • tableList
    • hasTable
    • enumValuesForField
    • beginSchemaUpdate and endSchemaUpdate -> Use schemaUpdate with a callback
    • cancelSchemaUpdate
    • isSchemaUpdating
    • doesSchemaNeedUpdating
    • transCreateTable
    • transAlterTable
    • transCreateField
    • transCreateField
    • transCreateIndex
    • transAlterField
    • transAlterIndex
    • requireTable
    • dontRequireTable
    • requireIndex
    • hasField
    • requireField
    • dontRequireField
  • Refactored into DBQueryBuilder
    • sqlQueryToString
  • Deprecated:
    • getConnect - Was intended for use with PDO, but was never implemented, and is now redundant, now that there is a stand-alone PDOConnector
    • prepStringForDB - Use quoteString instead
    • dropDatabase - Use dropSelectedDatabase
    • createDatabase - Use selectDatabase with the second parameter set to true instead
    • allDatabaseNames - Use databaseList instead
    • currentDatabase - Use getSelectedDatabase instead
    • addslashes - Use escapeString instead
• LogErrorEmailFormatter now better displays SQL queries in errors by respecting line breaks
• Installer has been majorly upgraded to handle the new database configuration options and additional PDO functionality.
• Created SS_DatabaseException to emit database errors. Query information such as SQL and any relevant parameters may be used by error handling user code that catches this exception.
• The SQLConditionGroup interface has been created to represent dynamically evaluated SQL conditions. This may be used to wrap a class that generates a custom SQL clause(s) to be evaluated at the time of execution.
• DataObject constants CHANGE_NONE, CHANGE_STRICT, and CHANGE_VALUE have been created to provide more verbosity to field modification detection. This replaces the use of various magic numbers with the same meaning.
• create_table_options now uses constants as API specific filters rather than strings. This is in order to promote better referencing of elements across the codebase. See FulltextSearchable->enable for example.