silverstripe-framework/docs/en/changelogs/3.2.0.md

# 3.2.0 (unreleased)

## Overview

### Framework

 * Minimum PHP version raised to 5.3.3
 * DataObject::validate() method visibility changed to public
 * NumericField now uses HTML5 "number" type instead of "text"
 * UploadField "Select from files" shows files in all folders by default
 * UploadField won't display an overwrite warning unless Upload:replaceFile is true
 * HtmlEditorField no longer substitutes `<blockquote />` for indented text
 * ClassInfo::dataClassesFor now returns classes which should have tables, regardless of whether those
   tables actually exist.
 * SS_Filterable, SS_Limitable and SS_Sortable now explicitly extend SS_List
 * ToggleField was deprecated in 3.1, and has been removed. Use custom Javascript with ReadonlyField instead.
 * ExactMatchMultiFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter instead.
 * NegationFilter was deprecated in 3.1, and has been removed. Use ExactMatchFilter:not instead.
 * StartsWithMultiFilter was deprecated in 3.1, and has been removed. Use StartsWithFilter instead.
 * ScheduledTask and subclasses like DailyTask were deprecated in 3.1, and have been removed.
 Use custom code instead, or a module like silverstripe-crontask: https://github.com/silverstripe-labs/silverstripe-crontask

### CMS

 * `SearchForm::getSearchQuery` no longer pre-escapes search keywords and must be cast in your template

## Changelog

### CMS

### DataObject::validate() method visibility changed to public

The visibility of `DataObject::validate()` has been changed from `protected` to `public`.

Any existing classes that currently set this as `protected` should be changed like in
this example:

	::php
	class MyDataClass extends DataObject {
		...
		public function validate() {
			...
		}
		...
	}

### UploadField "Select from files" shows files in all folders by default

In order to list files in a single folder by default (previous default behaviour), 
use `setDisplayFolderName()` with a folder path relative to `assets/`:

	UploadField::create('MyField')->setDisplayFolderName('Uploads');

### UploadField won't display an overwrite warning unless Upload:replaceFile is true

The configuration setting `UploadField:overwriteWarning` is dependent on `Upload:replaceFile` 
which is set to false by default.

To display a warning before overwriting a file:

Via config:

	::yaml
	Upload:
	  # Replace an existing file rather than renaming the new one.
	  replaceFile: true
	UploadField:
	  # Warning before overwriting existing file (only relevant when Upload: replaceFile is true)
	  overwriteWarning: true

Or per instance:

	::php
	$uploadField->getUpload()->setReplaceFile(true);
	$uploadField->setOverwriteWarning(true);

### File.allowed_extensions restrictions

Certain file types such as swf, html, htm, xhtml and xml have been removed from the list
of allowable file uploads. If your application requires the ability to upload these,
you will need to append these to the `File.allowed_extensions` config as necessary.
Also if uploading other file types, it's necessary to ensure that `File.allowed_extensions`
includes that extension, as extensions passed to `[api:UploadField]` will be filtered against
this list.

### Removed format detection in i18n::$date_format and i18n::$time_format

Localized dates cause inconsistencies in client-side vs. server-side formatting
and validation, particularly in abbreviated month names. The default date
format has been changed to "yyyy-MM-dd" (e.g. 2014-12-31). 
New users will continue to have the option for a localized date
format in their profile (based on their chosen locale).
If you have existing users with `Member.DateFormat` set to a format
including "MMM" or "MMMM", consider deleting those formats to fall back to
the global (and more stable) default.

### Bugfixes
 * Migration of code to use new parameterised framework

### Framework

 * Implementation of a parameterised query framework eliminating the need to manually escape variables for
   use in SQL queries. This has been integrated into nearly every level of the database ORM.
 * Refactor of database connectivity classes into separate components linked together through dependency injection
 * Refactor of `SQLQuery` into separate objects for each query type: `SQLSelect`, `SQLDelete`, `SQLUpdate` and `SQLInsert`
 * Rename of API methods to conform to coding conventions
 * PDO is now a standard connector, and is available for all database interfaces
 * Additional database and query generation tools

## Bugfixes

 * Reduced database regeneration chances on subsequent rebuilds after the initial dev/build
 * Elimination of various SQL injection vulnerability points
 * `DataObject::writeComponents()` now called correctly during `DataObject::write()`
 * Fixed missing theme declaration in installer
 * Fixed incorrect use of non-existing exception classes (e.g. `HTTPResponse_exception`)
 * `GridState` fixed to distinguish between check for missing values, and creation of
   nested state values, in order to prevent non-empty values being returned for
   missing keys. This was breaking `DataObject::get_by_id` by passing in an object
   for the ID.
 * Fixed order of `File` fulltext searchable fields to use same order as actual fields.
   This is required to prevent unnecessary rebuild of MS SQL databases when fulltext
   searching is enabled.

## Upgrading

### Update code that uses SQLQuery

SQLQuery is still implemented, but now extends the new SQLSelect class and has some methods
deprecated. Previously this class was used for both selecting and deleting, but these
have been superceded by the specialised SQLSelect and SQLDelete classes. Additionally,
3.2 now provides SQLUpdate and SQLInsert to generate parameterised query friendly
data updates.

SQLSelect, SQLDelete and SQLUpdate all inherit from SQLConditionalExpression, which
implements toSelect, toDelete, and toUpdate to generate basic transformations
between query types.

In the past SQLQuery->setDelete(true) would be used to turn a select into a delete, 
although now a new SQLDelete object should be created from a separate SQLSelect.

Before:

	:::php
	<?php
	$query = new SQLQuery('*');
	$query->setFrom('"SiteTree"');
	$query->setWhere('"SiteTree"."ShowInMenus" = 0');
	$query->setDelete(true);
	$query->execute();

After:

	:::php
	<?php
	$query = SQLDelete::create()
		->setFrom('"SiteTree"')
		->setWhere(array('"SiteTree"."ShowInMenus"' => 0));
	$query->execute();

Alternatively:

	:::php
	<?php
	$query = SQLSelect::create()
		->setFrom('"SiteTree"')
		->setWhere(array('"SiteTree"."ShowInMenus"' => 0))
		->toDelete();
	$query->execute();

Also, take care for any code or functions which expect an object of type `SQLQuery`, as 
these references should be replaced with `SQLSelect`. Legacy code which generates
`SQLQuery` can still communicate with new code that expects `SQLSelect` as it is a
subclass of `SQLSelect`, but the inverse is not true.

### Update code that interacts with SQL strings to use parameters

The Silverstripe ORM (object relation model) has moved from using escaped SQL strings
to query the database, to a combination of parameterised SQL expressions alongside
a related list of parameter values. As a result of this, it is necessary to assume
that any `SQLSelect` object (previously `SQLQuery`) may, and will usually, have 
un-injected parameters.

All database queries performed through `DataList`, `DataQuery` and `SQLQuery` will continue
to work, as will those through `DataObject::get()` (which returns a filterable `DataList`).
However, any conditional expression that includes values escaped with `Convert::raw2sql()`
should use the new standard syntax. This new querying standard method enforces a much
higher level of security than was previously available, and all code using manual
escaping should be upgraded.

See [the security topic](/topics/security#parameterised-queries) for details on why this is necessary, or
[the databamodel topic](/topics/datamodel#raw-sql-options-for-advanced-users) for more information.

As a result of this upgrade there are now very few cases where `Convert::raw2sql` needs to be used.

Examples of areas where queries should be upgraded are below:

1. #### Querying the database directly through DB, including non-SELECT queries

	Before:

		:::php
		<?php

		// Note: No deprecation notices will be caused here
		DB::query("UPDATE \"SiteTree\" SET \"Title\" LIKE '%" . Convert::raw2sql($myTitle) . "%' WHERE \"ID\" = 1");
		$myPages = DB::query(sprintf('SELECT "ID" FROM "MyObject" WHERE "Title" = \'%s\'', Convert::raw2sql($parentTitle)));

	After:

		:::php
		<?php

		DB::prepared_query(
			'UPDATE "SiteTree" SET "Title" LIKE ? WHERE "ID" = ?',
			array("%{$myTitle}%", 1)
		);
		$myPages = DB::prepared_query(
			'SELECT "ID" FROM "MyObject" WHERE "Title" = ?',
			array($parentTitle)
		);

2. #### Querying the database through `SQLQuery` (deprecated)

	Before:

	Note: Use of SQLQuery would generate a deprecation notice if left un-upgraded.

		:::php
		<?php

		$query = new SQLQuery('*', '"SiteTree"', "\"URLSegment\" = '".Convert::raw2sql($testURL)."'");

		$query->addWhere(array(
			'"ParentID" = \''.intval($parentID).'\'',
			'"ID" IN (SELECT "PageID" FROM "MyObject")'
		));

		$query->addWhere("\"Title\" LIKE '%".Convert::raw2sql($myText)."' OR \"Title\" LIKE '".Convert::raw2sql($myText)."%'");

	After, substituting `SQLSelect` for the deprecated `SQLQuery`:

	Note: The inclusion of properly ANSI quoted symbols with the table name included,
	as per best coding practices.

		:::php
		<?php

		$query = SQLSelect::create('*', '"SiteTree"', array('"SiteTree"."URLSegment" = ?' => $testURL));

		$query->addWhere(array(
			'"SiteTree"."ParentID"' =>  // Note that the " = ?" is optional for simple comparison
				array( // Syntax for parameter casting for supporting databases
					'value' => $parentID,
					'type' => 'integer'
				),
			'"SiteTree"."ID" IN (SELECT "MyObject"."PageID" FROM "MyObject")' // Raw SQL condition with no parameters
		));

		// Multiple parameters may be assigned for a single query (this should not be associative)
		$query->addWhere(array(
			'"SiteTree"."Title" LIKE %? OR "SiteTree"."Title" LIKE %?' => array($myText, $myText)
		));

3. #### Querying the database through `DataList`, `DataQuery`, and `DataObject`

	Before:

		:::php
		<?php

		$items = DataObject::get_one('MyObject', '"Details" = \''.Convert::raw2sql($details).'\'');
		$things = MyObject::get()->where('"Name" = \''.Convert::raw2sql($name).'\'');
		$list = DataList::create('Banner')->where(array(
			'"ParentID" IS NOT NULL',
			'"Title" = \'' . Convert::raw2sql($title) . '\''
		);

	After:

		:::php
		<?php

		$items = DataObject::get_one('MyObject', array('"MyObject"."Details"' => $details));
		$things = MyObject::get()->where(array('"MyObject"."Name" = ?' => $name));
		$list = DataList::create('Banner')->where(array(
			'"ParentID" IS NOT NULL',
			'"Title" = ?', $title
		);

4. #### Interaction with the `DataList::sql()`, `DataQuery::sql()` or `SQLSelect::sql()` methods

	The place where legacy code would almost certainly fail is any code that calls
	`SQLQuery::sql`, `DataList::sql` or `DataQuery::sql`, as the api requires that user
	code passes in an argument here to retrieve SQL parameters by value.

	User code that assumes parameterless queries will likely fail, and need to be
	updated to handle this case properly.

	Before:

		:::php
		<?php

		// Generate query
		$argument = 'whatever';
		$query = SQLSelect::create()
			->setFrom('"SiteTree"')
			->setWhere(array("\"SiteTree\".\"Title\" LIKE '" . Convert::raw2sql($argument) . "'"));

		// Inspect elements of the query
		$sql = $query->sql();
		$sql = preg_replace('/LIKE \'(.+)\'/', 'LIKE \'%${1}%\'', $sql); // Adds %% around the argument

		// Pass new query to database connector
		DB::query($sql);

	After:

		:::php
		<?php

		// Generate query
		$argument = 'whatever';
		$query = SQLSelect::create()
			->setFrom('"SiteTree"')
			->setWhere(array('"SiteTree"."Title" LIKE ?' => $argument));

		// Inspect elements of the query
		$sql = $query->sql($parameters);
		foreach($parameters as $key => $value) {
			// Adds %% around arguments
			$parameters[$key] = "%{$value}%";
		}

		// Pass new query to database connector
		// Note that DB::query($sql) would fail, as it would contain ? with missing parameters
		DB::prepared_query($sql, $parameters);

	Also note that the parameters may not be a single level array, as certain values
	may be forced to be cast as a certain type (where supported by the current API).

	E.g.

		:::php
		<?php

		$parameters = array(
			10,
			array('value' => 0, 'type' => 'boolean') // May also contain other database API specific options
		)
		DB::prepared_query('DELETE FROM "MyObject" WHERE ParentID = ? OR IsValid = ?', $parameters);

5. #### Update implementations of augmentSQL

Since this method now takes a `SQLSelect` as a first parameter, existing code referencing the deprecated `SQLQuery`
type will raise a PHP error.

Furthermore, it's important to note that even though the signature of `SQLSelect::getWhere` is similar to the old
`SQLQuery::getWhere` the result will actually be an associative array of SQL fragments mapped to arrays of
parameters, and any transformation of these values will require parameters to be maintained.

If your code doesn't modify the parameters then `SQLSelect::getWhereParameterised` can be used in order to return
these SQL statements as a simple array of strings. The resulting parameters are still maintained, but are
instead be returned by referenced through the first parameter to this method.

E.g.

Before:

	:::php
	function augmentSQL(SQLQuery $query, DataQuery $dataQuery = null) {
		$locale = Translatable::get_current_locale();
		if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhere())))  {
			$qry = sprintf('"Locale" = \'%s\'', Convert::raw2sql($locale));
			$query->addWhere($qry); 
		}
	}

After:

	:::php
	function augmentSQL(SQLSelect $query, DataQuery $dataQuery = null) {
		$locale = Translatable::get_current_locale();
		if(!preg_match('/("|\'|`)Locale("|\'|`)/', implode(' ', $query->getWhereParameterised($parameters))))  {
			$query->addWhere(array(
				'"Locale"' => $locale
			)); 
		}
	}


### Update code that interacts with the DB schema

Updating database schema is now done by `updateSchema` with a callback, rather than relying
on user code to call `beginSchemaUpdate` and `endSchemaUpdate` around the call.

Since the schema management object is separate from the database controller you
interact with it via `DB::get_schema` instead of `DB::get_conn` (previously named
`DB::getConn`)

Before:

	:::php
	<?php
	$conn = DB::getConn();
	$conn->beginSchemaUpdate();
	foreach($dataClasses as $dataClass) {
		singleton($dataClass)->requireTable();
	}
	$conn->endSchemaUpdate();

After:

	:::php
	<?php
	$schema = DB::get_schema();
	$schema->schemaUpdate(function() use($dataClasses){
		foreach($dataClasses as $dataClass) {
			singleton($dataClass)->requireTable();
		}
	});

Also should be noted is that many functions have been renamed to conform better with 
coding conventions. E.g. `DB::requireTable` is now `DB::require_table`

### Other

 * Helper function `DB::placeholders` can be used to generate a comma separated list of placeholders
   useful for creating "WHERE ... IN (?,...)" SQL fragments
 * Implemented Convert::symbol2sql to safely encode database and table names and identifiers.
   E.g. `Convert::symbol2sql('table.column') => '"table"."column"';`
 * `Convert::raw2sql` may now quote the escaped value, as well as safely escape it, according to the current
   database adaptor's preference.
 * `DB` class has been updated and many static methods have been renamed to conform to coding convention.
	* Renamed API:
		- `affectedRows` -> `affected_rows`
		- `checkAndRepairTable` -> `check_and_repair_table`
		- `createDatabase` -> `create_database`
		- `createField` -> `createField`
		- `createTable` -> `createTable`
		- `dontRequireField` -> `dont_require_field`
		- `dontRequireTable` -> `dont_require_table`
		- `fieldList` -> `field_list`
		- `getConn` -> `get_conn`
		- `getGeneratedID` -> `get_generated_id`
		- `isActive` -> `is_active`
		- `requireField` -> `require_field`
		- `requireIndex` -> `require_index`
		- `requireTable` -> `require_table`
		- `setConn` -> `set_conn`
		- `tableList` -> `table_list`
	* Deprecated API:
		- `getConnect` (Was placeholder for PDO connection string building code, but is made
		  redundant after the PDOConnector being fully abstracted)
	* New API:
		- `build_sql` - Hook into new SQL generation code
		- `get_connector` (Nothing to do with getConnect)
		- `get_schema`
		- `placeholders`
		- `prepared_query`
 * `SS_Database` class has been updated and many functions have been deprecated, or refactored into
   the various other database classes. Most of the database management classes remain in the database
   controller, due to individual databases (changing, creating of, etc) varying quite a lot from
   API to API, but schema updates within a database itself is managed by an attached DBSchemaManager
	* Refactored into DBSchemaManager:
		- `createTable`
		- `alterTable`
		- `renameTable`
		- `createField`
		- `renameField`
		- `fieldList`
		- `tableList`
		- `hasTable`
		- `enumValuesForField`
		- `beginSchemaUpdate` and `endSchemaUpdate` -> Use `schemaUpdate` with a callback
		- `cancelSchemaUpdate`
		- `isSchemaUpdating`
		- `doesSchemaNeedUpdating`
		- `transCreateTable`
		- `transAlterTable`
		- `transCreateField`
		- `transCreateField`
		- `transCreateIndex`
		- `transAlterField`
		- `transAlterIndex`
		- `requireTable`
		- `dontRequireTable`
		- `requireIndex`
		- `hasField`
		- `requireField`
		- `dontRequireField`
	* Refactored into DBQueryBuilder
		- `sqlQueryToString`
	* Deprecated:
		- `getConnect` - Was intended for use with PDO, but was never implemented, and is now
		  redundant, now that there is a stand-alone `PDOConnector`
		- `prepStringForDB` - Use `quoteString` instead
		- `dropDatabase` - Use `dropSelectedDatabase`
		- `createDatabase` - Use `selectDatabase` with the second parameter set to true instead
		- `allDatabaseNames` - Use `databaseList` instead
		- `currentDatabase` - Use `getSelectedDatabase` instead
		- `addslashes` - Use `escapeString` instead
 * LogErrorEmailFormatter now better displays SQL queries in errors by respecting line breaks
 * Installer has been majorly upgraded to handle the new database configuration options
   and additional PDO functionality.
 * Created SS_DatabaseException to emit database errors. Query information such as SQL
   and any relevant parameters may be used by error handling user code that catches
   this exception.
 * The SQLConditionGroup interface has been created to represent dynamically 
   evaluated SQL conditions. This may be used to wrap a class that generates
   a custom SQL clause(s) to be evaluated at the time of execution.
 * DataObject constants CHANGE_NONE, CHANGE_STRICT, and CHANGE_VALUE have been created
   to provide more verbosity to field modification detection. This replaces the use of
   various magic numbers with the same meaning.
 * create_table_options now uses constants as API specific filters rather than strings.
   This is in order to promote better referencing of elements across the codebase.
   See `FulltextSearchable->enable` for example.