AS requested by tractorcow: https://github.com/silverstripe/silverstripe-postgresql/pull/26
Tested on MySQL (succeeds as expected) Tested on PostgreSQL (and fails as expected, but passes with patch). The fixes test only DataQueryTest related items for consistency, e.g. by avoiding DataObject calls.
Versioned is not writing Version to _version tables for subclasses of Version dataobjects which have their own DB fields
- Fix disjoint of ID / RecordID (which should be the same)
- Fix calculation of new record version
- Fix use of empty vs !isset to check for existing version
When sorting a DataQuery over a relation, the SQLQuery automatically included the sort column. The issue with the implement is that potentially the joined record has a field with the same name as the source record causing it to be overridden.
In the attached test case, without the patch the title will be set to 'Bar' rather than 'Foo'.
This patch aliases the sort column. Alternativally a patch would be to
BUG Fixes missing i18n translation in Date::TimeDiffIn
BUG Fixes Date::TimeDiffIn not respecting mocked SS_Datetime::now
This provides less vague date periods. I.e. "36 days" has a lot more relevance that "1 month"
Reduced duplication of time period calculation code
(ref: CWPBUG-141)
This would ideally be fixed with the ability to use an external library
like gettext, but that's an API change. This for now fixes the issue
where a singular like "Page" returns "Pags" for the plural name.
This is a bug that combines Hierarchy, Versioned and LeftAndMain admins and CMSSiteTreeFilters.
This bug can be reproduced by having a large site tree with enough deleted pages in it so it doesn't
pre load all the children pages when initially opening an admin. Filter by either 'All pages including deleted'
or 'Deleted pages'. For CMS users it will look like deleted pages are gone.
The solution involves a couple of smaller fixes in both CMS and framework modules.
1) Ensure that 'numHistoricalChildren' are used instead of 'numChildren' when dealing with deleted pages
2) LeftAndMain::currentPage() deletes all the 'marking' cache previously built up by Hierarchy::markPartialTree()
3) Use Versioned::get_included_deleted() instead of raw DB queries against the DataObject tables when calculating parents in CMSSiteTreeFilter
BUG Fix incompatibility in Member_GroupList
Fix regressions in merges from 3.1
BUG Fix Security failing on test classes
BUG Fix postgresql compatibility
Clarify sql encoding of table names
Database abstraction broken up into controller, connector, query builder, and schema manager, each independently configurable via YAML / Injector
Creation of new DBQueryGenerator for database specific generation of SQL
Support for parameterised queries, move of code base to use these over escaped conditions
Refactor of SQLQuery into separate query classes for each of INSERT UPDATE DELETE and SELECT
Support for PDO
Installation process upgraded to use new ORM
SS_DatabaseException created to handle database errors, maintaining details of raw sql and parameter details for user code designed interested in that data.
Renamed DB static methods to conform correctly to naming conventions (e.g. DB::getConn -> DB::get_conn)
3.2 upgrade docs
Performance Optimisation and simplification of code to use more concise API
API Ability for database adapters to register extensions to ConfigureFromEnv.php
Previously selectFromTable would simply try to select the composite field name. This expands the extraField name to include the children field names and uses CompositeDBField::writeToManipulation to generate the correct SQL for the queries.
Move functionality from static start and destroy functions into instance
methods, allowing these to be overloaded. This works the same way as
calling Session::set() which then in turn calls inst_set()
Additionally use Injector to create the default Session instance to
allow the class to be swapped out.
BUG Disabled disruptive test case in DirectorTest
API RequestProcessor and VersionedRequestFilter now both correctly implement RequestFilter
Better PHPDoc on RequestFilter and implementations
This issue is caused by the odd default behaviour of Zend_Date, which attempts to parse yyyy-mm-dd format date and times as though they were yyyy-dd-mm.
DataObject::validate() is currently set to protected, but this means
you can't call validate() from outside the context of itself unless
you overload the method to use a public visibility and then call
parent::validate()
As it would turn out, most classes that overload this method already
set the visibility to public, so it would make sense the parent matches
that as well.
- Document the format for descriptor arrays
- Implement the behaviour that developers have come to expect for
string descriptors of indexes
- Add test for handling of unique indexes (MySQL & sqlite3)
- Resolve#2403
Versioned needs to convert unique indexes to non-unique for its suffixed
tables, such as Foo_Live and Foo_versions. Because DataObject accepts
string descriptors such as array('UniqIDX' => 'unique (Uniq)') as well
as array-based descriptors, Versioned needs to recognize string
descriptors. This patch accomplishes that. Before, Versioned would fail
to convert string-described indexes to non-unique, resulting in run-time
errors when creating a new version of an object.
If more than two $from were added through SQLQuery->addFrom(),
the getOrderedJoins() comparison kicks in. It assumes all $from
parts are in array notation, which isn't always the case.
For legacy reasons, and because we don't have full API support,
you can still add literal joins through addFrom('INNER JOIN ...').
On PHP 5.3, the ordering comparison still works because it
allows array access in strings, with string rather than numeric indexes.
Thankfully that's no longer supported in PHP 5.4.
DataQuery::initialiseQuery() will add a default sort to a query,
and when calling up an aggregate it will make a query like this
which doesn't make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID"
In this case there is no need to add the ORDER BY, and it will
break databases like MSSQL in cases such as
GenericTemplateGlobalProvider
which provides a default List() function for adding aggregates
into SSViewer template cacheblocks.
If we add a limit, however, then it does make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID" LIMIT 10
This fixes SQLQuery::aggregate() to NOT add an ORDER BY to an
aggregate call if there is no limit.
If multiple image manipulations are performend the resulting cached image is stored in assets/_resampled because the cached version of the image has no ParentID, which cacheFilename needs to set the correct path.
Allow DataList::limit() to take a null value to remove the limit.
Added tests for limit(). Note the one failure, currently the ORM doesn't support unlimited values with an offset.
The function "first" on ArrayList uses the PHP function "reset", which
returns false if there aren't any elements in the array. Two functions
inside ArrayList use this function, "canFilterBy" and "byID". I've
changed these functions to catch the possibility of a false return from
first().
Commit 964b3f2 fixed an issue where dbObject was returning casting helpers for
fields that were not actually DB objects, but had something in $casting config.
However, because dbObject was no longer calling DataObject->castingHelper, this
exposed a bug that the underlying function db($fieldName) was not returning
field specs for the base fields that are created by SS automatically on all
DataObjects (i.e. Created, LastEdited, etc).
This commit fixes the underlying issue that DataObject->db($fieldName) should
return the field specs for *all* DB fields like its documentation says it will,
including those base fields that are automatically created and do not appear in
$db.
SQLQuery->setLimit(0, 99) should result in "SELECT ... LIMIT 0 OFFSET 1".
In fact it does "SELECT ..." without a LIMIT clause at all,
which is unexpected. This is regardless of the $offset value.
This caused problems when duplicate() was used in the CMS UI
to duplicate a SiteTree object. Since every object of this type
has a ParentID relation, it copied this empty relation into
new "ghost page".
See https://github.com/silverstripe/silverstripe-cms/issues/689
When DataObject::update() is run with relation fields and the relationship
is new the relationship ID was not set on the DataObject. This patch fixes
this. Fixes issue 6195 in open.silverstripe.org.
With a many to many relation, e.g. SiteTree_MyRelation, and listing
them in your template then adding ?archiveDate=x in the URL, a SQL
error is shown because Versioned::augmentSQL() tries to query the
non-existent table "SiteTree_MyRelation_versions" assuming there's
versioning setup, but there isn't.
Rendering potentially 1000s of nodes can exceed the CPU and memory constraints
of a normal PHP process, as well as the rendering capabilities of browsers.
Set a hard maximum for the renderable nodes, deferring to a "show as list" action
in the main CMS tree. For TreeDropdownField, we don't have the list fallback option,
so ask the user to search for the node title instead.
Also makes both the "node_threshold_total" and "node_threshold_leaf" values configurable
Strictly speaking, no longer required since we auto-quote simple
field names in DataQuery now, but since the majority of sorts in core is
already quoted we should stay consistent.
- Renamed $minNodeCount to more accurate $nodeCountThreshold
- The $minNodeCount attribute wasn't properly respected
during actual querying, so SilverStripe would always traverse
the entire tree (and load all objects into memory),
before then marking nodes as "unexpanded", which prevents
them from actually being rendered.
- Fixes nodes on search results to be expanded by default
- Fixes nodes on search results to correctly ajax-expand
This is to fix a bug that caused CheckboxSetFields to throw an error
when trying to call this function when editing a new DataObject. This
occurred when using the advancedworkflow module.
Thanks to simonwelsh for the majority of the work on this fix.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
This causes a 'Fatal error: Call to a member function hasMethod() on a non-object'.
This can happen when displaying a field in a gridfield on a belongs_to relationship.
+ has a special meaning in the URLs so overall it's a good idea to
strip them out. Otherwise they would need to appear in their ugly url
encoded form "%2B".
Refer: http://open.silverstripe.org/ticket/7929
BUG Issue with deleted records not being queried properly.
API DataQuery::expressionForField no longer requires a second parameter. Rather the query object is inferred from the DataQuery itself. This should improve consistency of use of this function.
In 3.0 there was some confusion about whether DataLists and ArrayLists
were mutable or not. If DataLists were immutable, they'd return the result, and your code
would look like
$list = $list->filter(....);
If DataLists were mutable, they'd operate on themselves, returning nothing, and your code
would look like
$list->filter(....);
This makes all DataLists and ArrayList immutable for all _searching_ operations.
Operations on DataList that modify the underlying SQL data store remain mutating.
- These functions no longer mutate the existing object, and if you do not capture the value
returned by them will have no effect:
ArrayList#reverse
ArrayList#sort
ArrayList#filter
ArrayList#exclude
DataList#dataQuery (use DataList#alterDataQuery to modify dataQuery in a safe manner)
DataList#where
DataList#limit
DataList#sort
DataList#addFilter
DataList#applyFilterContext
DataList#innerJoin
DataList#leftJoin
DataList#find
DataList#byIDs
DataList#reverse
- DataList#setDataQueryParam has been added as syntactic sugar around the most common
cause of accessing the dataQuery directly - setting query parameters
- RelationList#setForeignID has been removed. Always use RelationList#forForeignID
when querying, and overload RelationList#foreignIDList when subclassing.
- Relatedly,the protected variable RelationList->foreignID has been removed, as the ID is
now stored on a query parameter. Use RelationList#getForeignID to read it.
Session is not initialized by the time we need to use
the setting in DB::connect(). Cookie values get initialized
automatically for each request.
Tightened name format validation to ensure it can only
be used for temporary databases, rather than switching
the browser session to a different production database.
Encrypting token for secure cookie usage.
Added dev/generatesecuretoken to generate this token.
Not storing in YML config directly because of web access issues.
Enables more generic use of the fixture facilities
without dependency on the YAML format, for example
when creating fixtures from Behat step definitions.
Note: The YamlFixture class needs to be created via
Injector::inst()->create('YamlFixture') now,
direct instantiation is no longer supported.
This bug will surface when using the ORM and adding an join to DataList
where a DataObject inherits another DataObject.
If you for example want to restrict the number of pages that only have a
related Staff object:
$list = DataList::create('Page')
->InnerJoin('Staff', '"Staff"."ID" = "Page"."StaffID");
This will create a SQL query where the INNER JOIN is before the
LEFT JOIN of Page and SiteTree in the resulting SQL string. In MySQL
and PostgreSQL this will create an invalid query.
This patch solves the problem by sorting the joins.
Avoid PHPUnit throwing "test didn't run any assertions"
notices in PHP. If nothing else, it keeps test output
looking less broken by default, making it more likely
that actual errors do get noticed.
Quoted table / column names to make test cases work in postgres
BUG Fixed issue with SQLQuery::lastRow crashing on empty set. Added test cases for lastRow and firstRow.
Quoted table / column names to make test cases work in postgres
Merge branch '3.0-sqlquery-lastrow-fix' of github.com:tractorcow/sapphire into 3.0-sqlquery-lastrow-fix
The specific situation where this is useful is where populateDefaults on
DataObjects needs to query the database. This will break the dev/build
when it tries to create the object via singleton - the query will not be
able to be executed if the table is not there or its schema has changed.
For an example of such use case see Translatable::populateDefaults.
The entire framework repo (with the exception of system-generated files) has been amended to respect the 120c line-length limit. This is in preparation for the enforcement of this rule with PHP_CodeSniffer.
Adds three extra methods to Data/SQLQuery query that allow for starting
a disjunctive subgroup, a conjunctive subgroup and for ending a subgroup.
Database::sqlWhereToString() now builds up the WHERE clause one by one
instead of with a straight implode. Uses a stack to know which conenctive
to use.
ADDED: Test cases correctly checking for changes (and no changes) to the data model for both fields and indexes.
FIXED: References to indexes throughough the code that probably should have quoted field names. This prevents a lot of 'spam' during dev build. This includes an updated FulltextSearchable test case.
The table name in the join was being escaped, though table
names aren't escaped anywhere else. This breaks
namespaced classes, which rely on unescaped backslashes.
The values in $values aren't returned in any order, so this test can randomly fail. This
changes the check from expected = $values to $values \cap expected = $value. PHP's array_intersect
maintains the keys of the first array, so order is preserved. The intersect also guarentees that the
only accepted values are the expected ones.
While well-intentioned, this test keeps causing problems
due to wrong timezone settings in test mode.
It shouldn't completely abort test execution,
since its more of an environment sanity check than a failed test.
Refactored to mark test skipped (regardless of offset, as long as
its greater than 5 seconds). And skipping tests altogether
on SQLite3 with new supportsTimezoneOverride() check.
SapphireTest->setUp() sets the PHP timezone to UTC (see 59547745),
but SQLite doesn't support this for a DB connection.
Since changing it on a global UNIX system level is infeasible,
the tests need to be skipped.
If the applyRelation() was passed a relation that went to a class with a parent
class with a database table, applyRelation would return the name of the parent
class, rather than the class the relation was actually too.
Without this bugfix, if you had a Page that used to be a SiteTree, and you tried to use Versiond::get_version() or Versioned::get_latest_version() to return the older SiteTree version, nothing would be returned, because the results were being filtered by ClassName. This caused bugs in the history panel for certain combinbations of page classname alteration.
If the applyRelation() was passed a relation that went to a class with a parent
class with a database table, applyRelation would return the name of the parent
class, rather than the class the relation was actually too.
This bug was caused by the fact that SQLQuery::whereAny() removed existing filters. In line with addWhere() and setWhere(), I split this into addWhereAny() and setWhereAny(). Strictly speaking, this drops the method SQLQuery::whereAny(), but it was really just an internal function for exclude, and so I think that's acceptable.
In some circumstances a custom generated list will already only contain
the items for the current page. The automatic limiting will then limit
the already limited list, breaking pagination. This allows you to disable
automatic limiting so all items are shown regardless of the current page.
