This is a bug that combines Hierarchy, Versioned and LeftAndMain admins and CMSSiteTreeFilters.
This bug can be reproduced by having a large site tree with enough deleted pages in it so it doesn't
pre load all the children pages when initially opening an admin. Filter by either 'All pages including deleted'
or 'Deleted pages'. For CMS users it will look like deleted pages are gone.
The solution involves a couple of smaller fixes in both CMS and framework modules.
1) Ensure that 'numHistoricalChildren' are used instead of 'numChildren' when dealing with deleted pages
2) LeftAndMain::currentPage() deletes all the 'marking' cache previously built up by Hierarchy::markPartialTree()
3) Use Versioned::get_included_deleted() instead of raw DB queries against the DataObject tables when calculating parents in CMSSiteTreeFilter
BUG Fix incompatibility in Member_GroupList
Fix regressions in merges from 3.1
BUG Fix Security failing on test classes
BUG Fix postgresql compatibility
Clarify sql encoding of table names
Database abstraction broken up into controller, connector, query builder, and schema manager, each independently configurable via YAML / Injector
Creation of new DBQueryGenerator for database specific generation of SQL
Support for parameterised queries, move of code base to use these over escaped conditions
Refactor of SQLQuery into separate query classes for each of INSERT UPDATE DELETE and SELECT
Support for PDO
Installation process upgraded to use new ORM
SS_DatabaseException created to handle database errors, maintaining details of raw sql and parameter details for user code designed interested in that data.
Renamed DB static methods to conform correctly to naming conventions (e.g. DB::getConn -> DB::get_conn)
3.2 upgrade docs
Performance Optimisation and simplification of code to use more concise API
API Ability for database adapters to register extensions to ConfigureFromEnv.php
Previously selectFromTable would simply try to select the composite field name. This expands the extraField name to include the children field names and uses CompositeDBField::writeToManipulation to generate the correct SQL for the queries.
BUG Disabled disruptive test case in DirectorTest
API RequestProcessor and VersionedRequestFilter now both correctly implement RequestFilter
Better PHPDoc on RequestFilter and implementations
The docs incorrectly stated that DataList::filter() needed escaped input, but this isn't true as the ExactMatch filter (and others) escape the values for you.
Anyone following that advice would have double escaped arguments
Seconds, not milliseconds.
microtime(true) returns "a float, which represents the current time in seconds since the Unix epoch accurate to the nearest microsecond" as per php docs.
This change fixes an issue where old/existing formatted images are used
when a filename is reused (by overwrite or by coincidence), regardless
of if the file contents have changed. To users this mainly manifests
as a file overwrite appearing not to work; the thumbnails in the CMS
show the original image until regeneration is forced.
Calling Image::deleteFormattedImages() after image upload ensures that
no stagnant formatted images will be used.
This issue is caused by the odd default behaviour of Zend_Date, which attempts to parse yyyy-mm-dd format date and times as though they were yyyy-dd-mm.
Allows usage of one consistent Zend_Cache backend
for all SilverStripe core storage, greatly simplifying
its configuration. This means a call to Aggregate::flushCache()
will indeed flush all caches if the backend doesn't support tags,
including any custom caches defined through SS_Cache.
Given caches are regarded transient that's an acceptable limitation.
DataObject::validate() is currently set to protected, but this means
you can't call validate() from outside the context of itself unless
you overload the method to use a public visibility and then call
parent::validate()
As it would turn out, most classes that overload this method already
set the visibility to public, so it would make sense the parent matches
that as well.
If we want DataObject->validate() to be used instead of
the form layer, we should allow for validation errors
to be passed through unchanged to the controller layer
so we can present them to the user. The context of
which class is written should be apparent from the stacktrace
of the exception.
- Document the format for descriptor arrays
- Implement the behaviour that developers have come to expect for
string descriptors of indexes
- Add test for handling of unique indexes (MySQL & sqlite3)
- Resolve#2403
Versioned needs to convert unique indexes to non-unique for its suffixed
tables, such as Foo_Live and Foo_versions. Because DataObject accepts
string descriptors such as array('UniqIDX' => 'unique (Uniq)') as well
as array-based descriptors, Versioned needs to recognize string
descriptors. This patch accomplishes that. Before, Versioned would fail
to convert string-described indexes to non-unique, resulting in run-time
errors when creating a new version of an object.
This ensures that the correct stage is selected, even if the request
does not come through the model as controller system. This fixes an
issue where custom controllers would always be on the "Stage" stage.
If more than two $from were added through SQLQuery->addFrom(),
the getOrderedJoins() comparison kicks in. It assumes all $from
parts are in array notation, which isn't always the case.
For legacy reasons, and because we don't have full API support,
you can still add literal joins through addFrom('INNER JOIN ...').
On PHP 5.3, the ordering comparison still works because it
allows array access in strings, with string rather than numeric indexes.
Thankfully that's no longer supported in PHP 5.4.
DataQuery::initialiseQuery() will add a default sort to a query,
and when calling up an aggregate it will make a query like this
which doesn't make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID"
In this case there is no need to add the ORDER BY, and it will
break databases like MSSQL in cases such as
GenericTemplateGlobalProvider
which provides a default List() function for adding aggregates
into SSViewer template cacheblocks.
If we add a limit, however, then it does make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID" LIMIT 10
This fixes SQLQuery::aggregate() to NOT add an ORDER BY to an
aggregate call if there is no limit.
If multiple image manipulations are performend the resulting cached image is stored in assets/_resampled because the cached version of the image has no ParentID, which cacheFilename needs to set the correct path.
Not a security issue as such, since the user input is sanitized
before being used in Versioned->augmentSQL(). But it shouldn't
reach the session state either, since that's commonly assumed
to be sanitized data, and it leaves unnecessary room for error.
strtotime() has fairly loose validation rules around dates,
but its a good "first line of defence".
Doesn't have much effect in practice, because charset and collation
are already hardcoded on an ALTER TABLE level (field definitions),
which take priority. Since most MySQL installs will still default
to a latin1 encoding, this propagates to the table though,
confusing devs and in some cases causing wrong data.
Example: A MSSQL->MySQL DB migration tool used the table metadata
to determine the charset, creating encoding issues.
In terms of hardcoding, we don't really support anything other than UTF8,
and the field-level settings are already hardcoded.
We should probably remove the field-specific settings and rely
on the DB defaults, but that's a sensitive API change
(need to set on existing DBs during upgrade).
Allow DataList::limit() to take a null value to remove the limit.
Added tests for limit(). Note the one failure, currently the ORM doesn't support unlimited values with an offset.
The function "first" on ArrayList uses the PHP function "reset", which
returns false if there aren't any elements in the array. Two functions
inside ArrayList use this function, "canFilterBy" and "byID". I've
changed these functions to catch the possibility of a false return from
first().
Commit 964b3f2 fixed an issue where dbObject was returning casting helpers for
fields that were not actually DB objects, but had something in $casting config.
However, because dbObject was no longer calling DataObject->castingHelper, this
exposed a bug that the underlying function db($fieldName) was not returning
field specs for the base fields that are created by SS automatically on all
DataObjects (i.e. Created, LastEdited, etc).
This commit fixes the underlying issue that DataObject->db($fieldName) should
return the field specs for *all* DB fields like its documentation says it will,
including those base fields that are automatically created and do not appear in
$db.
Since ViewableData was returning a casting helper for Link, but DataObject was
only using $this->$fieldname to set values on that casting helper, you could
not use <% if Link %> (or <% if $Link %>) in your templates because Link is not
a field, and thus had no value to be set on the casting helper, causing
hasValue to think that there was no value. Since DataObject->dbObject says that
"it only matches fields and not methods", it seems safe to have it call db(..)
to get the field spec, and not call ViewableData->castingHelper at all.
SQLQuery->setLimit(0, 99) should result in "SELECT ... LIMIT 0 OFFSET 1".
In fact it does "SELECT ..." without a LIMIT clause at all,
which is unexpected. This is regardless of the $offset value.
In large sites this can take a very long time, drastically slowing down the CMS
admin. Even though the versions will then need to be queried individually,
this is still significantly faster than loading hundreds of thousands of
version numbers in one query and populating the cache array.
This caused problems when duplicate() was used in the CMS UI
to duplicate a SiteTree object. Since every object of this type
has a ParentID relation, it copied this empty relation into
new "ghost page".
See https://github.com/silverstripe/silverstripe-cms/issues/689
This references silverstripe/silverstripe-translatable#113
For that issue, we needed to have the DataQuery as the second parameter to
DataQuery's augmentSQL call. Fortunately, DataQuery was already passing this
argument. However, where the function was defined in DataExtension, the
argument was not present. Thus, subclasses of DataExtension could not add the
parameter to their function signature if they were running in PHP strict mode
because PHP will complain that the signatures don't match.
This resulted in Object extensions not working for it, and methods not existing
where they should have. It also resulted in poor error messages appearing when
thrown from Object since $this->class was empty since the constructor was never
called in Object.
Cleanup of framework's use of @package and @subpackage labels and additional of labels for classes missing packages.
Moved all GridField related components to the one name.
Countless spelling fixes, grammar for other comments.
Link ClassName references in file headers.
When DataObject::update() is run with relation fields and the relationship
is new the relationship ID was not set on the DataObject. This patch fixes
this. Fixes issue 6195 in open.silverstripe.org.
With a many to many relation, e.g. SiteTree_MyRelation, and listing
them in your template then adding ?archiveDate=x in the URL, a SQL
error is shown because Versioned::augmentSQL() tries to query the
non-existent table "SiteTree_MyRelation_versions" assuming there's
versioning setup, but there isn't.
$forceWrite was being ignored because it was tested in a part of the
code that is reachable if and only if there are changes to the object.
This patch adds an additional test to correct that logic error.
Also, refrain from needlessly checking for changes when $forceWrite is
true.
Fixes#1687
API: CompositeDBField::setValue() may be passed an object as its second argument, in addition to array.
These changes provide a 15% - 20% performance improvement, and as such justify an small API change in the 3.0 branch. It will likely affect anyone who has created their own composite fields, which is fortunately not all that common.
Rendering potentially 1000s of nodes can exceed the CPU and memory constraints
of a normal PHP process, as well as the rendering capabilities of browsers.
Set a hard maximum for the renderable nodes, deferring to a "show as list" action
in the main CMS tree. For TreeDropdownField, we don't have the list fallback option,
so ask the user to search for the node title instead.
Also makes both the "node_threshold_total" and "node_threshold_leaf" values configurable
This breaks databases like MSSQL which don't allow an ORDER BY with
a subquery at the same time. DELETE queries don't need to be ordered,
so we can safely remove the default.
Otherwise aggregate queries through DataQuery->column() fail unless the
passed in field is specifically quoted already. This fixes ManyManyListTest->testRemoveAll()
This should allow custom meta tags to be saved in the admin panel.
Also, if you use html5 video or audio the source sub tag
would have been stripped two these are all the tags that may be empty
and are valid.
- Renamed $minNodeCount to more accurate $nodeCountThreshold
- The $minNodeCount attribute wasn't properly respected
during actual querying, so SilverStripe would always traverse
the entire tree (and load all objects into memory),
before then marking nodes as "unexpanded", which prevents
them from actually being rendered.
- Fixes nodes on search results to be expanded by default
- Fixes nodes on search results to correctly ajax-expand
This means that you dont have to worry about casting it
as HTMLText again when using the result in a template or other context
However in some situations code might be assuming it can
check with is_string, in which case you now need to use instanceof HTMLText
Extracted common code out to SS_HTMLValue and made abstract, then
put HTML 4 specific code in SS_HTML4Value. Its now possible to
replace HTMLValue with one designed for HTML 5 or XHTML
Requires a code change from new SS_HTMLValue to
Injector::inst()->create(HTMLValue)
This is to fix a bug that caused CheckboxSetFields to throw an error
when trying to call this function when editing a new DataObject. This
occurred when using the advancedworkflow module.
Thanks to simonwelsh for the majority of the work on this fix.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
Same fix as be97535b for 3.1. Makes the method signature
more consistent with other DataExtension methods,
and comply with its subclass implementation in
Hierarchy->validate(). See accbd7f1e2 for more comments.
Before this was only possible for some specific ones, like onBeforeWrite.
This excludes any callbacks with augment*() or update*() naming,
since these are assumed to be on extension only, with a corresponding
base method available on the class itself (e.g. "updateCMSFields()"
vs "getCMSFields()").
This allows a developer to programatically access the size of the DB Varchar field. This allows us to be a bit more DRY and to define the size in one place and limit TextFields to the same value
This reverts commit 1960df8bc3.
Revert "FIX: validate doesn't take var by reference"
This reverts commit 866bb0713b.
@ajshort has changed the method signatures in 1f6f7f08. While it wasn't explicitly noted in the commit message, I think its a good change - objects like a FieldList are always passed by reference in PHP, no need to declare that behaviour.
PHP is throwing strict error warnings when overriding the
updateCMSFields and other functions in custom DataExtensions due to
the fact that the abstract class doesn't declare the variables should
be passed by reference
small change in DataList::filterByCallback. new ArrayList; changed to new ArrayList(); and added @return as this is particular important for this method (unexpected return type).
