- Document the format for descriptor arrays
- Implement the behaviour that developers have come to expect for
string descriptors of indexes
- Add test for handling of unique indexes (MySQL & sqlite3)
- Resolve#2403
Versioned needs to convert unique indexes to non-unique for its suffixed
tables, such as Foo_Live and Foo_versions. Because DataObject accepts
string descriptors such as array('UniqIDX' => 'unique (Uniq)') as well
as array-based descriptors, Versioned needs to recognize string
descriptors. This patch accomplishes that. Before, Versioned would fail
to convert string-described indexes to non-unique, resulting in run-time
errors when creating a new version of an object.
This ensures that the correct stage is selected, even if the request
does not come through the model as controller system. This fixes an
issue where custom controllers would always be on the "Stage" stage.
If more than two $from were added through SQLQuery->addFrom(),
the getOrderedJoins() comparison kicks in. It assumes all $from
parts are in array notation, which isn't always the case.
For legacy reasons, and because we don't have full API support,
you can still add literal joins through addFrom('INNER JOIN ...').
On PHP 5.3, the ordering comparison still works because it
allows array access in strings, with string rather than numeric indexes.
Thankfully that's no longer supported in PHP 5.4.
DataQuery::initialiseQuery() will add a default sort to a query,
and when calling up an aggregate it will make a query like this
which doesn't make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID"
In this case there is no need to add the ORDER BY, and it will
break databases like MSSQL in cases such as
GenericTemplateGlobalProvider
which provides a default List() function for adding aggregates
into SSViewer template cacheblocks.
If we add a limit, however, then it does make sense:
SELECT MAX("LastEdited") FROM "Member" ORDER BY "ID" LIMIT 10
This fixes SQLQuery::aggregate() to NOT add an ORDER BY to an
aggregate call if there is no limit.
If multiple image manipulations are performend the resulting cached image is stored in assets/_resampled because the cached version of the image has no ParentID, which cacheFilename needs to set the correct path.
Not a security issue as such, since the user input is sanitized
before being used in Versioned->augmentSQL(). But it shouldn't
reach the session state either, since that's commonly assumed
to be sanitized data, and it leaves unnecessary room for error.
strtotime() has fairly loose validation rules around dates,
but its a good "first line of defence".
Doesn't have much effect in practice, because charset and collation
are already hardcoded on an ALTER TABLE level (field definitions),
which take priority. Since most MySQL installs will still default
to a latin1 encoding, this propagates to the table though,
confusing devs and in some cases causing wrong data.
Example: A MSSQL->MySQL DB migration tool used the table metadata
to determine the charset, creating encoding issues.
In terms of hardcoding, we don't really support anything other than UTF8,
and the field-level settings are already hardcoded.
We should probably remove the field-specific settings and rely
on the DB defaults, but that's a sensitive API change
(need to set on existing DBs during upgrade).
Allow DataList::limit() to take a null value to remove the limit.
Added tests for limit(). Note the one failure, currently the ORM doesn't support unlimited values with an offset.
The function "first" on ArrayList uses the PHP function "reset", which
returns false if there aren't any elements in the array. Two functions
inside ArrayList use this function, "canFilterBy" and "byID". I've
changed these functions to catch the possibility of a false return from
first().
Commit 964b3f2 fixed an issue where dbObject was returning casting helpers for
fields that were not actually DB objects, but had something in $casting config.
However, because dbObject was no longer calling DataObject->castingHelper, this
exposed a bug that the underlying function db($fieldName) was not returning
field specs for the base fields that are created by SS automatically on all
DataObjects (i.e. Created, LastEdited, etc).
This commit fixes the underlying issue that DataObject->db($fieldName) should
return the field specs for *all* DB fields like its documentation says it will,
including those base fields that are automatically created and do not appear in
$db.
Since ViewableData was returning a casting helper for Link, but DataObject was
only using $this->$fieldname to set values on that casting helper, you could
not use <% if Link %> (or <% if $Link %>) in your templates because Link is not
a field, and thus had no value to be set on the casting helper, causing
hasValue to think that there was no value. Since DataObject->dbObject says that
"it only matches fields and not methods", it seems safe to have it call db(..)
to get the field spec, and not call ViewableData->castingHelper at all.
SQLQuery->setLimit(0, 99) should result in "SELECT ... LIMIT 0 OFFSET 1".
In fact it does "SELECT ..." without a LIMIT clause at all,
which is unexpected. This is regardless of the $offset value.
In large sites this can take a very long time, drastically slowing down the CMS
admin. Even though the versions will then need to be queried individually,
this is still significantly faster than loading hundreds of thousands of
version numbers in one query and populating the cache array.
This caused problems when duplicate() was used in the CMS UI
to duplicate a SiteTree object. Since every object of this type
has a ParentID relation, it copied this empty relation into
new "ghost page".
See https://github.com/silverstripe/silverstripe-cms/issues/689
This references silverstripe/silverstripe-translatable#113
For that issue, we needed to have the DataQuery as the second parameter to
DataQuery's augmentSQL call. Fortunately, DataQuery was already passing this
argument. However, where the function was defined in DataExtension, the
argument was not present. Thus, subclasses of DataExtension could not add the
parameter to their function signature if they were running in PHP strict mode
because PHP will complain that the signatures don't match.
This resulted in Object extensions not working for it, and methods not existing
where they should have. It also resulted in poor error messages appearing when
thrown from Object since $this->class was empty since the constructor was never
called in Object.
Cleanup of framework's use of @package and @subpackage labels and additional of labels for classes missing packages.
Moved all GridField related components to the one name.
Countless spelling fixes, grammar for other comments.
Link ClassName references in file headers.
When DataObject::update() is run with relation fields and the relationship
is new the relationship ID was not set on the DataObject. This patch fixes
this. Fixes issue 6195 in open.silverstripe.org.
With a many to many relation, e.g. SiteTree_MyRelation, and listing
them in your template then adding ?archiveDate=x in the URL, a SQL
error is shown because Versioned::augmentSQL() tries to query the
non-existent table "SiteTree_MyRelation_versions" assuming there's
versioning setup, but there isn't.
$forceWrite was being ignored because it was tested in a part of the
code that is reachable if and only if there are changes to the object.
This patch adds an additional test to correct that logic error.
Also, refrain from needlessly checking for changes when $forceWrite is
true.
Fixes#1687
API: CompositeDBField::setValue() may be passed an object as its second argument, in addition to array.
These changes provide a 15% - 20% performance improvement, and as such justify an small API change in the 3.0 branch. It will likely affect anyone who has created their own composite fields, which is fortunately not all that common.
Rendering potentially 1000s of nodes can exceed the CPU and memory constraints
of a normal PHP process, as well as the rendering capabilities of browsers.
Set a hard maximum for the renderable nodes, deferring to a "show as list" action
in the main CMS tree. For TreeDropdownField, we don't have the list fallback option,
so ask the user to search for the node title instead.
Also makes both the "node_threshold_total" and "node_threshold_leaf" values configurable
This breaks databases like MSSQL which don't allow an ORDER BY with
a subquery at the same time. DELETE queries don't need to be ordered,
so we can safely remove the default.
Otherwise aggregate queries through DataQuery->column() fail unless the
passed in field is specifically quoted already. This fixes ManyManyListTest->testRemoveAll()
This should allow custom meta tags to be saved in the admin panel.
Also, if you use html5 video or audio the source sub tag
would have been stripped two these are all the tags that may be empty
and are valid.
- Renamed $minNodeCount to more accurate $nodeCountThreshold
- The $minNodeCount attribute wasn't properly respected
during actual querying, so SilverStripe would always traverse
the entire tree (and load all objects into memory),
before then marking nodes as "unexpanded", which prevents
them from actually being rendered.
- Fixes nodes on search results to be expanded by default
- Fixes nodes on search results to correctly ajax-expand
This means that you dont have to worry about casting it
as HTMLText again when using the result in a template or other context
However in some situations code might be assuming it can
check with is_string, in which case you now need to use instanceof HTMLText
Extracted common code out to SS_HTMLValue and made abstract, then
put HTML 4 specific code in SS_HTML4Value. Its now possible to
replace HTMLValue with one designed for HTML 5 or XHTML
Requires a code change from new SS_HTMLValue to
Injector::inst()->create(HTMLValue)
This is to fix a bug that caused CheckboxSetFields to throw an error
when trying to call this function when editing a new DataObject. This
occurred when using the advancedworkflow module.
Thanks to simonwelsh for the majority of the work on this fix.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
Lazy loading no longer loads fields from the versions table when querying. This could lead to incorrect data being displayed if the data on the object and the version it pointed to did not match.
API methods to allow setting of the context of the query that generated the DataObject on that object (used by the lazy loading mechanism to correctly query the Stage, Live, or Versions tables)
See https://github.com/silverstripe/sapphire/pull/1178 for context.
Same fix as be97535b for 3.1. Makes the method signature
more consistent with other DataExtension methods,
and comply with its subclass implementation in
Hierarchy->validate(). See accbd7f1e2 for more comments.
Before this was only possible for some specific ones, like onBeforeWrite.
This excludes any callbacks with augment*() or update*() naming,
since these are assumed to be on extension only, with a corresponding
base method available on the class itself (e.g. "updateCMSFields()"
vs "getCMSFields()").
This allows a developer to programatically access the size of the DB Varchar field. This allows us to be a bit more DRY and to define the size in one place and limit TextFields to the same value
This reverts commit 1960df8bc3.
Revert "FIX: validate doesn't take var by reference"
This reverts commit 866bb0713b.
@ajshort has changed the method signatures in 1f6f7f08. While it wasn't explicitly noted in the commit message, I think its a good change - objects like a FieldList are always passed by reference in PHP, no need to declare that behaviour.
PHP is throwing strict error warnings when overriding the
updateCMSFields and other functions in custom DataExtensions due to
the fact that the abstract class doesn't declare the variables should
be passed by reference
small change in DataList::filterByCallback. new ArrayList; changed to new ArrayList(); and added @return as this is particular important for this method (unexpected return type).
Needed for introspection for many_many relationships
without knowing the name of the relationship,
meaning we can't use DataObject->many_many_extraFields().
Does not actually change behaviour, but ensures that the hyphen
is not interpreted as a range identifier should it be placed
between two characters which PCRE regards as "rangeable".
This causes a 'Fatal error: Call to a member function hasMethod() on a non-object'.
This can happen when displaying a field in a gridfield on a belongs_to relationship.
+ has a special meaning in the URLs so overall it's a good idea to
strip them out. Otherwise they would need to appear in their ugly url
encoded form "%2B".
Refer: http://open.silverstripe.org/ticket/7929
BUG Issue with deleted records not being queried properly.
API DataQuery::expressionForField no longer requires a second parameter. Rather the query object is inferred from the DataQuery itself. This should improve consistency of use of this function.
In 3.0 there was some confusion about whether DataLists and ArrayLists
were mutable or not. If DataLists were immutable, they'd return the result, and your code
would look like
$list = $list->filter(....);
If DataLists were mutable, they'd operate on themselves, returning nothing, and your code
would look like
$list->filter(....);
This makes all DataLists and ArrayList immutable for all _searching_ operations.
Operations on DataList that modify the underlying SQL data store remain mutating.
- These functions no longer mutate the existing object, and if you do not capture the value
returned by them will have no effect:
ArrayList#reverse
ArrayList#sort
ArrayList#filter
ArrayList#exclude
DataList#dataQuery (use DataList#alterDataQuery to modify dataQuery in a safe manner)
DataList#where
DataList#limit
DataList#sort
DataList#addFilter
DataList#applyFilterContext
DataList#innerJoin
DataList#leftJoin
DataList#find
DataList#byIDs
DataList#reverse
- DataList#setDataQueryParam has been added as syntactic sugar around the most common
cause of accessing the dataQuery directly - setting query parameters
- RelationList#setForeignID has been removed. Always use RelationList#forForeignID
when querying, and overload RelationList#foreignIDList when subclassing.
- Relatedly,the protected variable RelationList->foreignID has been removed, as the ID is
now stored on a query parameter. Use RelationList#getForeignID to read it.
Session is not initialized by the time we need to use
the setting in DB::connect(). Cookie values get initialized
automatically for each request.
Tightened name format validation to ensure it can only
be used for temporary databases, rather than switching
the browser session to a different production database.
Encrypting token for secure cookie usage.
Added dev/generatesecuretoken to generate this token.
Not storing in YML config directly because of web access issues.
This bug will surface when using the ORM and adding an join to DataList
where a DataObject inherits another DataObject.
If you for example want to restrict the number of pages that only have a
related Staff object:
$list = DataList::create('Page')
->InnerJoin('Staff', '"Staff"."ID" = "Page"."StaffID");
This will create a SQL query where the INNER JOIN is before the
LEFT JOIN of Page and SiteTree in the resulting SQL string. In MySQL
and PostgreSQL this will create an invalid query.
This patch solves the problem by sorting the joins.
Avoid PHPUnit throwing "test didn't run any assertions"
notices in PHP. If nothing else, it keeps test output
looking less broken by default, making it more likely
that actual errors do get noticed.
The original intention of this method was to be a rewrite
check by the installer, but it doesn't belong in
DatabaseAdmin, nor is it used by the installer anymore,
since the rewrite test is done by the installer directly.
Quoted table / column names to make test cases work in postgres
BUG Fixed issue with SQLQuery::lastRow crashing on empty set. Added test cases for lastRow and firstRow.
Quoted table / column names to make test cases work in postgres
Merge branch '3.0-sqlquery-lastrow-fix' of github.com:tractorcow/sapphire into 3.0-sqlquery-lastrow-fix
In a usual CMS request, DataObject::db() is called potentially
thousands of times, calling Config::get() constantly for the same
uninherited statics, which is slow. This improves performance
by caching those into DataObject::$_cache_db
On sites with lots of modules, and pages with plenty of database
queries, DataObject::custom_database_fields() can be called
thousands of times, and slow down page render times. This fixes
it so the fields are cached by class in a static variable, and
are cleared when reset() is called on the DataObject.
SS_HTMLValue::getContent() uses urldecode() on the content returned
by saveHTML() -- this was done to fix encoded HTML entities like
square brackets, which are used by shortcodes. Unfortunately, this
also removes valid characters like "+" from the content.
This fixes it so square bracket entities are decoded *only*, and
leaves everything else as-is.
The specific situation where this is useful is where populateDefaults on
DataObjects needs to query the database. This will break the dev/build
when it tries to create the object via singleton - the query will not be
able to be executed if the table is not there or its schema has changed.
For an example of such use case see Translatable::populateDefaults.
If formatOutput is set to TRUE, then the regexes in getContent()
will not match the newlines, and the output will include html, body
and meta tags. Introduce a few new tests to ensure the output is
correct, and fix the regex.
Setting session directly through $_SESSION relies on
session_autostart which might not be set on every environment,
and isn't consistent with other framework use.
In cases where a getter on a DataObject calls getComponent() or
other relational getter, $this->model won't have been set at
this point, and a fatal error is triggered.
This fixes it so $this->model is set *before* populateDefaults()
in DataObject::__construct() and the getters can operate normally.
The entire framework repo (with the exception of system-generated files) has been amended to respect the 120c line-length limit. This is in preparation for the enforcement of this rule with PHP_CodeSniffer.
Adds three extra methods to Data/SQLQuery query that allow for starting
a disjunctive subgroup, a conjunctive subgroup and for ending a subgroup.
Database::sqlWhereToString() now builds up the WHERE clause one by one
instead of with a straight implode. Uses a stack to know which conenctive
to use.
The issue was raised in #7628, where an anchor tag was being changed from
<a name="anchor"></a> to <a name="anchor"/> by SS_HTMLValue, when
HtmlEditorField::saveInto() parses the HTML fragments.
This is because SS_HTMLValue uses DOMDocument::saveXML(), which is fine
for saving an XML document, but not suitable for HTML. This fix changes
that to use DOMDocument::saveHTML() instead.
Note that we can't use the parameter to saveHTML() for selecting a single
node only, as that's only supported in PHP 5.3.6+, SilverStripe 3.0 supports
PHP 5.3.2 as a minimum. The workaround for this shortcoming is to replace
unncessary output by DOMDocument with a regular expression.
ADDED: Test cases correctly checking for changes (and no changes) to the data model for both fields and indexes.
FIXED: References to indexes throughough the code that probably should have quoted field names. This prevents a lot of 'spam' during dev build. This includes an updated FulltextSearchable test case.
We know the subclass of a record by its ClassName value, but code changes
might have meant that class no longer exists. We used to just break,
but this patch overrides the apparent value of ClassName to be
one that exists in that situation
The table name in the join was being escaped, though table
names aren't escaped anywhere else. This breaks
namespaced classes, which rely on unescaped backslashes.
Config system used to provide an add_static_source method, which was intended for
use by Extensions to add statics. But extensions for a class arent initialised
until at least one instance of that class is created, so before that the
Config system didnt include values from extensions
This patch reverses the control flow, so that the Config system explictly asks
each Object for its additional config sources via the new method
get_extra_config_sources. This method returns an array that can contain
string names of classes and also raw associative arrays.
The developer visible change is that Extension#add_to_class has been
deprecated. Instead there is a new method, get_extra_config, which has
the same method signature but needs to guarantee that it doesnt
cause side effects. Additionally there is no need to call
parent::get_extra_config - this is handled automatically.
Hierarchy#liveChildren was generating a list of all IDs of all pages
on staging. When a site had lots of pages, this basically killed the
tree.
Fix by adding new versioned mode, stage_unique, which uses a
subselect to only return items from a stage that are in no
other stage.
The augmentSQL DataExtension method is always extended on the base data
class of data objects in DataQuery::getFinilisedQuery(). This results
in augmentSQL not being called for extensions that are applied to non-
base data classes when finalizing the query.
For example, if Versioned was applied to class B which extends class A,
which in turn extends DataObject, then augmentSQL would be extended for
class A in DataQuery::getFinilisedQuery(). Since class A doesn't have
the Versioned extension in this example, it would not work for class B.
Fixed this by extending augmentSQL on the actual data class and not
on the base class.
DataList had several methods that should act on a copy and return
that copy, but was instead mutating the existing list.
We cant change this behaviour in the 3.0 line for backwards compt.
reasons, but this makes the desired behavior the default, and
makes disabling the mutation in 3.1 easier
It also introduces two new methods to deal with the common pattern
of wanting to modify the underlying dataQuery, which we want to be
able to reliably do in a way that always acts immutably. The main
method of these two is alterDataQuery
ArrayList had several methods that should act on a copy and return
that copy, but was instead mutating the existing list.
We cant change this behaviour in the 3.0 line for backwards compt.
reasons, but this makes the desired behavior the default, and
makes disabling the mutation in 3.1 easier
When publishing to live, DataObject#forceChange is called, which wasnt correctly loading
in fields that were lazy (unloaded) if those fields were from composite fields
like Money. The end result is that any Money values would be forced to null on
publish to live
Also changes the API of the (internal, protected) loadLazyFields method so that
not passing a class argument just unlazys all lazy fields regardless of source table
In getField we check if the field we are getting is currently lazy (unloaded), and
load it if it is. This was only working for simple fields though - composite
fields like Money werent working
Use third party tools like XHProf instead.
Removed defunct or unnecessary debug GET parameters:
debug_profile, debug_memory, profile_trace, debug_javascript, debug_behaviour
When querying DataObjects by a generic parent class (like SiteTree for instance), fields added via $db
set on child classes wouldnt appear.
This is because Object::__construct wasnt called early enough in DataObject::__construct, so
extensions werent initialised when $db was first accessed
If the applyRelation() was passed a relation that went to a class with a parent
class with a database table, applyRelation would return the name of the parent
class, rather than the class the relation was actually too.
PaginatedList needs to be notified about this, otherwise it will
errorneously try to further limit the already limited set, making the
subsequent pages empty.
Without this bugfix, if you had a Page that used to be a SiteTree, and you tried to use Versiond::get_version() or Versioned::get_latest_version() to return the older SiteTree version, nothing would be returned, because the results were being filtered by ClassName. This caused bugs in the history panel for certain combinbations of page classname alteration.
If the applyRelation() was passed a relation that went to a class with a parent
class with a database table, applyRelation would return the name of the parent
class, rather than the class the relation was actually too.
This bug was caused by the fact that SQLQuery::whereAny() removed existing filters. In line with addWhere() and setWhere(), I split this into addWhereAny() and setWhereAny(). Strictly speaking, this drops the method SQLQuery::whereAny(), but it was really just an internal function for exclude, and so I think that's acceptable.
The intl extension in PHP 5.4 provides a Transliterator class, which
conflicts with the SilverStripe one. This leads to some really weird
ReflectionExceptions about Transliterator's constructor being
private.
DataObject::__construct()
The database adapter uses smallint instead of the boolean datatype,
which works around the issue instead of converting the value.
MySQLDatabase::enumValuesForField() - Added stripslashes because backslashes are escaped in the type description
DataObject::requireTable() - Added double quotes to column names in automatically generated indexes for many_many relationships
DataObject::write() - Escaped class name for DB query
DataQuery::getFinalisedQuery() - Escaped class names for DB query
MINOR Use injector for creating many many list objects
MINOR Use injector for creating objects from within the DataList
MINOR Use Injector::inst() for creating objects; cannot rely on this->injector being present due to many classes being created with 'new', so use inst() directly
MINOR Remove injector autoset property for now; automatically setting it breaks a few test cases that don't know about it for now, and it's not needed just yet
BUGFIX Versioned's constructor doesn't provide suitable defaults. Previously a bug/feature in singleton, where it would pass null,true as params to strong_create, which would then get passed through as params to Versioned's constructor, meant that the code still executed fine (as was set to something that wasn't an array, so the null and true were instead taken as args). The fact that the usage of singleton(Versioned) never really used the classes code, purely for value lookup, meant that this never propagated errors. I've now switched singleton() to use the injector for retrieving values, which means these dud values are no longer passed through
CHANGE Given that Config::inst is an implementation of the singleton pattern itself, I've removed the extra call to singleton(). A side effect of this is that it gets around a possibly nasty circular reference with the dependency injector (which relies on the config object); in future, this dependency structure should really be structured from the DI directly.
MINOR Change singleton and strong_create to use dependency injector
CHANGE Given that Config::inst is an implementation of the singleton pattern itself, I've removed the extra call to singleton(). A side effect of this is that it gets around a possibly nasty circular reference with the dependency injector (which relies on the config object); in future, this dependency structure should really be structured from the DI directly.
MINOR Change singleton and strong_create to use dependency injector
BUGFIX: Provide default constructor values for classes (fixes issues when used in 'singleton' scenario during dev/build in particular)
MINOR Clear out injector state when resetting db schema during tests (a follow on from changing singleton() calls to use the injector underneath)
CHANGE Given that Config::inst is an implementation of the singleton pattern itself, I've removed the extra call to singleton(). A side effect of this is that it gets around a possibly nasty circular reference with the dependency injector (which relies on the config object); in future, this dependency structure should really be structured from the DI directly.
MINOR Change singleton and strong_create to use dependency injector
---
Dont start the session until its actually necessary, which is to say there is a cookie available with the current PHP session name (or a request variable with the session_name() - typically PHPSESSID.) The latter allows for passing session ID through as an alternative to cookies.
---
The primary goal of this branch was to fix the sort bugs in AssetAdmin, however, it started a bit of a yak shave in that the API around SQLQuery was poor. The biggest change that this pull request makes is it changes the format of SQLQuery::$select to contain aliases as array keys (and consistently puts the "implicit alias" in there to assist with various query generation logic), but it also makes a bunch of changes to avoid direct access of that property.
