Commit Graph

26 Commits

Author SHA1 Message Date
Damian Mooyman
22ccf3e2f9
[ss-2017-007] Ensure xls formulae are safely sanitised on output
CSVParser now strips leading tabs on cells
2017-12-01 10:19:48 +13:00
Christopher Pitt
1c5089f7fc Splitting BulkLoader imports to reduce memory consumption 2015-09-15 10:06:06 +12:00
Damian Mooyman
eb069e605d Remove all redundant whitespace 2014-08-19 09:17:15 +12:00
Damian Mooyman
d8e9af8af8 API New Database abstraction layer. Ticket #7429
Database abstraction broken up into controller, connector, query builder, and schema manager, each independently configurable via YAML / Injector
Creation of new DBQueryGenerator for database specific generation of SQL
Support for parameterised queries, move of code base to use these over escaped conditions
Refactor of SQLQuery into separate query classes for each of INSERT UPDATE DELETE and SELECT
Support for PDO
Installation process upgraded to use new ORM
SS_DatabaseException created to handle database errors, maintaining details of raw sql and parameter details for user code designed interested in that data.
Renamed DB static methods to conform correctly to naming conventions (e.g. DB::getConn -> DB::get_conn)
3.2 upgrade docs
Performance Optimisation and simplification of code to use more concise API
API Ability for database adapters to register extensions to ConfigureFromEnv.php
2014-07-09 18:04:05 +12:00
Will Rossiter
c8af0fd7d1 FIX: If CSV column mapping maps to function, keep key value as key.
Fixes http://open.silverstripe.org/ticket/6473

When using CSVParser::$columnMapping to map columns to a callback action, it previously used the action name as the key value. This prevented users from defining multiple entries to the same callback. This patch retains those key values and simply runs the callback field name filter later on.
2013-05-14 22:00:52 +12:00
Jeremy Shipman
d47b202697 Restored c4eac5310e (merge error)
FIX: Instead of CsvBulkLoader->findExistingRecord out right failing (i.e. no duplicate found) when the duplicate check field is empty, it will now continue on to check other duplicateCheck fields.
Added extra testing data to CSVBulkLoaderTest so that it fails.
2013-05-09 23:06:02 +02:00
Fred Condo
bd5c8520bb FIX: Use the correct variable as the key into $record
It was using $fieldName, which is the CSV field name, not the database
field name. This prevents duplicate detection from working. It now
properly uses $SQL_fieldName.

Update CsvBulkLoaderTest to remove keys that are nonexistent in the CSV
test data. Having them causes the test to fail with an undefined-index
error. This did not previously fail because of the bug in CsvBulkLoader
that this patch fixes. This partially reverts c4eac53.
2013-05-01 06:50:40 -07:00
Jeremy Shipman
c4eac5310e FIX: Instead of CsvBulkLoader->findExistingRecord out right failing (i.e. no duplicate found) when the duplicate check field is empty, it will now continue on to check other duplicateCheck fields.
Added extra testing data to CSVBulkLoaderTest so that it fails.
2013-04-19 16:52:39 +12:00
Ingo Schommer
3334eafcb1 API Marked statics private, use Config API instead (#8317)
See "Static configuration properties are now immutable, you must use Config API." in the 3.1 change log for details.
2013-03-24 17:20:53 +01:00
Simon Welsh
b0121b541c Add codesniffer that ensures indentation is with tabs. 2012-12-12 17:33:31 +13:00
Sam Minnee
1f7fc1f76a FIX Remove instances of lines longer than 120c
The entire framework repo (with the exception of system-generated files) has been amended to respect the 120c line-length limit.  This is in preparation for the enforcement of this rule with PHP_CodeSniffer.
2012-09-30 17:18:13 +13:00
Ingo Schommer
e2f073f38a Method visibility according to coding conventions 2012-09-20 10:46:59 +02:00
Simon Welsh
3a6341a251 API-CHANGE sapphire folder can now be renamed. 2012-04-15 10:50:19 +12:00
Andrew O'Neil
de2832e65f ENHANCEMENT: Allow Object::create() to be called with late static binding.
This allows DataList::create('SiteTree') as equivalent to Object::create('DataList', 'SiteTree'), without
having to have a create() function on DataList. Required for E_STRICT compliance.
2012-03-27 17:57:42 +13:00
Fred Condo
d370423825 Clean up trailing ?> per coding standard
All sapphire but the lang directory
2012-02-12 12:40:16 -08:00
Ingo Schommer
2870eb9212 MINOR Changed $fixture_path to relative filenames in all sapphire test cases in order to allow easier file moving and less verbosity in tests 2011-03-30 23:04:48 +13:00
Ingo Schommer
e921b376bc API CHANGE: Don't generate TestOnly DataObjects in the database immediately; instead let test developers specify them in SapphireTest::$extraDataObjects.
API CHANGE: Added SapphireTest::resetDBSchema() (from r90054) (from r96734)

git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@102356 467b73ca-7a2a-4603-9d3b-597d59a354a9
2010-04-12 02:03:16 +00:00
Ingo Schommer
ef28a27b25 ENHANCEMENT Added BulkLoader->deleteExistingRecords(), removed unnecessary parameters from BulkLoader->load()
ENHANCEMENT Decreased memory usage in BulkLoader->load() when deleting all records before importing
MINOR Re-enabled CsvBulkloaderTest cases, were disabled by accident

git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@94250 467b73ca-7a2a-4603-9d3b-597d59a354a9
2009-12-02 09:40:28 +00:00
Ingo Schommer
aa127e52c9 BUGFIX Fixed CSVBulkLoaderTest not to assume ID ordering in the assertions, which breaks with databases not ordering by PK automatically (e.g. Postgres)
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@88003 467b73ca-7a2a-4603-9d3b-597d59a354a9
2009-10-02 00:07:36 +00:00
Geoff Munn
34db005f65 MINOR FIX: columns quoted properly
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@86502 467b73ca-7a2a-4603-9d3b-597d59a354a9
2009-09-17 00:02:29 +00:00
Julian Seidenberg
2cf002e333 ENHANCEMENT: added option to truncate (clear) database table before importing a new CSV file with CSVBulkerLoader and ModelAdmin.
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@85709 467b73ca-7a2a-4603-9d3b-597d59a354a9
2009-09-04 00:31:08 +00:00
Andrew O'Neil
60f75c5ca4 Merged changes from 2.3 branch
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@71172 467b73ca-7a2a-4603-9d3b-597d59a354a9
2009-02-01 23:49:53 +00:00
Sam Minnee
96c5be8252 Updating queries to be more DB agnostic
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@66507 467b73ca-7a2a-4603-9d3b-597d59a354a9
2008-11-24 09:31:14 +00:00
Sam Minnee
3d9532db83 Merged branches/2.3
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@66108 467b73ca-7a2a-4603-9d3b-597d59a354a9
2008-11-18 01:48:37 +00:00
Ingo Schommer
c0c4ff8a9d BUGFIX Fixed CsvBulkLoaderTest to comply to hasHeaderRow API change (r64806)
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@65026 467b73ca-7a2a-4603-9d3b-597d59a354a9
2008-11-01 13:47:01 +00:00
Sam Minnee
cbc228f9a2 API CHANGE: Created CSVParser class and updated CSVBulkLoader to use it
git-svn-id: svn://svn.silverstripe.com/silverstripe/open/modules/sapphire/trunk@63563 467b73ca-7a2a-4603-9d3b-597d59a354a9
2008-10-03 00:27:58 +00:00