Merge pull request #2406 from dangerdan/testing

Resubmitting pull request, changes to docs: topics/testing

docs/en/topics/testing/create-silverstripe-test.md

@@ -1,69 +0,0 @@
-#  How To Create a SilverStripe Test
-
-A unit test class will test the behaviour of one of your `[api:DataObjects]`.  This simple fragment of `[api:SiteTreeTest]`
-provides us the basics of creating unit tests.
-
-	:::php
-	<?php
-	class SiteTreeTest extends SapphireTest {
-		
-		// Define the fixture file to use for this test class
-		private static $fixture_file = 'SiteTreeTest.yml';
-	
-		/**	
-		 * Test generation of the URLSegment values.
-		 *  - Turns things into lowercase-hyphen-format
-		 *  - Generates from Title by default, unless URLSegment is explicitly set
-		 *  - Resolves duplicates by appending a number
-		 */
-		public function testURLGeneration() {
-			$expectedURLs = array(
-				'home' => 'home',
-				'staff' => 'my-staff',
-				'about' => 'about-us',
-				'staffduplicate' => 'my-staff-2',
-				'product1' => '1-1-test-product',
-				'product2' => 'another-product',
-				'product3' => 'another-product-2',
-				'product4' => 'another-product-3',
-			);
-			
-			foreach($expectedURLs as $fixture => $urlSegment) {
-				$obj = $this->objFromFixture('Page', $fixture);
-				$this->assertEquals($urlSegment, $obj->URLSegment);
-			}
-		}
-	}
-	
-
-
-There are a number of points to note in this code fragment:
-
-*  Your test is a **subclass of SapphireTest**.  Both unit tests and functional tests are a subclass of `[api:SapphireTest]`.
-*  **static $fixture_file** is defined.  The testing framework will automatically set up a new database for **each** of
-your tests.  The initial database content will be sourced from the YML file that you list in $fixture_file. The property can take an array of fixture paths.
-*  Each **method that starts with the word "test"** will be executed by the TestRunner.  Define as many as you like; the
-database will be rebuilt for each of these.
-*  **$this->objFromFixture($className, $identifier)** can be used to select one of the objects named in your fixture
-file.  To identify to the object, we provide a class name and an identifier.  The identifier is specified in the YML
-file but not saved in the database anywhere.  objFromFixture() looks the `[api:DataObject]` up in memory rather than using the
-database.  This means that you can use it to test the functions responsible for looking up content in the database.
-
-## Assertion commands
-
-**$this->assertEquals()** is an example of an assertion function.  
-These functions form the basis of our tests - a test
-fails if and only if one or more of the assertions fail.  
-See [the PHPUnit manual](http://www.phpunit.de/manual/current/en/api.html#api.assert)
-for a listing of all PHPUnit's built-in assertions.
-
-The `[api:SapphireTest]` class comes with additional assertions which are more
-specific to the framework, e.g. `[assertEmailSent](api:SapphireTest->assertEmailSent())`
-which can simulate sending emails through the `Email->send()` API without actually
-using a mail server (see the [testing emails](email-sending)) guide.
-
-## Fixtures
-
-Often you need to test your functionality with some existing data, so called "fixtures".
-These records are inserted on a fresh test database automatically. 
-[Read more about fixtures](fixtures).

docs/en/topics/testing/creating-a-functional-test.md

@@ -1,4 +1,4 @@
-# Writing functional tests
+# Creating a functional tests
 
 Functional tests test your controllers.  The core of these are the same as unit tests:
 
@@ -13,7 +13,7 @@ URLs.  Here is an example from the subsites module:
 	:::php
 	class SubsiteAdminTest extends SapphireTest {
 		private static $fixture_file = 'subsites/tests/SubsiteTest.yml';
-	
+
 		/**
 		 * Return a session that has a user logged in as an administrator
 		 */
@@ -22,27 +22,27 @@ URLs.  Here is an example from the subsites module:
 				'loggedInAs' => $this->idFromFixture('Member', 'admin')
 			));
 		}
-	
+
 		/**
 		 * Test generation of the view
 		 */
 		public function testBasicView() {
 			// Open the admin area logged in as admin
 			$response1 = Director::test('admin/subsites/', null, $this->adminLoggedInSession());
-	
+
 			// Confirm that this URL gets you the entire page, with the edit form loaded
 			$response2 = Director::test('admin/subsites/show/1', null, $this->adminLoggedInSession());
 			$this->assertTrue(strpos($response2->getBody(), 'id="Root_Configuration"') !== false);
 			$this->assertTrue(strpos($response2->getBody(), '<head') !== false);
-	
+
 			// Confirm that this URL gets you just the form content, with the edit form loaded
 			$response3 = Director::test('admin/subsites/show/1', array('ajax' => 1), $this->adminLoggedInSession());
-	
+
 			$this->assertTrue(strpos($response3->getBody(), 'id="Root_Configuration"') !== false);
 			$this->assertTrue(strpos($response3->getBody(), '<form') === false);
 			$this->assertTrue(strpos($response3->getBody(), '<head') === false);
 		}
-	
+
 
 
 We are using a new static method here: **Director::test($url, $postVars, $sessionObj)**
@@ -65,4 +65,5 @@ If you're testing for natural language responses like error messages, make sure
 the *_t()* method to avoid tests failing when i18n is enabled.
 
 Note that for a more highlevel testing approach, SilverStripe also supports 
-[behaviour-driven testing through Behat](https://github.com/silverstripe-labs/silverstripe-behat-extension). It interacts directly with your website or CMS interface by remote controlling an actual browser, driven by natural language assertions.
+[behaviour-driven testing through Behat](https://github.com/silverstripe-labs/silverstripe-behat-extension). It interacts
+directly with your website or CMS interface by remote controlling an actual browser, driven by natural language assertions.

docs/en/topics/testing/creating-a-silverstripe-test.md

@@ -0,0 +1,66 @@
+# Creating a SilverStripe Test
+
+A test is created by extending one of two classes, SapphireTest and FunctionalTest. You would subclass SapphireTest to
+test your application logic, for example testing the behaviour of one of your `[api:DataObjects]`, whereas FunctionalTest
+is extended when you want to test your application's functionality, such as testing the results of GET and POST requests,
+and validating the content of a page. `[api:FunctionalTest]` is a subclass of `[api:SapphireTest]`.
+
+## Creating a test from SapphireTest
+
+Here is an example of a test which extends SapphireTest:
+
+	:::php
+	<?php
+	class SiteTreeTest extends SapphireTest {
+
+		// Define the fixture file to use for this test class
+		private static $fixture_file = 'SiteTreeTest.yml';
+
+		/**
+		 * Test generation of the URLSegment values.
+		 *  - Turns things into lowercase-hyphen-format
+		 *  - Generates from Title by default, unless URLSegment is explicitly set
+		 *  - Resolves duplicates by appending a number
+		 */
+		public function testURLGeneration() {
+			$expectedURLs = array(
+				'home' => 'home',
+				'staff' => 'my-staff',
+				'about' => 'about-us',
+				'staffduplicate' => 'my-staff-2',
+				'product1' => '1-1-test-product',
+				'product2' => 'another-product',
+				'product3' => 'another-product-2',
+				'product4' => 'another-product-3',
+			);
+
+			foreach($expectedURLs as $fixture => $urlSegment) {
+				$obj = $this->objFromFixture('Page', $fixture);
+				$this->assertEquals($urlSegment, $obj->URLSegment);
+			}
+		}
+	}
+
+Firstly we define a static member `$fixture_file`, this should point to a file that represents the data we want to test,
+represented in YAML. When our test is run, the data from this file will be loaded into a test database for our test to use.
+This property can be an array of strings pointing to many .yml files, but for our test we are just using a string on its
+own. For more detail on fixtures, see the [page on fixtures](fixtures).
+
+The second part of our class is the `testURLGeneration` method. This method is our test. You can asign many tests, but
+again for our purposes there is just the one. When the test is executed, methods prefixed with the word **test** will be
+run. The test database is rebuilt everytime one of these methods is run.
+
+Inside our test method is the `objFromFixture` method that will generate an object for us based on data from our fixture
+file. To identify to the object, we provide a class name and an identifier. The identifier is specified in the YAML file
+but not saved in the database anywhere, `objFromFixture` looks the `[api:DataObject]` up in memory rather than using the
+database. This means that you can use it to test the functions responsible for looking up content in the database.
+
+The final part of our test is an assertion command, `assertEquals`. An assertion command allows us to test for something
+in our test methods (in this case we are testing if two values are equal). A test method can have more than one assertion
+command, and if anyone of these tests fail, then the whole test method will fail.
+
+For more information on PHPUnit's assertions see the [PHPUnit manual](http://www.phpunit.de/manual/current/en/api.html#api.assert).
+
+The `[api:SapphireTest]` class comes with additional assertions which are more specific to the Sapphire, for example the
+`[assertEmailSent](api:SapphireTest->assertEmailSent())` method, which simulates sending emails through the `Email->send()`
+API without actually using a mail server. For more details on this see th [testing emails](testing-email)) guide.

docs/en/topics/testing/glossary.md

@@ -0,0 +1,46 @@
+# Glossary
+
+**Assertion:** A predicate statement that must be true when a test runs.
+
+**Behat:** A behaviour-driven testing library used with SilverStripe as a higher-level
+alternative to the `FunctionalTest` API, see [http://behat.org](http://behat.org).
+
+**Test Case:** The atomic class type in most unit test frameworks. New unit tests are created by inheriting from the
+base test case.
+
+**Test Suite:** Also known as a 'test group', a composite of test cases, used to collect individual unit tests into
+packages, allowing all tests to be run at once.
+
+**Fixture:** Usually refers to the runtime context of a unit test - the environment and data prerequisites that must be
+in place in order to run the test and expect a particular outcome. Most unit test frameworks provide methods that can be
+used to create fixtures for the duration of a test - `setUp` - and clean them up after the test is done - `tearDown'.
+
+**Refactoring:** A behavior preserving transformation of code. If you change the code, while keeping the actual
+functionality the same, it is refactoring. If you change the behavior or add new functionality it's not.
+
+**Smell:** A code smell is a symptom of a problem. Usually refers to code that is structured in a way that will lead to
+problems with maintenance or understanding.
+
+**Spike:** A limited and throwaway sketch of code or experiment to get a feel for how long it will take to implement a
+certain feature, or a possible direction for how that feature might work.
+
+**Test Double:** Also known as a 'Substitute'. A general term for a dummy object that replaces a real object with the
+same interface. Substituting objects is useful when a real object is difficult or impossible to incorporate into a unit
+test.
+
+**Fake Object**: A substitute object that simply replaces a real object with the same interface, and returns a
+pre-determined (usually fixed) value from each method.
+
+**Mock Object:** A substitute object that mimicks the same behavior as a real object (some people think of mocks as
+"crash test dummy" objects). Mocks differ from other kinds of substitute objects in that they must understand the
+context of each call to them, setting expectations of which, and what order, methods will be invoked and what parameters
+will be passed.
+
+**Test-Driven Development (TDD):** A style of programming where tests for a new feature are constructed before any code
+is written. Code to implement the feature is then written with the aim of making the tests pass. Testing is used to
+understand the problem space and discover suitable APIs for performing specific actions.
+
+**Behavior Driven Development (BDD):** An extension of the test-driven programming style, where tests are used primarily
+for describing the specification of how code should perform. In practice, there's little or no technical difference - it
+all comes down to language. In BDD, the usual terminology is changed to reflect this change of focus, so *Specification*
+is used in place of *Test Case*, and *should* is used in place of *expect* and *assert*.

docs/en/topics/testing/index.md

@@ -2,105 +2,65 @@
 
 The SilverStripe core contains various features designed to simplify the process of creating and managing automated tests.
 
-* [Create a unit test](create-silverstripe-test): Writing tests to check core data objects
-* [Creating a functional test](create-functional-test): An overview of functional tests and how to write a functional test
-* [Email Sending](email-sending): An overview of the built-in email testing code
-* [Troubleshooting](testing-guide-troubleshooting): Frequently asked questions list for testing issues
-* [Why Unit Test?](why-test): Why should you test and how to start testing
-
 If you are familiar with PHP coding but new to unit testing, you should read the [Introduction](/topics/testing) and
 check out Mark's presentation [Getting to Grips with SilverStripe Testing](http://www.slideshare.net/maetl/getting-to-grips-with-silverstripe-testing).
+This section's page [Why Unit Test?](why-should-i-test) will give you the reasons behind why you should be testing your
+code.
 
 You should also read over [the PHPUnit manual](http://www.phpunit.de/manual/current/en/). It provides a lot of
 fundamental concepts that we build on in this documentation.
 
 If you're more familiar with unit testing, but want a refresher of some of the concepts and terminology, you can browse
-the [Testing Glossary](#glossary).
+the [Testing Glossary](glossary). To get started now, follow the installation instructions below, and check
-To get started now, follow the installation instructions below, and check
+[Troubleshooting](troubleshooting) in case you run into any problems.
-[Troubleshooting](/topics/testing/testing-guide-troubleshooting) in case you run into any problems.
 
 ## Installation
 
 ### Via Composer
 
-Unit tests are not included in the normal SilverStripe downloads,
+Unit tests are not included in the normal SilverStripe downloads, you are expected to work with local git repositories
-you are expected to work with local git repositories 
 ([installation instructions](/topics/installation/composer)).
 
-Once you've got the project up and running,
+Once you've got the project up and running, check out the additional requirements to run unit tests:
-check out the additional requirements to run unit tests:
 
-	composer update --dev
+    composer update --dev
 
-The will install (among other things) the [PHPUnit](http://www.phpunit.de/) dependency,
+This will install (among other things) the [PHPUnit](http://www.phpunit.de/) dependency, which is the framework we're
-which is the framework we're building our unit tests on.
+building our unit tests on. Composer installs it alongside the required PHP classes into the `vendor/bin/` directory.
-Composer installs it alongside the required PHP classes into the `vendor/bin/` directory.
+You can either use it through its full path (`vendor/bin/phpunit`), or symlink it into the root directory of your website:
-You can either use it through its full path (`vendor/bin/phpunit`), or symlink it
-into the root directory of your website:
 
-	ln -s vendor/bin/phpunit phpunit
+    ln -s vendor/bin/phpunit phpunit
 
 ### Via PEAR
 
-Alternatively, you can check out phpunit globally via the PEAR packanage manager
+Alternatively, you can check out PHPUnit globally via the PEAR packanage manager
 ([instructions](https://github.com/sebastianbergmann/phpunit/)).
 
-	pear config-set auto_discover 1
+    pear config-set auto_discover 1
-	pear install pear.phpunit.de/PHPUnit
+    pear install pear.phpunit.de/PHPUnit
 
-## Running Tests
+## Configuration
 
-### Via the "phpunit" Binary on Command Line
+### phpunit.xml
 
-The `phpunit` binary should be used from the root directory of your website.
+The `phpunit` executable can be configured by commandline arguments or through an XML file. File-based configuration has
+the advantage of enforcing certain rules across test executions (e.g. excluding files from code coverage reports), and
+of course this information can be version controlled and shared with other team members.
 
-	# Runs all tests defined in phpunit.xml
+**Note: This doesn't apply for running tests through the "sake" wrapper**
-	phpunit
-	
-	# Run all tests of a specific module
- 	phpunit framework/tests/
-	
- 	# Run specific tests within a specific module
- 	phpunit framework/tests/filesystem
-	
- 	# Run a specific test
-	phpunit framework/tests/filesystem/FolderTest.php
-	
-	# Run tests with optional `$_GET` parameters (you need an empty second argument)
- 	phpunit framework/tests '' flush=all
 
-All command-line arguments are documented on 
+SilverStripe comes with a default `phpunit.xml.dist` that you can use as a starting point. Copy the file into a new
-[phpunit.de](http://www.phpunit.de/manual/current/en/textui.html).
+`phpunit.xml` and customize to your needs - PHPUnit will auto-detect its existence, and prioritize it over the default
+file.
 
-### Via the "sake" Wrapper on Command Line
+There's nothing stopping you from creating multiple XML files (see the `--configuration` flag in
+[PHPUnit documentation](http://www.phpunit.de/manual/current/en/textui.html)). For example, you could have a
+`phpunit-unit-tests.xml` and `phpunit-functional-tests.xml` file (see below).
 
-The [sake](/topics/commandline) executable that comes with SilverStripe can trigger a customized
+### Database Permissions
-"[api:TestRunner]" class that handles the PHPUnit configuration and output formatting.
-While the custom test runner a handy tool, its also more limited than using `phpunit` directly,
-particularly around formatting test output.
 
-	# Run all tests
+SilverStripe tests create thier own database when they are run. Because of this the database user in your config file
-	sake dev/tests/all
+should have the appropriate permissions to create new databases on your server, otherwise tests will not run.
-	
-	# Run all tests of a specific module (comma-separated)
-	sake dev/tests/module/framework,cms
-	
-	# Run specific tests (comma-separated)
-	sake dev/tests/FolderTest,OtherTest
-	
-	# Run tests with optional `$_GET` parameters
-	sake dev/tests/all flush=all
-	
-	# Skip some tests
-	sake dev/tests/all SkipTests=MySkippedTest
-
-### Via Web Browser
-
-Executing tests from the command line is recommended, since it most closely reflects
-test runs in any automated testing environments. If for some reason you don't have
-access to the command line, you can also run tests through the browser.
-
-	 http://localhost/dev/tests
 
 ## Writing Tests
 
@@ -120,71 +80,60 @@ Some people may note that we have used the same naming convention as Ruby on Rai
 
 Tutorials and recipes for creating tests using the SilverStripe framework:
 
-*  **[Create a SilverStripe Test](/topics/testing/create-silverstripe-test)**
+* [Creating a SilverStripe test](creating-a-silverstripe-test): Writing tests to check core data objects
-*  **[Create a Functional Test](/topics/testing/create-functional-test)**
+* [Creating a functional test](creating-a-functional-test): An overview of functional tests and how to write a functional test
-*  **[Test Outgoing Email Sending](/topics/testing/email-sending)**
+* [Testing Outgoing Email](testing-email): An overview of the built-in email testing code
 
-## Configuration
+## Running Tests
 
-### phpunit.xml
+### Via the "phpunit" Binary on Command Line
 
-The `phpunit` executable can be configured by commandline arguments or through an XML file.
+The `phpunit` binary should be used from the root directory of your website.
-File-based configuration has the advantage of enforcing certain rules across
-test executions (e.g. excluding files from code coverage reports), and of course this
-information can be version controlled and shared with other team members.
 
-**Note: This doesn't apply for running tests through the "sake" wrapper**
+    # Runs all tests defined in phpunit.xml
+    phpunit
 
-SilverStripe comes with a default `phpunit.xml.dist` that you can use as a starting point.
+    # Run all tests of a specific module
-Copy the file into a new `phpunit.xml` and customize to your needs - PHPUnit will auto-detect
+    phpunit framework/tests/
-its existence, and prioritize it over the default file.
 
-There's nothing stopping you from creating multiple XML files (see the `--configuration` flag in [PHPUnit documentation](http://www.phpunit.de/manual/current/en/textui.html)).
+    # Run specific tests within a specific module
-For example, you could have a `phpunit-unit-tests.xml` and `phpunit-functional-tests.xml` file (see below).
+    phpunit framework/tests/filesystem
 
-## Glossary {#glossary}
+    # Run a specific test
+    phpunit framework/tests/filesystem/FolderTest.php
 
-**Assertion:** A predicate statement that must be true when a test runs.
+    # Run tests with optional `$_GET` parameters (you need an empty second argument)
+    phpunit framework/tests '' flush=all
 
-**Behat:** A behaviour-driven testing library used with SilverStripe as a higher-level
+All command-line arguments are documented on
-alternative to the `FunctionalTest` API, see [http://behat.org](http://behat.org).
+[phpunit.de](http://www.phpunit.de/manual/current/en/textui.html).
 
-**Test Case:** The atomic class type in most unit test frameworks. New unit tests are created by inheriting from the
+### Via the "sake" Wrapper on Command Line
-base test case.
 
-**Test Suite:** Also known as a 'test group', a composite of test cases, used to collect individual unit tests into
+The [sake](/topics/commandline) executable that comes with SilverStripe can trigger a customized
-packages, allowing all tests to be run at once.
+"[api:TestRunner]" class that handles the PHPUnit configuration and output formatting.
+While the custom test runner a handy tool, its also more limited than using `phpunit` directly,
+particularly around formatting test output.
 
-**Fixture:** Usually refers to the runtime context of a unit test - the environment and data prerequisites that must be
+    # Run all tests
-in place in order to run the test and expect a particular outcome. Most unit test frameworks provide methods that can be
+    sake dev/tests/all
-used to create fixtures for the duration of a test - `setUp` - and clean them up after the test is done - `tearDown'.
 
-**Refactoring:** A behavior preserving transformation of code. If you change the code, while keeping the actual
+    # Run all tests of a specific module (comma-separated)
-functionality the same, it is refactoring. If you change the behavior or add new functionality it's not.
+    sake dev/tests/module/framework,cms
 
-**Smell:** A code smell is a symptom of a problem. Usually refers to code that is structured in a way that will lead to
+    # Run specific tests (comma-separated)
-problems with maintenance or understanding.
+    sake dev/tests/FolderTest,OtherTest
 
-**Spike:** A limited and throwaway sketch of code or experiment to get a feel for how long it will take to implement a
+    # Run tests with optional `$_GET` parameters
-certain feature, or a possible direction for how that feature might work.
+    sake dev/tests/all flush=all
 
-**Test Double:** Also known as a 'Substitute'. A general term for a dummy object that replaces a real object with the
+    # Skip some tests
-same interface. Substituting objects is useful when a real object is difficult or impossible to incorporate into a unit
+    sake dev/tests/all SkipTests=MySkippedTest
-test.
 
-**Fake Object**: A substitute object that simply replaces a real object with the same interface, and returns a
+### Via Web Browser
-pre-determined (usually fixed) value from each method.
 
-**Mock Object:** A substitute object that mimicks the same behavior as a real object (some people think of mocks as
+Executing tests from the command line is recommended, since it most closely reflects
-"crash test dummy" objects). Mocks differ from other kinds of substitute objects in that they must understand the
+test runs in any automated testing environments. If for some reason you don't have
-context of each call to them, setting expectations of which, and what order, methods will be invoked and what parameters
+access to the command line, you can also run tests through the browser.
-will be passed.
 
-**Test-Driven Development (TDD):** A style of programming where tests for a new feature are constructed before any code
+     http://localhost/dev/tests
-is written. Code to implement the feature is then written with the aim of making the tests pass. Testing is used to
-understand the problem space and discover suitable APIs for performing specific actions.
-
-**Behavior Driven Development (BDD):** An extension of the test-driven programming style, where tests are used primarily
-for describing the specification of how code should perform. In practice, there's little or no technical difference - it
-all comes down to language. In BDD, the usual terminology is changed to reflect this change of focus, so *Specification*
-is used in place of *Test Case*, and *should* is used in place of *expect* and *assert*.

docs/en/topics/testing/testing-email.md

@@ -1,4 +1,4 @@
-# Email Sending
+# Testing Email
 
 SilverStripe's test system has built-in support for testing emails sent using the Email class.