From 6afad377cb4a57f65001b85ea530011c041057dc Mon Sep 17 00:00:00 2001 From: Dan Brooks Date: Thu, 12 Sep 2013 18:22:46 +0100 Subject: [PATCH] Changes to topics/testing --- .../testing/create-silverstripe-test.md | 69 ------- ...-test.md => creating-a-functional-test.md} | 17 +- .../testing/creating-a-silverstripe-test.md | 66 +++++++ docs/en/topics/testing/glossary.md | 46 +++++ docs/en/topics/testing/index.md | 185 +++++++----------- .../{email-sending.md => testing-email.md} | 2 +- .../{why-test.md => why-should-i-test.md} | 0 7 files changed, 189 insertions(+), 196 deletions(-) delete mode 100644 docs/en/topics/testing/create-silverstripe-test.md rename docs/en/topics/testing/{create-functional-test.md => creating-a-functional-test.md} (92%) create mode 100644 docs/en/topics/testing/creating-a-silverstripe-test.md create mode 100644 docs/en/topics/testing/glossary.md rename docs/en/topics/testing/{email-sending.md => testing-email.md} (99%) rename docs/en/topics/testing/{why-test.md => why-should-i-test.md} (100%) diff --git a/docs/en/topics/testing/create-silverstripe-test.md b/docs/en/topics/testing/create-silverstripe-test.md deleted file mode 100644 index 61ad83a5d..000000000 --- a/docs/en/topics/testing/create-silverstripe-test.md +++ /dev/null @@ -1,69 +0,0 @@ -# How To Create a SilverStripe Test - -A unit test class will test the behaviour of one of your `[api:DataObjects]`. This simple fragment of `[api:SiteTreeTest]` -provides us the basics of creating unit tests. - - :::php - 'home', - 'staff' => 'my-staff', - 'about' => 'about-us', - 'staffduplicate' => 'my-staff-2', - 'product1' => '1-1-test-product', - 'product2' => 'another-product', - 'product3' => 'another-product-2', - 'product4' => 'another-product-3', - ); - - foreach($expectedURLs as $fixture => $urlSegment) { - $obj = $this->objFromFixture('Page', $fixture); - $this->assertEquals($urlSegment, $obj->URLSegment); - } - } - } - - - -There are a number of points to note in this code fragment: - -* Your test is a **subclass of SapphireTest**. Both unit tests and functional tests are a subclass of `[api:SapphireTest]`. -* **static $fixture_file** is defined. The testing framework will automatically set up a new database for **each** of -your tests. The initial database content will be sourced from the YML file that you list in $fixture_file. The property can take an array of fixture paths. -* Each **method that starts with the word "test"** will be executed by the TestRunner. Define as many as you like; the -database will be rebuilt for each of these. -* **$this->objFromFixture($className, $identifier)** can be used to select one of the objects named in your fixture -file. To identify to the object, we provide a class name and an identifier. The identifier is specified in the YML -file but not saved in the database anywhere. objFromFixture() looks the `[api:DataObject]` up in memory rather than using the -database. This means that you can use it to test the functions responsible for looking up content in the database. - -## Assertion commands - -**$this->assertEquals()** is an example of an assertion function. -These functions form the basis of our tests - a test -fails if and only if one or more of the assertions fail. -See [the PHPUnit manual](http://www.phpunit.de/manual/current/en/api.html#api.assert) -for a listing of all PHPUnit's built-in assertions. - -The `[api:SapphireTest]` class comes with additional assertions which are more -specific to the framework, e.g. `[assertEmailSent](api:SapphireTest->assertEmailSent())` -which can simulate sending emails through the `Email->send()` API without actually -using a mail server (see the [testing emails](email-sending)) guide. - -## Fixtures - -Often you need to test your functionality with some existing data, so called "fixtures". -These records are inserted on a fresh test database automatically. -[Read more about fixtures](fixtures). \ No newline at end of file diff --git a/docs/en/topics/testing/create-functional-test.md b/docs/en/topics/testing/creating-a-functional-test.md similarity index 92% rename from docs/en/topics/testing/create-functional-test.md rename to docs/en/topics/testing/creating-a-functional-test.md index 8e7feb835..239c73893 100644 --- a/docs/en/topics/testing/create-functional-test.md +++ b/docs/en/topics/testing/creating-a-functional-test.md @@ -1,4 +1,4 @@ -# Writing functional tests +# Creating a functional tests Functional tests test your controllers. The core of these are the same as unit tests: @@ -13,7 +13,7 @@ URLs. Here is an example from the subsites module: :::php class SubsiteAdminTest extends SapphireTest { private static $fixture_file = 'subsites/tests/SubsiteTest.yml'; - + /** * Return a session that has a user logged in as an administrator */ @@ -22,27 +22,27 @@ URLs. Here is an example from the subsites module: 'loggedInAs' => $this->idFromFixture('Member', 'admin') )); } - + /** * Test generation of the view */ public function testBasicView() { // Open the admin area logged in as admin $response1 = Director::test('admin/subsites/', null, $this->adminLoggedInSession()); - + // Confirm that this URL gets you the entire page, with the edit form loaded $response2 = Director::test('admin/subsites/show/1', null, $this->adminLoggedInSession()); $this->assertTrue(strpos($response2->getBody(), 'id="Root_Configuration"') !== false); $this->assertTrue(strpos($response2->getBody(), ' 1), $this->adminLoggedInSession()); - + $this->assertTrue(strpos($response3->getBody(), 'id="Root_Configuration"') !== false); $this->assertTrue(strpos($response3->getBody(), 'assertTrue(strpos($response3->getBody(), ' 'home', + 'staff' => 'my-staff', + 'about' => 'about-us', + 'staffduplicate' => 'my-staff-2', + 'product1' => '1-1-test-product', + 'product2' => 'another-product', + 'product3' => 'another-product-2', + 'product4' => 'another-product-3', + ); + + foreach($expectedURLs as $fixture => $urlSegment) { + $obj = $this->objFromFixture('Page', $fixture); + $this->assertEquals($urlSegment, $obj->URLSegment); + } + } + } + +Firstly we define a static member `$fixture_file`, this should point to a file that represents the data we want to test, +represented in YAML. When our test is run, the data from this file will be loaded into a test database for our test to use. +This property can be an array of strings pointing to many .yml files, but for our test we are just using a string on its +own. For more detail on fixtures, see the [page on fixtures](fixtures). + +The second part of our class is the `testURLGeneration` method. This method is our test. You can asign many tests, but +again for our purposes there is just the one. When the test is executed, methods prefixed with the word **test** will be +run. The test database is rebuilt everytime one of these methods is run. + +Inside our test method is the `objFromFixture` method that will generate an object for us based on data from our fixture +file. To identify to the object, we provide a class name and an identifier. The identifier is specified in the YAML file +but not saved in the database anywhere, `objFromFixture` looks the `[api:DataObject]` up in memory rather than using the +database. This means that you can use it to test the functions responsible for looking up content in the database. + +The final part of our test is an assertion command, `assertEquals`. An assertion command allows us to test for something +in our test methods (in this case we are testing if two values are equal). A test method can have more than one assertion +command, and if anyone of these tests fail, then the whole test method will fail. + +For more information on PHPUnit's assertions see the [PHPUnit manual](http://www.phpunit.de/manual/current/en/api.html#api.assert). + +The `[api:SapphireTest]` class comes with additional assertions which are more specific to the Sapphire, for example the +`[assertEmailSent](api:SapphireTest->assertEmailSent())` method, which simulates sending emails through the `Email->send()` +API without actually using a mail server. For more details on this see th [testing emails](testing-email)) guide. \ No newline at end of file diff --git a/docs/en/topics/testing/glossary.md b/docs/en/topics/testing/glossary.md new file mode 100644 index 000000000..e8205ee4a --- /dev/null +++ b/docs/en/topics/testing/glossary.md @@ -0,0 +1,46 @@ +# Glossary + +**Assertion:** A predicate statement that must be true when a test runs. + +**Behat:** A behaviour-driven testing library used with SilverStripe as a higher-level +alternative to the `FunctionalTest` API, see [http://behat.org](http://behat.org). + +**Test Case:** The atomic class type in most unit test frameworks. New unit tests are created by inheriting from the +base test case. + +**Test Suite:** Also known as a 'test group', a composite of test cases, used to collect individual unit tests into +packages, allowing all tests to be run at once. + +**Fixture:** Usually refers to the runtime context of a unit test - the environment and data prerequisites that must be +in place in order to run the test and expect a particular outcome. Most unit test frameworks provide methods that can be +used to create fixtures for the duration of a test - `setUp` - and clean them up after the test is done - `tearDown'. + +**Refactoring:** A behavior preserving transformation of code. If you change the code, while keeping the actual +functionality the same, it is refactoring. If you change the behavior or add new functionality it's not. + +**Smell:** A code smell is a symptom of a problem. Usually refers to code that is structured in a way that will lead to +problems with maintenance or understanding. + +**Spike:** A limited and throwaway sketch of code or experiment to get a feel for how long it will take to implement a +certain feature, or a possible direction for how that feature might work. + +**Test Double:** Also known as a 'Substitute'. A general term for a dummy object that replaces a real object with the +same interface. Substituting objects is useful when a real object is difficult or impossible to incorporate into a unit +test. + +**Fake Object**: A substitute object that simply replaces a real object with the same interface, and returns a +pre-determined (usually fixed) value from each method. + +**Mock Object:** A substitute object that mimicks the same behavior as a real object (some people think of mocks as +"crash test dummy" objects). Mocks differ from other kinds of substitute objects in that they must understand the +context of each call to them, setting expectations of which, and what order, methods will be invoked and what parameters +will be passed. + +**Test-Driven Development (TDD):** A style of programming where tests for a new feature are constructed before any code +is written. Code to implement the feature is then written with the aim of making the tests pass. Testing is used to +understand the problem space and discover suitable APIs for performing specific actions. + +**Behavior Driven Development (BDD):** An extension of the test-driven programming style, where tests are used primarily +for describing the specification of how code should perform. In practice, there's little or no technical difference - it +all comes down to language. In BDD, the usual terminology is changed to reflect this change of focus, so *Specification* +is used in place of *Test Case*, and *should* is used in place of *expect* and *assert*. diff --git a/docs/en/topics/testing/index.md b/docs/en/topics/testing/index.md index 9f2053d4b..51fd86e29 100644 --- a/docs/en/topics/testing/index.md +++ b/docs/en/topics/testing/index.md @@ -2,105 +2,65 @@ The SilverStripe core contains various features designed to simplify the process of creating and managing automated tests. -* [Create a unit test](create-silverstripe-test): Writing tests to check core data objects -* [Creating a functional test](create-functional-test): An overview of functional tests and how to write a functional test -* [Email Sending](email-sending): An overview of the built-in email testing code -* [Troubleshooting](testing-guide-troubleshooting): Frequently asked questions list for testing issues -* [Why Unit Test?](why-test): Why should you test and how to start testing - If you are familiar with PHP coding but new to unit testing, you should read the [Introduction](/topics/testing) and check out Mark's presentation [Getting to Grips with SilverStripe Testing](http://www.slideshare.net/maetl/getting-to-grips-with-silverstripe-testing). +This section's page [Why Unit Test?](why-should-i-test) will give you the reasons behind why you should be testing your +code. You should also read over [the PHPUnit manual](http://www.phpunit.de/manual/current/en/). It provides a lot of fundamental concepts that we build on in this documentation. If you're more familiar with unit testing, but want a refresher of some of the concepts and terminology, you can browse -the [Testing Glossary](#glossary). -To get started now, follow the installation instructions below, and check -[Troubleshooting](/topics/testing/testing-guide-troubleshooting) in case you run into any problems. +the [Testing Glossary](glossary). To get started now, follow the installation instructions below, and check +[Troubleshooting](troubleshooting) in case you run into any problems. ## Installation ### Via Composer -Unit tests are not included in the normal SilverStripe downloads, -you are expected to work with local git repositories +Unit tests are not included in the normal SilverStripe downloads, you are expected to work with local git repositories ([installation instructions](/topics/installation/composer)). -Once you've got the project up and running, -check out the additional requirements to run unit tests: +Once you've got the project up and running, check out the additional requirements to run unit tests: - composer update --dev + composer update --dev -The will install (among other things) the [PHPUnit](http://www.phpunit.de/) dependency, -which is the framework we're building our unit tests on. -Composer installs it alongside the required PHP classes into the `vendor/bin/` directory. -You can either use it through its full path (`vendor/bin/phpunit`), or symlink it -into the root directory of your website: +This will install (among other things) the [PHPUnit](http://www.phpunit.de/) dependency, which is the framework we're +building our unit tests on. Composer installs it alongside the required PHP classes into the `vendor/bin/` directory. +You can either use it through its full path (`vendor/bin/phpunit`), or symlink it into the root directory of your website: - ln -s vendor/bin/phpunit phpunit + ln -s vendor/bin/phpunit phpunit ### Via PEAR -Alternatively, you can check out phpunit globally via the PEAR packanage manager +Alternatively, you can check out PHPUnit globally via the PEAR packanage manager ([instructions](https://github.com/sebastianbergmann/phpunit/)). - pear config-set auto_discover 1 - pear install pear.phpunit.de/PHPUnit + pear config-set auto_discover 1 + pear install pear.phpunit.de/PHPUnit -## Running Tests +## Configuration -### Via the "phpunit" Binary on Command Line +### phpunit.xml -The `phpunit` binary should be used from the root directory of your website. +The `phpunit` executable can be configured by commandline arguments or through an XML file. File-based configuration has +the advantage of enforcing certain rules across test executions (e.g. excluding files from code coverage reports), and +of course this information can be version controlled and shared with other team members. - # Runs all tests defined in phpunit.xml - phpunit - - # Run all tests of a specific module - phpunit framework/tests/ - - # Run specific tests within a specific module - phpunit framework/tests/filesystem - - # Run a specific test - phpunit framework/tests/filesystem/FolderTest.php - - # Run tests with optional `$_GET` parameters (you need an empty second argument) - phpunit framework/tests '' flush=all +**Note: This doesn't apply for running tests through the "sake" wrapper** -All command-line arguments are documented on -[phpunit.de](http://www.phpunit.de/manual/current/en/textui.html). +SilverStripe comes with a default `phpunit.xml.dist` that you can use as a starting point. Copy the file into a new +`phpunit.xml` and customize to your needs - PHPUnit will auto-detect its existence, and prioritize it over the default +file. -### Via the "sake" Wrapper on Command Line +There's nothing stopping you from creating multiple XML files (see the `--configuration` flag in +[PHPUnit documentation](http://www.phpunit.de/manual/current/en/textui.html)). For example, you could have a +`phpunit-unit-tests.xml` and `phpunit-functional-tests.xml` file (see below). -The [sake](/topics/commandline) executable that comes with SilverStripe can trigger a customized -"[api:TestRunner]" class that handles the PHPUnit configuration and output formatting. -While the custom test runner a handy tool, its also more limited than using `phpunit` directly, -particularly around formatting test output. +### Database Permissions - # Run all tests - sake dev/tests/all - - # Run all tests of a specific module (comma-separated) - sake dev/tests/module/framework,cms - - # Run specific tests (comma-separated) - sake dev/tests/FolderTest,OtherTest - - # Run tests with optional `$_GET` parameters - sake dev/tests/all flush=all - - # Skip some tests - sake dev/tests/all SkipTests=MySkippedTest - -### Via Web Browser - -Executing tests from the command line is recommended, since it most closely reflects -test runs in any automated testing environments. If for some reason you don't have -access to the command line, you can also run tests through the browser. - - http://localhost/dev/tests +SilverStripe tests create thier own database when they are run. Because of this the database user in your config file +should have the appropriate permissions to create new databases on your server, otherwise tests will not run. ## Writing Tests @@ -120,71 +80,60 @@ Some people may note that we have used the same naming convention as Ruby on Rai Tutorials and recipes for creating tests using the SilverStripe framework: -* **[Create a SilverStripe Test](/topics/testing/create-silverstripe-test)** -* **[Create a Functional Test](/topics/testing/create-functional-test)** -* **[Test Outgoing Email Sending](/topics/testing/email-sending)** +* [Creating a SilverStripe test](creating-a-silverstripe-test): Writing tests to check core data objects +* [Creating a functional test](creating-a-functional-test): An overview of functional tests and how to write a functional test +* [Testing Outgoing Email](testing-email): An overview of the built-in email testing code -## Configuration +## Running Tests -### phpunit.xml +### Via the "phpunit" Binary on Command Line -The `phpunit` executable can be configured by commandline arguments or through an XML file. -File-based configuration has the advantage of enforcing certain rules across -test executions (e.g. excluding files from code coverage reports), and of course this -information can be version controlled and shared with other team members. +The `phpunit` binary should be used from the root directory of your website. -**Note: This doesn't apply for running tests through the "sake" wrapper** + # Runs all tests defined in phpunit.xml + phpunit -SilverStripe comes with a default `phpunit.xml.dist` that you can use as a starting point. -Copy the file into a new `phpunit.xml` and customize to your needs - PHPUnit will auto-detect -its existence, and prioritize it over the default file. + # Run all tests of a specific module + phpunit framework/tests/ -There's nothing stopping you from creating multiple XML files (see the `--configuration` flag in [PHPUnit documentation](http://www.phpunit.de/manual/current/en/textui.html)). -For example, you could have a `phpunit-unit-tests.xml` and `phpunit-functional-tests.xml` file (see below). + # Run specific tests within a specific module + phpunit framework/tests/filesystem -## Glossary {#glossary} + # Run a specific test + phpunit framework/tests/filesystem/FolderTest.php -**Assertion:** A predicate statement that must be true when a test runs. + # Run tests with optional `$_GET` parameters (you need an empty second argument) + phpunit framework/tests '' flush=all -**Behat:** A behaviour-driven testing library used with SilverStripe as a higher-level -alternative to the `FunctionalTest` API, see [http://behat.org](http://behat.org). +All command-line arguments are documented on +[phpunit.de](http://www.phpunit.de/manual/current/en/textui.html). -**Test Case:** The atomic class type in most unit test frameworks. New unit tests are created by inheriting from the -base test case. +### Via the "sake" Wrapper on Command Line -**Test Suite:** Also known as a 'test group', a composite of test cases, used to collect individual unit tests into -packages, allowing all tests to be run at once. +The [sake](/topics/commandline) executable that comes with SilverStripe can trigger a customized +"[api:TestRunner]" class that handles the PHPUnit configuration and output formatting. +While the custom test runner a handy tool, its also more limited than using `phpunit` directly, +particularly around formatting test output. -**Fixture:** Usually refers to the runtime context of a unit test - the environment and data prerequisites that must be -in place in order to run the test and expect a particular outcome. Most unit test frameworks provide methods that can be -used to create fixtures for the duration of a test - `setUp` - and clean them up after the test is done - `tearDown'. + # Run all tests + sake dev/tests/all -**Refactoring:** A behavior preserving transformation of code. If you change the code, while keeping the actual -functionality the same, it is refactoring. If you change the behavior or add new functionality it's not. + # Run all tests of a specific module (comma-separated) + sake dev/tests/module/framework,cms -**Smell:** A code smell is a symptom of a problem. Usually refers to code that is structured in a way that will lead to -problems with maintenance or understanding. + # Run specific tests (comma-separated) + sake dev/tests/FolderTest,OtherTest -**Spike:** A limited and throwaway sketch of code or experiment to get a feel for how long it will take to implement a -certain feature, or a possible direction for how that feature might work. + # Run tests with optional `$_GET` parameters + sake dev/tests/all flush=all -**Test Double:** Also known as a 'Substitute'. A general term for a dummy object that replaces a real object with the -same interface. Substituting objects is useful when a real object is difficult or impossible to incorporate into a unit -test. + # Skip some tests + sake dev/tests/all SkipTests=MySkippedTest -**Fake Object**: A substitute object that simply replaces a real object with the same interface, and returns a -pre-determined (usually fixed) value from each method. +### Via Web Browser -**Mock Object:** A substitute object that mimicks the same behavior as a real object (some people think of mocks as -"crash test dummy" objects). Mocks differ from other kinds of substitute objects in that they must understand the -context of each call to them, setting expectations of which, and what order, methods will be invoked and what parameters -will be passed. +Executing tests from the command line is recommended, since it most closely reflects +test runs in any automated testing environments. If for some reason you don't have +access to the command line, you can also run tests through the browser. -**Test-Driven Development (TDD):** A style of programming where tests for a new feature are constructed before any code -is written. Code to implement the feature is then written with the aim of making the tests pass. Testing is used to -understand the problem space and discover suitable APIs for performing specific actions. - -**Behavior Driven Development (BDD):** An extension of the test-driven programming style, where tests are used primarily -for describing the specification of how code should perform. In practice, there's little or no technical difference - it -all comes down to language. In BDD, the usual terminology is changed to reflect this change of focus, so *Specification* -is used in place of *Test Case*, and *should* is used in place of *expect* and *assert*. + http://localhost/dev/tests \ No newline at end of file diff --git a/docs/en/topics/testing/email-sending.md b/docs/en/topics/testing/testing-email.md similarity index 99% rename from docs/en/topics/testing/email-sending.md rename to docs/en/topics/testing/testing-email.md index b7bcda870..3c3349891 100644 --- a/docs/en/topics/testing/email-sending.md +++ b/docs/en/topics/testing/testing-email.md @@ -1,4 +1,4 @@ -# Email Sending +# Testing Email SilverStripe's test system has built-in support for testing emails sent using the Email class. diff --git a/docs/en/topics/testing/why-test.md b/docs/en/topics/testing/why-should-i-test.md similarity index 100% rename from docs/en/topics/testing/why-test.md rename to docs/en/topics/testing/why-should-i-test.md