diff --git a/doc/test/frontend_tests.md b/doc/test/frontend_tests.md new file mode 100644 index 000000000..9eaa974a3 --- /dev/null +++ b/doc/test/frontend_tests.md @@ -0,0 +1,60 @@ +# Frontend component tests + +Frontend tests test your code from the users perspective: By opening a specific url, executing a few clicks, strokes, etc. +and expecting something to happen. We use [CasperJS](http://casperjs.org/) for frontend testing, which is basically a +headless Webkit browser. + +## The current state of frontend testing + +Currently frontend tests are not very advanced: We spawn a small - non php - server on port 12999 to test static files +and javascript behaviour. This will change in the future where we are going to test an installation (or use PHP 5.4 +standalone server). + +## Writing tests + +### Test bootstrap + +In order to make testing more comfortable, the i2w config provides a few helpers to make testing more straightforward. +In general you start your test by including i2w-config: + + var i2w = require('./i2w-config'); + +and afterward creating a testenvironment with the getTestEnv() method: + + var casper = i2w.getTestEnv(); + +You can then start the test by calling casper.start with the startpage (the servers root is always frontend/static, where +public is a symlink to the icingaweb public folder). + + casper.start("http://localhost:12999/generic.html"); + +As we use requirejs, this has to be set up for our testcases. i2w provides a setupRequireJs function that does everything for you. +You just have to run this method on your testpage (note that the tested JavaScript is isolated from your test case's JavaScript, if +you want to execute JavaScript you must use the casper.page.evaluate method). + + casper.then(function() { + // Setup requirejs + casper.page.evaluate(i2w.setupRequireJs, {icinga: true}); + }); + +### Testing + +Afterwards, everything is like a normal CasperJS test, so you can wrap your assertions in a casper.then method: + + // assert a specific title + casper.then(function() { + this.test.assertTitle("Just an empty page"); + }); + +Note that asynchronous calls reuqire you to wait or provide a callback until the resource is loaded: + + // waitForSelector calls callbackFn as soon as the selector returns a non-empty set + casper.waitForSelector("div#icinga-main a", callbackFn); + +At the end of your test, you have to provide + + casper.run(function() { + this.test.done(); + }); + +Otherwise the tests won't be executed. diff --git a/doc/test/js_tests.md b/doc/test/js_tests.md new file mode 100644 index 000000000..aa08fb646 --- /dev/null +++ b/doc/test/js_tests.md @@ -0,0 +1,88 @@ +# Writing JavaScipt tests + +JavaScript tests are executed using [mocha](http://visionmedia.github.io/mocha/) as a test framework and +[should.js](https://github.com/visionmedia/should.js/) as the assertion framework. + +## Mocking require.js + +As we use require.js for asynchronous dependency resolution in JavaScript, this can lead to problems in our node.js +environment. In order to avoid requirejs calls to cause issues, it has been mocked in the testlib/asyncmock.js class +and should be required in every testcase: + + var rjsmock = require("requiremock.js"); + +rjsmock now makes dependency management comfortable and provides the following most important methods: + + // remove all registered dependencies from the rjsmock cache + rjsmock.purgeDependencies(); + + // register the following objects underneath the following requirejs paths: + rjsmock.registerDependencies({ + 'icinga/container' : { + updateContainer : function() {}, + createPopupContainer: function() {} + } + }); + // in your js code a require(['icinga/container], function(container) {}) would now have the above mock + // object in the container variable + + requireNew("icinga/util/async.js"); // requires icinga/util/async.js file - and ignores the requirejs cache + var async = rjsmock.getDefine(); // returns the last define, this way you can retrieve a specific javacript file + +## Faking async responses + +As we currently use the icinga/util/async.js class for all asynchronous requests, it's easy to fake responses. The asyncmock.js +class provides functions for this. To use it in your test, you first have to require it: + + var asyncMock = require("asyncmock.js"); + +You now can use asyncMock.setNextAsyncResult((async) asyncManager, (string) resultString, (bool) fails, (object) headers) to +let the next request of the passed asyncManager object return resultString as the response, with the headers provided as the +last parameter. If fails = true, the error callback of the request will be called. + + +## Example test + +The following example describes a complete test, (which tests whether the registerHeaderListener method in the async class works) : + + var should = require("should"); // require should.js for assertions + var rjsmock = require("requiremock.js"); // use the requiremock described above + var asyncMock = require("asyncmock.js"); // Helper class to fake async requests + + GLOBAL.document = $('body'); // needed when our test accesses window.document + + + describe('The async module', function() { // start the test scenario + it("Allows to react on specific headers", function(done) { // Start a test case - when done is called it is finished + rjsmock.purgeDependencies(); // Remove any dependency previously declared + rjsmock.registerDependencies({ // Mock icinga/container, as this is a dependency for the following include + 'icinga/container' : { + updateContainer : function() {}, + createPopupContainer: function() {} + } + }); + + requireNew("icinga/util/async.js"); // This is the file we want to test, load it and all of it's dependencies + var async = rjsmock.getDefine(); // Retrieve a reference to the loaded file + + // Use asyncMock.setNextAsyncResult to let the next async request return 'result' without failing and set + // the response headers 'X-Dont-Care' and 'X-Test-Header' + asyncMock.setNextAsyncResult(async, "result", false, { + 'X-Dont-Care' : 'Ignore-me', + 'X-Test-Header' : 'Testme123' + }); + + // register a listener for results with the X-Test-Header response + async.registerHeaderListener("X-Test-Header", function(value, header) { + // test for the correct header + should.equal("Testme123", value); + // call done to mark this test as succeeded + done(); + },this); + // run the faked request + var test = async.createRequest(); + }); + + }); + + diff --git a/doc/test/php_tests.md b/doc/test/php_tests.md new file mode 100644 index 000000000..6149e703d --- /dev/null +++ b/doc/test/php_tests.md @@ -0,0 +1,106 @@ +# Writing PHPUnit tests + +## Test path and filename + +The path where you should put your PHPUnit tests should reflect the path in the sourcetree, with test/php/ prepended. So +if you're testing a file library/Icinga/My/File.php the test file should be at test/php/library/Icinga/My/File.php. This +also applies for modules, where the test folder is underneath modules/myModule/test/php + +## Example test skeleton + +Let's assume you're testing a class MyClass underneath the MyModule module and the file can be found at +modules/mymodule/library/MyModule/Helper/MyClass.php. + + assertTrue(true, "Asserting that the world didn't end yet"); + } + } + +## Requirements and the dependency mess + +### spl_autoload_register vs. require + +When looking at our test classes, you'll notice that we don't use PHPs autoloader to automatically load dependency, but +write 'require_once' by ourselfs. This has the following reasons: + +- When writing tests, you to be aware of every dependency your testclass includes. With autoloading, it's not directly + obvious which classes are included during runtime. +- When mocking classes, you don't need to tell your autoloader to use this class instead of the one used in production +- Tests can't be run isolated without an boostrap class initializing the autoloader + +### How to avoid require_once massacres: LibraryLoader + +The downside of this approach is obvious: Especially when writing compoment tests you end up writing a lot of 'require' +classes. In the worst case, the PHP require_once method doesn't recognize a path to be already included and ends up +with an 'Cannot redeclare class XY' issue. + +To avoid this, you should implement a LibraryLoader class for your component that handles the require_once calls. +For example, the status.dat component tests has a TestLoader class that includes all dependencies of the component: + + + namespace Tests\Icinga\Protocol\Statusdat; + use Test\Icinga\LibraryLoader; + + require_once('library/Icinga/LibraryLoader.php'); + + /** + * Load all required libraries to use the statusdat + * component in integration tests + * + **/ + class StatusdatTestLoader extends LibraryLoader + { + /** + * @see LibraryLoader::requireLibrary + * + **/ + public static function requireLibrary() + { + // include Zend requirements + require_once 'Zend/Config.php'; + require_once 'Zend/Cache.php'; + require_once 'Zend/Log.php'; + + // retrieve the path to the icinga library + $libPath = self::getLibraryPath(); + + // require library dependencies + require_once($libPath."/Data/AbstractQuery.php"); + require_once($libPath."/Application/Logger.php"); + require_once($libPath."/Data/DatasourceInterface.php"); + + // shorthand for the folder where the statusdat component can be found + $statusdat = realpath($libPath."/Protocol/Statusdat/"); + require_once($statusdat."/View/AccessorStrategy.php"); + // ... a few more requires ... + require_once($statusdat."/Query/Group.php"); + } + } + +Now an component test (like tests/php/library/Icinga/Protocol/Statusdat/ReaderTest.php) can avoid the require calls and +just use the requireLibrary method: + + use Icinga\Protocol\Statusdat\Reader as Reader; + + // Load library at once + require_once("StatusdatTestLoader.php"); + StatusdatTestLoader::requireLibrary(); + +**Note**: This should be used for component tests, where you want to test the combination of your classes. When testing + a single execution unit, like a method, it's often better to explicitly write your dependencies. + +If you compare the first approach with the last one you will notice that, even if we produced more code in the end, our +test is more verbose in what it is doing. When someone is updating your test, he should easily see what tests are existing +and what scenarios are missing. \ No newline at end of file diff --git a/doc/test/running_tests.md b/doc/test/running_tests.md new file mode 100644 index 000000000..d34397e27 --- /dev/null +++ b/doc/test/running_tests.md @@ -0,0 +1,89 @@ +# Test organization + +## Testfolders + +Tests for the application can be found underneath the test folder: + + test/ + php/ PHPUnit tests for backend code + js/ mocha tests for JavaScript frontend code unittests + frontend/ Integration tests for the frontend using casperjs + +The same structure applies for modules, which also contain a toplevel test folder and suitable subtests. When you fix +a bug and write a regression test for it, put the test in the 'regression' and name it %DESCRIPTION%%TicketNumber% (myBug1234.js) + +## Running tests + +The tests can be run in the specific folder using the runtests script. + +Running PHP tests example: + + cd test/php + ./runtests + +In this case, all application and all module tests will be executed. The testrunners also support additional flags, which +affect they way the test is executed: + + Options: + -h, --help show this help message and exit + -b, --build Enable reporting. + -v, --verbose Be more verbose. + -i PATTERN, --include=PATTERN + Include only specific files/test cases. + -V, --vagrant Run in vagrant VM + +Some tests also support the --exclude method, it's best to use the --help option to see which flags are supported. + + +## Setting up databases + +Despite running most of the tests should work out of the box, a few specific cases require some setup. +At this moment, only database tests require additional setup and expect an icinga_unittest user with an icinga_unittest +database to exist and have rights in your database. + +### The database test procedure + +When testing PostgreSQL and MySQL databases, the test library (normally) executes the following test procedure for every +test case: + +- Log in to the rdbms as the user icinga_unittest with password icinga_unittest +- Use the icinga_unittest database (which must be existing) +- **Drop all tables** in the icinga_unittest database +- Create a new, clean database schema + +If anything goes wrong during this procedure, the test will be skipped (because maybe you don't have a pgsql database, but +want to test mysql, for example). + +### Setting up a test user and database in MySQL + +In MySQL, it's best to create a user icinga_unittest@localhost, a database icinga_unittest and grant all privileges on +this database: + + mysql -u root -p + mysql> CREATE USER `icinga_unittest`@`localhost` IDENTIFIED BY 'icinga_unittest'; + mysql> CREATE DATABASE `icinga_unittest`; + mysql> GRANT ALL PRIVILEGES ON `icinga_unittest`.* TO `icinga_unittest`@`localhost`; + mysql> FLUSH PRIVILEGES; + mysql> quit + +### Setting up a test user and database in PostgreSQL + +In PostgreSQL, you have to modify the pg_hba database if you don't have password authentication set up (which often is +the case). In this setup the icinga_unittest user is set to trust authentication on localhost, which means that no +password is queried when connecting from the local machine: + + sudo su postgres + psql + postgres=# CREATE USER icinga_unittest WITH PASSWORD 'icinga_unittest'; + postgres=# CREATE DATABASE icinga_unittest; + postgres=# \q + bash$ createlang plpgsql icinga; + + +Add the following lines to your pg_hba.conf (etc/postgresql/X.x/main/pg_hba.conf under debian, /var/lib/pgsql/data/pg_hba.conf for Redhat/Fedora) +to enable trust authentication for the icingaweb user when connecting from the localhost. + + local icinga_unittest icinga_unittest trust + host icinga_unittest icinga_unittest 127.0.0.1/32 trust + host icinga_unittest icinga_unittest ::1/128 trust + diff --git a/doc/test/styleguide.md b/doc/test/styleguide.md new file mode 100644 index 000000000..9f0bd62e9 --- /dev/null +++ b/doc/test/styleguide.md @@ -0,0 +1,451 @@ +# General testing guidelines + +## The short summary + +This list summarizes what will be described in the next few chapters: + +- You really should write tests for your code +- Think about your what you want to test and how you can assert the behaviour correctly. +- Isolate your tests and start without any assumptions about your test environment (like a specific user existing). The + only excuse here is to assume a correct login if you need a database for testing (but not the tables/content of the database!) +- Don't just test correct behaviour - test border cases and invalid input or assumptions to make sure the application handles + cases that might not occur in a normal test environment +- Use description strings in your assertions, this makes it easy to detect what's going wrong when they fail and it's easier + to follow what *exactly* you are asserting +- Test methods should be one scenario and the method name should describe this scenario. + *testLogin* is bad for example (do you test if the login fails? Or if it is correct? + *testLoginWithCorrectCredentials*, *testLoginWithWrongPassword* is a far better name here +- Your assertions should reflect one test scenario, i.e. don't write one test method that tests if something works **and** + if it correctly detects errors after it works. Write one test to determine the behaviour with correct input and one that + tests the behaviour with invalid input. +- When writing unit-tests (like function level tests), try to keep your dependencies as low as possible (best indicator herefor + is the number of require calls in your test). Mock external components and inject them into the class you want to test. If + your testsubject is not able to use mocked dependencies, it's often a design flaw and should be considered as a bug + (and be fixed) +- When writing component tests with a lot of dependencies, wrap the require calls in a LibraryLoader subclass + + +## What should be tested + +### Writing meaningful tests + +Writing tests doesn't ensure that your code is free of errors, but it can test if, in a specific scenario, the behaviour of +your code is as expected. This means that you have to think about your test scenario before writing a test. This involves +three steps: + +- Determine what you want to test: Which errors can occur, what is the normal input and what are the border cases. +- Define a test scenario: What datasets make sense for the test? +- How do I write my assertions so they are really meaningful about the correctness of the testcase + +Especially the border cases are important, often they lay inside of try/catch blocks or error detection routines. When +looking at your code coverages, these blocks should be covered. + +### Example + +Let's say you have the following function (the examples in this section are to be considered as php-like pseudocode) : + + function isValidName($name) + { + if (hasCorrectFormat($name)) { + return false; + } + + if (nameExistsInDatabase($name)) { + return false; + } + + return true; + } + + +#### Determine what to test: + +At first glance there can be 3 scenarios: + +1. The username is unique and valid +2. The username has the wrong format +3. The username has the correct format, but exists in the database + +But - what happens if the database is down? Should an exception be thrown or should it return false? This case has to be added +to your tests, so you have (at least!) the following scenarios: + +1. The username is unique and valid +2. The username has the wrong format +3. The username has the correct format, but exists in the database +4. The username has the correct format, but access to the database fails. + + +#### Determine meaningful testscenarios + +When it comes to creating your testscenario, we start with the easiest one: Test the wrongly formatted username. This +should be pretty straightforward, as we never reach the code where we need the database: + + function testWrongFormat() + { + assertFalse(isValidName("$$% 1_', "Assert a name with special characters to be considered an invalid name"); + } + +The first and third scenario are more sophisticated (like always, when a database comes into the game). There are two ways +to test this: +- Either you create an empty table on each test, fill them with the users and run the test +- or you Mock the database call with a class that behaves like querying the users and returns true or false in each case + +You **shouldn't** create a static database for all tests and assume this one to be existing - these are decoupled from the +actual test and soon get outdated or difficult to reflect all scenarios. Also it's not considered good practice to create +a precondition your tests have to rely on. In this case the second approach makes sense, as the mock class should be rather simple: + + + function testCorrectUserName() + { + // set no users in the database mock + $db = new UserDatabaseMock(array()); + setupUserDatabaseMock($db); + + assertTrue(isValidName("hans"), "Assert a correctly formatted and unique name to be considered valid"); + } + + function testNonUniqueUserName() + { + // set no users in the database mock + $db = new UserDatabaseMock(array("pete")); + setupUserDatabaseMock($db); + + assertFalse(isValidName("pete"), "Assert a correctly formatted, but existing name to be considered invalid"); + } + +The exception can be tested by providing invalid db credentials when using a real databse or by extending the mock (which +we will do here): + + function testDatabaseError() + { + // set no users in the database mock + $db = new UserDatabaseMock(array()); + $db->setThrowsException(true); + setupUserDatabaseMock($db); + + assertFalse(isValidName("hans"), "Assert a correct, unique user to be considered invalid when the database doesn't work"); + } + +This, of course, depends on how you want the behaviour to be when the db is down: Do you want to log the error and proceed +as usual or do you want the error to bubble up via an exception. + +#### Writing sensible assertions + +It's crucial to write sensible assertions in your test-classes: You can write a perfect test that covers the right scenario, +but don't catch errors because you aren't asking the correct questions. + +- Write assertions that cover your scenario - if you test a correct behaviour don't test what happens if something goes wrong + (this is a seperate scenario) +- While you should try to write redundant assertions it's better to assert more than to have a missing assertion +- A failed assertion means that the implementation is incorrect, other assertions are to be avoided (like testing if a + precondition applies) +- When testing one function, you have to be naive and assume that everything else is bug free, testing whether other parts + worked correctly before testing should be made in a different test. + +## How to test + +Unit tests should only test an isolated, atomic of your code - in theory just one single function - and shouldn't +need much dependency handling. An example for a unittest would be to test the following (hypothetical) class method: + + class UserManager + { + /** + * returns true when a user with this name exists and the + * password for this user is correct + **/ + public function isCorrectPassword($name, $password) + { + // You needn't to know the implementation. + } + } + +### The wrong way + +A unit test for this user could, but should not look like this (we'll explain why): + + require_once "../../library/Icinga/MyLibrary/UserManager.php"; + // needed by UserManager + require_once "../../library/Icinga/Authentication/Manager.php" + require_once "../../library/Icinga/Authentication/User.php" + // .. imagine a few more require_once + + use Icinga/MyLibrary/UserManager + + class UserManagerTest extends \PHPUnit_Framework_TestCase + { + /** + * Test whether an user is correctly recognized by the UserManager + * + **/ + public function testUserManager() + { + // Connect to the test database that contains jdoe and jsmith + $mgrConfg = new \Zend_Config(array( + "backend" => "db", + "user" => "dbuser.." + "pass" => + // all the other db credentials + )); + + $mgr = new UserManager($mgrConfig); + + $this->assertTrue($mgr->isCorrectPassword("jdoe", "validpassword")); + $this->assertTrue($mgr->isCorrectPassword("jsmith", "validpassword")); + $this->assertTrue($mgr->isCorrectPassword("jdoe", "nonvalidpassword")); + $this->assertTrue($mgr->isCorrectPassword("jsmith", "nonvalidpassword")); + $this->assertTrue($mgr->isCorrectPassword("hans", "validpasswor")); + } + +This test has obviously a few issues: + +- First, it assert a precondition to apply : A database must exist with the users jdoe and jsmith and the credentials + must match the ones provided in the test +- There are a lot of dependencies in this code, almost the complete Authentication code must exists. Our test + will fail if this code changes or contains bugs, even the isCorrectPassword method is correct. + +### Reducing dependencies + +To avoid these issues, you need to code your classes with testing in mind. Maybe now you're screaming *"Tests shouldn't +affect my code, just test if it is correct!"*, but it's not that easy. Testability should be considered as a quality aspect +of your code, just like commenting, keeping functions coherent, etc. Non-testable code should be considered as a bug, or +at least as a design-flaw. + +One big buzzword in development is now Inversion of Control and Dependency Injection. You can google the details, but in +our case it basically means: Instead of your class (in this case UserManager) setting up it's dependencies (creating an Authentication Manager and +then fetching users from it), the dependencies are given to the class. This has the advantage that you can mock the +dependencies in your testclasses which heavily reduces test-complexity. On the downside this can lead to more complicate +Api's, as you have to know the dependencies of your Object when creating it. Therefore we often allow to provide an +dependency from the outside (when testing), but normally create the dependencies when nothing is provided (normal use). + +In our case we could say that we allow our UserManager to use a provided set of Users instead of fetching it from the +Authmanger: + + class UserManager + { + + public function __construct($config, $authMgr = null) + { + if ($authMgr == null) { + // no Authmanager provided, resolve dependency by yourself + $this->authMgr = new AuthManager($config); + } else { + $this->authMgr = $authMgr; + } + } + } + +It would of course be best to create an Interface like UserSource which the AuthManger implements, but in this example +we trust our Programmer to provide a suitable object. We now can eliminate all the AuthManager dependencies by mocking the +AuthManager (lets dumb it down to just providing an array of users): + + require_once "../../library/Icinga/MyLibrary/UserManager.php"; + use Icinga/MyLibrary/UserManager + + class AuthManagerMock + { + public $users; + + /** + * Create a new mock classw with the provided users and their credentials + * + * @param array $userPasswordCombinations The users and password combinations to use + **/ + public function __construct(array $userPasswordCombinations) + { + $this->users = $userPasswordCombinations; + } + + public function getUsers() + { + return $this->users; + } + } + + class UserManagerTest extends \PHPUnit_Framework_TestCase + { + /** + * Test whether an user is correctly recognized by the UserManager + * + **/ + public function testUserManager() + { + $authMock = new AuthManagerMock(array( + "jdoe" => "validpassword", + "jsmith" => "validpassword" + )); + $mgrConfg = new \Zend_Config(array(), $authMock); + $mgr = new UserManager($mgrConfig); + + $this->assertTrue($mgr->isCorrectPassword("jdoe", "validpassword")); + $this->assertTrue($mgr->isCorrectPassword("jsmith", "validpassword")); + $this->assertFalse($mgr->isCorrectPassword("jdoe", "nonvalidpassword")); + $this->assertFalse($mgr->isCorrectPassword("jsmith", "nonvalidpassword")); + $this->assertFalse($mgr->isCorrectPassword("hans", "validpassword")); + } + +Ok, we might have more code here than before, but our test is now less like prone to fail: + +- Our test doesn't assume any preconditions to apply, like having a db server with correct users +- The require call to the AuthManager is gone, so if there's a bug in the AuthManager implementation our test is not affected + + + +### Splitting up assertions + +The test is now not that bad, but still has a few issues: + +- If an assert fails, we don't know which one, as the message will be rather generic ("failed asserting that False is True") +- In this case it might be obvious what we test, but if someone sees the class and the assertions, he doesn't know what +assumptions are made + +To fix those issues, we have to split up our big test method in several smaller one and give the testmethod **talking names**. +Also, the assertions should get an error message that will be printed on failure. + + /** + * Testcases for the UserManager class + * + **/ + class UserManagerTest extends \PHPUnit_Framework_TestCase + { + /** + * Creates a new UserManager with a mocked AuthManage + * + * @returns UserManager + **/ + public function getUserManager() + { + $authMock = new AuthManagerMock(array( + "jdoe" => "validpassword", + "jsmith" => "validpassword" + )); + $mgrConfg = new \Zend_Config(array(), $authMock); + return new UserManager($mgrConfig); + } + + /** + * Tests whether correct user/name combinations are considered valid + * + **/ + public function testCorrectUserPasswordCombinations() + { + $mgr = $this->getUserManager(); + $this->assertTrue( + $mgr->isCorrectPassword("jdoe", "validpassword"), + "Asserted that a correct user/password combination is considered valid for jdoe" + ); + $this->assertTrue( + $mgr->isCorrectPassword("jsmith", "validpassword"), + "Asserted that a correct user/password combination is considered valid for jsmith" + ); + } + + /** + * Tests whether invalid names are rejected + * + **/ + public function testInvalidUsernameRecognition() + { + $mgr = $this->getUserManager(); + $this->assertFalse( + $mgr->isCorrectPassword("hans", "validpassword"), + "Asserted a non-existing user to be be considered invalid" + ); + } + + /** + * Tests whether invalid passwords for existing users are rejected + * + **/ + public function testInvalidPasswordRecognition() + { + $mgr = $this->getUserManager(); + $this->assertFalse( + $mgr->isCorrectPassword("jsmith", "nonvalidpassword"), + "Asserted that an invalid password for an existing user is considered invalid" + ); + } + } + +Now if something fails, we now see what has been tested via the testmethod and what caused the test to fail in the +assertion error message. You could also leave the comments and everybody knows what you are doing. + +## Testing PHP + + +## Requirements and the dependency mess + +### spl_autoload_register vs. require + +When looking at our test classes, you'll notice that we don't use PHPs autoloader to automatically load dependency, but +write 'require_once' by ourselfs. This has the following reasons: + +- When writing tests, you to be aware of every dependency your testclass includes. With autoloading, it's not directly + obvious which classes are included during runtime. +- When mocking classes, you don't need to tell your autoloader to use this class instead of the one used in production +- Tests can't be run isolated without an boostrap class initializing the autoloader + +### How to avoid require_once massacres: LibraryLoader + +The downside of this approach is obvious: Especially when writing compoment tests you end up writing a lot of 'require' +classes. In the worst case, the PHP require_once method doesn't recognize a path to be already included and ends up +with an 'Cannot redeclare class XY' issue. + +To avoid this, you should implement a LibraryLoader class for your component that handles the require_once calls. +For example, the status.dat component tests has a TestLoader class that includes all dependencies of the component: + + + namespace Tests\Icinga\Protocol\Statusdat; + use Test\Icinga\LibraryLoader; + + require_once('library/Icinga/LibraryLoader.php'); + + /** + * Load all required libraries to use the statusdat + * component in integration tests + * + **/ + class StatusdatTestLoader extends LibraryLoader + { + /** + * @see LibraryLoader::requireLibrary + * + **/ + public static function requireLibrary() + { + // include Zend requirements + require_once 'Zend/Config.php'; + require_once 'Zend/Cache.php'; + require_once 'Zend/Log.php'; + + // retrieve the path to the icinga library + $libPath = self::getLibraryPath(); + + // require library dependencies + require_once($libPath."/Data/AbstractQuery.php"); + require_once($libPath."/Application/Logger.php"); + require_once($libPath."/Data/DatasourceInterface.php"); + + // shorthand for the folder where the statusdat component can be found + $statusdat = realpath($libPath."/Protocol/Statusdat/"); + require_once($statusdat."/View/AccessorStrategy.php"); + // ... a few more requires ... + require_once($statusdat."/Query/Group.php"); + } + } + +Now an component test (like tests/php/library/Icinga/Protocol/Statusdat/ReaderTest.php) can avoid the require calls and +just use the requireLibrary method: + + use Icinga\Protocol\Statusdat\Reader as Reader; + + // Load library at once + require_once("StatusdatTestLoader.php"); + StatusdatTestLoader::requireLibrary(); + +**Note**: This should be used for component tests, where you want to test the combination of your classes. When testing + a single execution unit, like a method, it's often better to explicitly write your dependencies. + +If you compare the first approach with the last one you will notice that, even if we produced more code in the end, our +test is more verbose in what it is doing. When someone is updating your test, he should easily see what tests are existing +and what scenarios are missing. + diff --git a/modules/monitoring/docs/test/how_to_test_controllers.md b/modules/monitoring/docs/test/how_to_test_controllers.md new file mode 100644 index 000000000..d23c06c12 --- /dev/null +++ b/modules/monitoring/docs/test/how_to_test_controllers.md @@ -0,0 +1,98 @@ +# Testing controllers with different backends + +Icingaweb's monitoring controllers support a variety of different backends (IDO, Statusdat, Livestatus) and make it +therefore hard to test for every backend. In order to make life a little bit easier, Test-Fixtures allow you to setup +a backend with specific monitoring data and test it afterwards by running the controller's action. + +## Example + +It's best to subclass MonitoringControllerTest (found in modules/monitoring/test/testlib), as this handles depency resoultion +and setup for you: + + + // assume our test is underneath the test/application/controllers folder in the monitoring module + require_once(dirname(__FILE__).'/../../testlib/MonitoringControllerTest.php'); + + use Test\Monitoring\Testlib\MonitoringControllerTest; + use Test\Monitoring\Testlib\Datasource\TestFixture; + use Test\Monitoring\Testlib\Datasource\ObjectFlags; + + class MyControllerTest extends MonitoringControllerTest + { + public function testSomething() + { + // Create a test fixture + $fixture = new TestFixture() + $fixture->addHost('host', 0) // Add a host with state OK + ->addToHostgroup('myHosts') // Add host to hostgroup + ->addService('svc1', 1) // Add a warning service 'svc1' underneath host + ->addToServiceGroup('svc_warning') // add this service to a servicegroup svc_warning + ->addService('svc2', 2, ObjectFlags::ACKNOWLEDGED()) // Add a critical, but acknowledged service to this host + ->addService( + 'svc3', + 1, + new ObjectFlags(), + array("customvariables" => + array("customer" => "myCustomer") + ) + ); // add a warning service with a customvariable + + $this->setupFixture($fixture, "mysql"); // setup the fixture for MySQL, so the backend is populated with the data set above + // backends can be mysql, pgsql, statusdat (and in the future livestatus) + $controller = $this->requireController('MyController', 'mysql'); // request a controller with the given backend injected + $controller->myAction(); // controller is now the Zend controller instance, perform an action + $result = $controller->view->hosts->fetchAll(); // get the result of the query + // and assert stuff + $this->assertEquals(1, count($result), "Asserting one host being retrieved in the controller"); + } + } + +## The Test-fixture API + +In order to populate your backend with specific monitoring objects, you have to create a TestFixture class. This class +allows you to setup the monitoring objects in a backend independent way using a few methods: + +### TestFixture::addHost($name, $state, [ObjectFlags $flags], [array $properties]) + +The addHost method adds a host with the name $name and the status $state (0-2) to your testfixture. When no ObjectFlags +object is provided, the default flags are used (not flapping, notifications enabled, active and passive enabled, not +acknowledged, not in downtime and not pending). The $properties array can contain additional settings like 'address', + an 'customvariables' array, 'notes_url', 'action_url' or 'icon_image'. + +Subsequent addToHostgroup and addService calls will affect this host (so the service will be added to this host) + +### TestFixture::addService($name, $state, [ObjectFlags $flags], [array $properties]) + +The addHost method adds a service with the name $name and the status $state (0-3) to your testfixture. When no ObjectFlags +object is provided, the default flags are used (not flapping, notifications enabled, active and passive enabled, not +acknowledged, not in downtime and not pending). The $properties array can contain additional settings like an +'customvariables' array, 'notes_url', 'action_url' or 'icon_image'. + +Subsequent addToServicegroup calls will affect this service. + + +### ObjectFlags + +The Objectflags object encapsulates the following monitoring states with the following default values: + + public $flapping = 0; // Whether this host is flapping + public $notifications = 1; // Whether notifications are enabled + public $active_checks = 1; // Whether actice checks are enabled + public $passive_checks = 1; // Whether passive checks are enabled + public $acknowledged = 0; // Whether this object has been acknowledged + public $in_downtime = 0; // Whether this object is in a scheduled downtime + public $is_pending = 0; // Whether this object is currently pending + public $time = time(); // The last check and statechange time + +ObjectFlags can either be created using new ObjectFlags([$ageInSeconds]) and directly modify the attributes or by +calling one of the following factory methods: + + + ObjectFlags::FLAPPING() + ObjectFlags::PENDING() + ObjectFlags::DISABLE_NOTIFICATIONS() + ObjectFlags::PASSIVE_ONLY() + ObjectFlags::ACTIVE_ONLY() + ObjectFlags::DISABLED() { + ObjectFlags::ACKNOWLEDGED() + ObjectFlags::IN_DOWNTIME()