UA Testing with Selenium and PHPUnit
Last week I spoke at php[tek] 2013 where I explained to people how to get started with Selenium IDE to record user interaction with the web interface, convert them to PHPUnit testcases and automatically execute them on multiple browsers on multiple platforms.
The feedback I got was awesome, you're all a great crowd! But on twitter I also received a bunch of questions regarding how to set up multiple platforms and why I used Windows in my presentation to deploy to.
So today I deceided it was time to write a full article on this subject.
What is Selenium?
Selenium is a tool that allows you to continuously test user interfaces of web applications. The most common usages for Selenium testing are the following:
- testing elements are (not) on the web interface
- users can't break out a certain flow on the web interface
- calculated values are correct after modification
- errors appear on screen when mistakes are made by users
- reported issues are valid
In general we call these type of tests User Acceptance Tests or UAT and are all focused from the point of the end-user, the person using the web interface to accomplish a certain goal.
Why are they important?
UAT have their own right to exist. Just like regular unit, performance and stress tests they have their own agenda and are adressing a particular part of your application that needs testing. All to prevent your customers/visitors from finding issues, bugs or just unfunctional pieces on your web application and loose their trust in your products or services.
Therefor it's always good to invest in the "visibile" part of your web application. Especially when using javascript, you want to ensure it always works as intended.
Disclaimer
Selenium tests are in no way a replacement for regular unit tests. Their focus is on generated output of your web application within a browser. Unit tests are still necessary to ensure the logic of your application is not broken when making modifications or adding new functionality!
Setting things up
You can write your own Selenium tests by hand, but the easiest way is to use the Firefox plugin to recored Selenium tests.
Start Firefox and download the plugin at the Selenium IDE download page.
When you are asked to "restart", wait a little and scroll down on the downloads page to PHP Formatter and install this formatter plugin as well.
Once all is installed, restart your Firefox browser.
Now you're all set to get started creating UA tests.
Overview of what we're doing here
The Selenium IDE plugin allows you to record all events you make on the web application you want to test. These can be mouse clicks, entering text inside input fields, pressing enter, drag-and-drop and so much more. All activities are being captured using this recording tool.
For my example I used a simple project that's still a work-in-progress. There were already some issues reported in the bug tracker and I'm showing here how to get started with confirming the issue is truely an issue. Then we fix the issue and rerun our test to see it resolves.
Next we're going to export the Selenium test as PHPUnit test and run it as we'd normally run our unit tests. We can use our IDE, command line (as I will use) or a continuous integration (CI) system to run these tests over and over again.
Since we have a Selenium test we can modify our test case to verify our application works on multiple browsers.
Step 1: Verify an issue from bug tracker
In many cases you're already working on a project and you have issues to resolve. The easiest way to get started with UAT would be to pick an issue and verify the issue is genuine.
Go to your bug tracker and pick an issue. Look at the detailed description of the issue as this will be your test goal.
Now that we have a full understanding of the issue, we can start recording a Selenium test. Our goal is to ensure we can reproduce the issue which is hard sometimes as reports (like this) can specify items that are missing.
Therefor we're going to add "assertions" in our Selenium test of items we expect to be present, but aren't at the moment. This may sound weird, but this way we think ahead on how we need to resolve the issue if the reported issue is genuine.
Step 2: Fix the reported issue and test again
Once we fixed the issue, we can then rerun our test and see it succeed.
We now save the Selenium test as .html file.
Step 3: Convert to PHPUnit test
In the menu of your Selenium IDE window, you can select "File" -> "Export as" -> "PHP (PHPUnit)" and store this in the application's tests directory. I have chosen to create an additional directory called "seleneum" so I can differentiate unit tests for the project.
Step 5: Set up Selenium Server
In order to run your exported Selenium test in an automated, headless way you need to set up a computer that will run the Selenium Server service (as our unit tests will try to contact it for running the tests within a browser). Selenium Server is a Java daemon that will continuously listen to incomming calls from unit tests, so no matter if you run it manually from your IDE or command line, or you have a CI system to run your unit tests automatically, it should be able to contact the system.
I have chosen to use a Windows 7 64bit virtual machine as it supports the most essential browsers like Internet Explorer, Firefox, Google Chrome and others, but as the Selenium Server is a Java service, you can run it on Windows, Linux and Mac OS.
Go back to the Selenium HQ website and download the selenium-server-standalone-X.X.X.jar where X.X.X represents the current version of the tool.
Once downloaded, you can set up your test machine for receiving Selenium Test calls. I created a simple startSeleniumStandalone.BAT file that I launch during startup.
"C:\Program Files (x64)\Java\jre6\bin\java.exe" -jar "C:\Tools\Selenium\selenium-server-standalone-X.X.X.jar" --port 12666
This will launch Selenium Server on port 12666 and will execute for each browser defined a new window.
Step 6: Modifying your PHPUnit Selenium Test
When you first export your Selenium test to a PHPUnit Testcase you need to modify a couple of things before you can execute it. This is the rawexport from Selenium IDE:
<?php
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("http://www.theialive.com/");
}
public function testMyTestCase()
{
$this->open("/");
$this->click("link=login");
$this->waitForPageToLoad("30000");
$this->type("id=email", "dragonbe+tek13@gmail.com");
$this->type("id=password", "test1234");
$this->click("id=signin");
$this->waitForPageToLoad("30000");
$this->click("link=Test demo");
$this->waitForPageToLoad("30000");
$this->assertEquals("Done", $this->getText("xpath=//th[5]"));
$this->click("link=[EDIT]");
$this->waitForPageToLoad("30000");
$this->assertTrue($this->isElementPresent("id=done"));
$this->click("link=sign off");
$this->waitForPageToLoad("30000");
}
}
?>
First we need to rename our test class to something more meaningful, like
MarkTaskDoneTest.class MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase
Secondly we need to modify our setup so it can connect with our Selenium Server and run Internet Explorer.
protected function setUp()
{
$this->setBrowser("*iexplore");
$this->setBrowseUrl("http://www.theialive.com/");
$this->setHost('192.168.10.33');
$this->setPort(12666);
}
Lastly we modify our test method to something meaningful like
testMarkTaskAsDone:public function testMarkTaskAsDone()
We also add the following line to our test so on our test machine, all will be executed in full screen. Not really a required feature, but allways nice to see stuff running automatically.
$this->windowMaximize();
So our full test class would look like the following:
<?php
/**
* Class MarkTaskDoneTest
*
* @group SeleniumTest
*/
class MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase
{
protected function setUp()
{
$this->setBrowser("*iexplore");
$this->setBrowserUrl("http://www.theialive.com/");
$this->setHost('192.168.10.33');
$this->setPort(12666);
}
public function testMarkTaskAsDone()
{
$this->windowMaximize();
$this->open("/");
$this->click("link=login");
$this->waitForPageToLoad("30000");
$this->type("id=email", "dragonbe+tek13@gmail.com");
$this->type("id=password", "test1234");
$this->click("id=signin");
$this->waitForPageToLoad("30000");
$this->click("link=Test demo");
$this->waitForPageToLoad("30000");
$this->assertEquals("Done", $this->getText("xpath=//th[5]"));
$this->click("link=[EDIT]");
$this->waitForPageToLoad("30000");
$this->assertTrue($this->isElementPresent("id=done"));
$this->click("link=sign off");
$this->waitForPageToLoad("30000");
}
}
That's it! Now all is ready to run our tests using PHPUnit on the command line.
And this is how our unit tests run on our test machine:
Step 7: Testing with multiple browsers
Running these tests is fun, but they still test a single browser. A better way would be to set up a base TestCase where you set all browser configurations and extend this base TestCase whenever you create a Selenium Test.
File: tests/SeleniumTestCase.php
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class SeleniumTestCase extends PHPUnit_Extensions_SeleniumTestCase
{
const TEST_HUB = '192.168.10.33';
const TEST_PORT = 12666;
const USERNAME = 'dragonbe+tek13@gmail.com';
const PASSWORD = 'test1234';
const BASURL = 'http://www.theialive.com';
public static $browsers = array (
array (
'name' => 'Internet Explorer 8 on Windows 7',
'browser' => '*iexplore',
'host' => self::TEST_HUB,
'port' => self::TEST_PORT,
),
array (
'name' => 'Firefox on Windows 7',
'browser' => '*firefox',
'host' => self::TEST_HUB,
'port' => self::TEST_PORT,
),
array (
'name' => 'Google Chrome on Windows 7',
'browser' => '*googlechrome',
'host' => self::TEST_HUB,
'port' => self::TEST_PORT,
),
);
protected function setUp()
{
$this->setBrowserUrl(self::BASURL);
}
}
Now we just need to extend from our base test case to continuously use all configured browsers.
File: tests/selenium/TaskTest.php
<?php
/**
* Class MarkTaskDoneTest
*
* @group SeleniumTest
*/
require_once 'SeleniumTestCase.php';
class MarkTaskDoneTest extends SeleniumTestCase
{
public function testMarkTestAsDone()
{
$this->windowMaximize();
$this->open("/");
$this->click("link=login");
$this->waitForPageToLoad("30000");
$this->type("id=email", TestCase::USERNAME);
$this->type("id=password", TestCase::PASSWORD);
$this->click("id=signin");
$this->waitForPageToLoad("30000");
$this->click("link=Test demo");
$this->waitForPageToLoad("30000");
$this->assertEquals("Done", $this->getText("xpath=//th[5]"));
$this->click("link=[EDIT]");
$this->waitForPageToLoad("30000");
$this->assertTrue($this->isElementPresent("id=done"));
$this->click("link=sign off");
$this->waitForPageToLoad("30000");
}
}
Running it on CLI will execute 3 times this test as we're testing 3 browsers: Internet Explorer, Firefox and Google Chrome.
On our test machine it would look something like this:
Closing Remarks
Authentication
When running Selenium Tests and your application requires authentication, the best thing you can do is to have your test do the following:
- log into your application (carefull, passwords are plain text!!!)
- run your tests and assertsions
- log out of your application
This way your tests always behave the same.
Multi platform setup
As good developer I not only test on different browsers, but also on multiple platforms. I use virtual machines for this that are always running and I use the following setup to achieve this:
Dom changes
Be careful as Selenium tests heavily depend on the DOM of a web application. Any changes there can have an effect on your Selenium Tests and might result in failing tests.
Recommended reading
See my presentation as it was given at php[tek] 2013
These books I can highly recommend reading!
Comments
Post a Comment