I detest monkey work and I’ll do anything I can to avoid it.
I attribute this intense, emotional reaction to a job I had at age 16, where I worked at a plastic factory inspecting box after box of clear vials to look for the defects (e.g. air bubbles, carbon black, mechanical indentations, etc). Overall the experience was mind numbingly tedious, and it inspired me to work hard through the rest of high school to ensure that I would never have to do that type of work again!
Fast forwarding many years later to the start of my career in Drupal web development…
On one of my first projects, I had to add some additional logic at the end of the customer checkout process on a Drupal 6 site using Ubercart. I started having flashbacks of the plastic factory experience while my vision became blurred staring at my monitor after entering the 100th test order over a 10 hour period. At least 80% of my time was being wasted clicking through each page (store -> product -> add to cart -> checkout -> review -> complete) and entering information fake billing, shipping, and credit card information (24 fields in total). If only I had known about Selenium IDE at that time, I could have reclaimed most of that time and focused my attention on the one thing that truly mattered: testing my code. Instead, I had wasted most of my mental RAM on the monkey tasks.
This particular example may seem a bit extreme. However, the reality is that there are many points along the Drupal development cycle where a significant amount of human resources can be spent detecting and/or fixing issues (examples: adding new features, upgrading modules, applying patches, exploring module functionality, etc). And without proper testing tools and systems in place, it’s possible to enter a vicious cycle where changes introduce new bugs that become even harder to track down by hand.
The flip side is also true. If we as a community start investing in testing tools like Selenium, we can start to reclaim time and mental RAM. And like compound interest, reinvesting that back into further improvements will result in even larger returns. In the case of web development, that means more time spent building bigger and better websites and less times slogging through a QA backlog.
Let’s get started!
Selenium at a Glance
The Selenium homepage says it best: Selenium automates browsers. In short, anything that a user can do in a web browser on a web page (click, select, toggle, submit, etc), Selenium can be programmed to repeat.
Selenium comes in two flavors: Selenium IDE (a firefox plugin) and Selenium WebDriver. The firefox plugin allows one to record your actions and play them back directly in firefox, while the WebDriver approach requires one to write tests directly into code, which opens the door for integration with Drupal’s simpletest framework. Both methods are extremely useful and will be covered in this article.
Selenium is not only useful for back end developers. Site builders, front end developers, trainers, content managers, etc can all find places where they can offload some of the more repetitive tasks into a series of Selenium tests that they can re-use over and over again.
Selenium is useful at all stages of a websites lifecycle. Examples:
- Development (i.e. eliminate repetitive tasks during a build out phase).
- Deliverables (i.e. prove things work)
- Dependability (i.e. prove things still work)
The remainder of this article will expand on these three items.
Authors Note: although the information below is highly specific to Drupal, the principles can be applied to many other web based technologies and communities.
Level 1: Development
The goal at this level is to reduce the time it takes to create a quality product. This is where Selenium truly shines for newcomers because its the easiest way to get a quick and significant return on one’s investment of time. Here are several examples where one could get started:
- Testing the Drupal Commerce checkout process.
- Testing webform, contact, and form submissions.
- Testing that rules are firing off properly on certain events.
- Testing that ajax requests are returning the correct data.
- Testing exposed filters on a view.
- Testing adding/updating/deleting users and content.
- Testing permission differences by masquerading between several different user roles.
- Etc.
Hopefully this effectively highlights the speed advantage of using Selenium. However, there are several other factors that we wish to highlight:
- Consistency: Every iteration is exactly the same, eliminating human error out of the equation.
- Shareable: Each test can be distributed to colleagues and clients.
- Version Controlled: Each test can be committed to a repository where changes can be tracked.
- Repetitive Stress Syndrome: One mouse click now replaces a significant amount of typing.
- Focus: One can monitor important aspects of the system instead of wasting time entering in data.
It’s also worth noting that Selenium doesn’t limit you to just mouse clicks and entering in form data. There are at least 100 different commands that you can use to accomplish everything from verifying elements appear on a page to verifying that a particular cookie is present.
Level 2. Deliverables
In the user story paradigm, acceptance tests are attached to each story to prove that the delivered code achieves the intended goal. Additionally, clients typically need some level of training so that they can effectively complete day to day operations with the new product. Both processes (acceptance and training) can be a largely manual process. However, it is possible to leverage Selenium to reduce the time to accomplish each.
Here are just a few tests one could create in Selenium:
- Users of role X have access to premium content.
- Users of role Y DO NOT have access to node types A and B.
- Customers need to register before completing the checkout process.
- Users of role X can edit all fields on content type Y except for field Z.
- Logged in users see a personalized message at the top of their screen.
- Users of role X are directed to /page1 when they login.
- Rule X fires off when a user clicks submit on /page1.
- Content type X has 3 related articles displaying in the right sidebar.
- Etc
In terms of the specifics of a client training, one could simply pre-record all of the steps into a series of Selenium tests. That way nothing is missed and the trainer can spend more time fielding questions from clients while clicking through each step of each training process. In fact, the client could be provided with the tests so that they could step through it themselves outside of the training sessions.
Further Refining the Acceptance Testing Model
To make the acceptance testing more systematic, one could adopt the following workflow.
- Each client request becomes a user story.
- Each user story becomes a ticket that includes a series of tests.
- Each ticket number corresponds to a Selenium test file.
- Each test file is committed to the code repository.
- Each test is run and verified prior to closing out each ticket.
- All tests are run and verified prior to delivering the project to the client.
This may seem like a lot of work at first until one compares it to one of the many alternatives. In some cases, there simply are no tests and therefore there is no assurance that the final product works as designed. In other cases, these tests are all completed manually and the results never recorded. And if they are recorded, it’s not necessarily quick and easy to repeat the entire process again in the future to ensure that the the deliverable is still working.
Which leads us to the final section…
Level 3: Dependability
While a project may have a specific and definable delivery date, the truth is that websites are never done. They are living things that evolve with changes in technology, business needs, customer needs, etc. That said, we need a means to incorporate these changes into a site without breaking what has already been built. This is where things can get tricky because, without automation, it can be very time consuming to re-audit the site by hand after each and every change. And given budget and time constraints, this can lead to risky behavior where an audit is simply not performed at all.
I have a personal example to share. I was asked to update an extremely old website that had not been updated in over 6 months. In order to get a new feature in place, I had to upgrade the workflow module to its latest version. However, in doing so, a bug was introduced that impacted the 50+ custom rules that were responsible for sending out specific, targeted emails during every state change along the approval process.
If Selenium test coverage had already been in place, I could have easily identified this defect before it shipped out the door. However, I completed the upgrade and achieved the desired outcome on the new client request. A quick spot check did not indicate any issues and there was no obvious paper trail to indicate that I should verify all the rules were still operational. Unfortunately, the bug hit production and it took a significant amount of time (10+ hours) to trace the bug back to an incompatibility in the new versions of rules and workflow.
My personal embarrassment aside, these types of situations will simply become more and more common on in Drupal projects. Client websites are getting larger and more complex, more modules are being used, new technologies are integrating with Drupal, development teams are getting larger and more distributed, etc. The point is, there will be a point sometime in the near future where it’s going to be very difficult to stay competitive in the Drupal community without some form of automated test coverage in place.
Enter Selenium WebDriver
While Selenium IDE is great at running tests directly in a local firefox browser, it too can become extremely laborious to use when one has to run hundreds of tests at a time to confirm that new features do not break old features. In these instance, Selenium WebDriver is the more appropriate choice because it can run tests across multiple browsers, offload the process to a dedicated test server, create reports to summarize the results, and more.
There are many differences between Selenium IDE and WebDriver that you should be aware of before you embark down this path. The most notable one that you’ll face initially is that you can no longer easily record tests through the web browser and store them as HTML. Instead, you’ll have to write tests using one of several languages (php, java, etc) as well as use a different triggering mechanism to initiate tests (e.g. drush, Drupal simpletest UI, etc). To get some familiarity and experience with this, I’d recommend watching some of the youtube videos on the Drupal Selenium project page and then installing the module to try out some of the example tests.
Further Refining the Acceptance Testing Model
In the previous section on deliverables, I outlined a basic way that one can start using tests as a means to successfully deliver a project. But in order to continuously develop a website AFTER the initial delivery, we need to enhance this model a little more. Ones testing workflow has to be modified to the point where each commit (or branch merge or manual request) triggers a process to run through each test again to ensure they are still passing.
The point I’m trying to underscore in this section is that in order to keep using Selenium on these larger projects, we need to automate the usage of Selenium itself. By being able to trigger the tests automatically (versus using the IDE plugin, which requires human intervention to start/stop) we can extract the most amount of value out of this tool. And at the end of the day, we can spend more and more of our valuable mental RAM working on creative processes versus slogging through tedious tasks.
Summary
Hopefully I’ve provided enough motivation to get started. I know it can be an overwhelming amount of information to take in at first, but Rome wasn’t built in a day. The advice I’ll give is the very advice we are following at NEWMEDIA: keep focusing on systematic and incremental improvements. Specifically, start with investing 5-10% of your work time to learn and incorporate these tools where appropriate. Eventually that time invested is gained back many times over and some of it can be reinvested to learn even more, which starts a virtuous cycle and benefits us in many ways: as individuals, as a company, and as a community.