I had never heard of CasperJS before, and if you haven't either you might want to go check out the CasperJS site then come back here. In short, CasperJS has a couple things going for it:
- It's very fast. I haven't run any benchmarking tests but it's clearly much faster than Behat + Selenium
- It's built on top of the excellent headless WebKit, PhantomJS
On the other hand, Behat is a more mature framework and has the advantage of being written in PHP, plus, there is the outstanding Drupal Extension which provides Drush integration and simplifies content and user creation / deletion. For beginners you also have a library of tests to reference in the Drupal.org BDD tests project a.k.a. "Doobie".
With CasperJS, Drush is not available to us. But for our tests to really be useful, we'll need a way to access Drush during the test process, and we'll need a way to automatically create our own users and content and do other setup/teardown tasks. Clearly we don't want to write dozens of steps for submitting user/content creation forms; we want to use Drush. This could be done by placing a list of Drush commands in
teardown.sh bash scripts, but this still doesn't solve the problem of accessing Drush in the middle of a test. So how can this be done?
Run Drush in the browser
Drush has a (probably not that well-known) command for running a web server on your machine:
drush runserver. It turns out, extending the
runserver command to expose Drush as a web service is really easy; instead of serving up a Drupal site with
runserver, we will provide interaction with Drush.
EDIT: This is now a full-blown project, Drush REST API over on Drupal.org.
You can try it out for yourself. Grab this
composer.json file and place it somewhere on your machine. Then switch to that directory and run
composer install; this will download a customized Drush install into
vendor (you can see and contribute to the pull request to Drush 7 here). Now run
vendor/bin/drush runserver --drush-server; then visit
https://localhost:8888/core-status --json and you'll see the output of Drush's core-status command in your browser, in JSON format, it should look something like:
drush runserver --drush-server command, any time you pass the
drush cc all) will return a JSON encoded success / error message depending on their results. For example, visiting
https://localhost:8888/made-up-command will return a 500 error code along with:
This is great because in CasperJS, you can, for example, test creating some content then call
drush watchdog-show --format=json to check the logs and see if an email was correctly generated and format - much like you can with Behat.
Even better, this means that now with we can include a
setup.js file. CasperJS lets you specify
--post options to include tests that run before and after your main test suite.
So we can do
casperjs test --pre=setup.js --post=teardown.js /path/to/tests/dir. Then in our
setup.js file, we can have code like:
Tip: give unique names for your test users based on the test that's being run (i.e. "testOnlineHelpAdmin" instead of "testAdmin"). Why? Because with Grunt and the grunt-casperjs plugin, you can run CasperJS tests asynchronously. Having different usernames per test will allow you to avoid collisions with CasperJS trying to simultaneously use the same user account for multiple test scenarios.
Finally, here's an example test - a few simple checks that only users with the Administrator role can access our Drupal site's "Online Help" is below:
Running the test with
casperjs test tests/casperjs/online-help.js --pre=tests/casperjs/helpers/setup.js --post=tests/casperjs/helpers/teardown.js --uri=mysite.localhost then results in this output:
Test file: tests/casperjs/tests/online-help.js
# Restrict access to online help to Administrators only
PASS Anonymous user denied access to restricted node.
PASS Login form present on Access Denied page
PASS School Administrator user denied access to restricted node.
PASS Loaded login page.
PASS Admin can access Online Help
PASS Login form not present on handbook page
Putting it all together with Jenkins CI
Behavior driven development is a great software development process. You can spec out a feature, write the tests for it, then write the code to implement the feature. Or you can write tests against an existing set of features to catch regressions.
One thing I've learned, though, is that unless you have a policy for running tests, and enforce that policy, the tests won't get run; they'll eventually become out-dated, and then your tests are no longer useful.
This is where Jenkins comes in. While it's out of the scope of this post to write out how to set up Jenkins, this guide, Drupal Continuous Integration with Jenkins is really helpful and you can be up and running within a few hours. With a little bit of work, I configured a Jenkins instance so that on each commit that's pushed to our git repository, Jenkins will (1) import our
@prod database and files to our
@test instance on Acquia Dev Cloud (check out this gist for an example python script that will poll Acquia Cloud to see when the DB import is done, so you don't start your tests before the DB import task finishes!), (2) check out the git branch that was committed to in the
@test environment, (3) run some setup tasks using Drush, (4) execute the CasperJS tests, and (5) run some teardown tasks. If any part of this fails, e-mails are sent out to the project manager and the committer with details about the failed build, so it can be fixed.
Questions? Ideas on how to do things differently? Please let us know in the comments.
I admire your solution. It is just what I have been looking for. Have you looked into reporting? Basic stuff like Test Cases ran and failures. Is the email the only notification of a failed test?
Sorry for the delay in replying. You should take a look at this PR: https://github.com/Lullabot/drupal-boilerplate/pull/26
Jenkins will provide the full console output, and it will let you look at the XML result files generated by CasperJS so you can see exactly where errors occurred.
Some great ideas here. I've been using CasperJS for Drupal integration testing for a while now, and had planned to integrate this with Jenkins soon. I hadn't even thought about running the tests asynchronously, but that would reduce the amount of time my tests take significantly.
I'm currently creating all my test content and users by manually submitting forms via CasperJS which is quite a slow way to do it, so using drush to reduce the amount of time the whole test-suite takes is appealing. However AFAIK, you can't use drush to create specific test content (only dummy content using devel generate), so I think I'd have to write a custom module to create the content anyway.
Thanks for your article... great to see a post on an alternative to Behat in a Drupal context.
I have a question: rather than use Drush's runserver, would it be possible to achieve a similar integration between Casper and Drush by using node.js to execute Drush stuff via shell commands?