This is the second in a series of articles about using automated testing tools on a Canvas-based Web application. Each article covers a different testing tool or technique. (See the first on WebDriver.)
The specific application I’m testing is sized for a tablet, and uses a combination of a Canvas taking up roughly 2/3 of the space and a set of JQuery Mobile widgets taking up most of the remaining 1/3 of the space (there’s also a small space on top showing a read-only “status bar” with key statistics). The basic concept is an editable diagram in the Canvas, with specific edit controls in the JQuery Mobile area. You can hover, drag, and click parts of the diagram in the Canvas, which can change what’s displayed in the edit area, and changes you make in the edit area can add, remove, or change elements of the diagram in the canvas as well as updating the values shown in the status bar.
My goals for the automated testing are:
- Make it easy to write tests
- Run the tests in a real browser, to ensure I’m testing what a user would actually see
- Make it easy to run a suite of tests and drill into specific test failures
- Make the tests run fast enough that it’s not terribly onerous to run before every commit
- Be able to verify the visible state of the HTML showing on the screen (e.g. in the edit area, in the status area)
- Be able to verify the underlying diagram model that dictates what is drawn to the Canvas. (I haven’t attempted to go as far as testing the actual Canvas state — as in the pixel color at some specific coordinate. The bugs I get aren’t that the Canvas is drawing something that does not accurately represent its model state.)
Summary of Geb
Good: All the advantages of WebDriver (which it uses under the covers), plus lots of syntactic sugar for writing tests, plus you can write functional tests in Spock or traditional tests in JUnit, TestNG, or etc. using the lovely Geb syntax. (WebDriver launches the browser and runs through the tests, clicking on various things on the screen, waiting for screens to appear, executing JavaScript as needed, etc.) Supports multiple browsers.
Bad: There’s no built-in way to record Geb tests by interacting with an existing Web site. Inherits the limited WebDriver support for interacting with a Canvas (requiring JavaScript workarounds). The documentation doesn’t include the single simple step you need to drive single-page applications.
Bottom Line: All the advantages of WebDriver, plus fixes a lot of the WebDriver limitations. If I was considering WebDriver, I’d definitely use Geb instead.
Detailed Review
Instead of providing many language bindings like WebDriver, Geb focuses on producing an outstanding syntax using Groovy. On the one hand, it means that you have to learn and use Groovy. On the other hand, the syntax is really amazing. I think it’s worth the trade.
However, most of the Geb examples assume you are using Spock and writing functional tests. The tests I’m writing are probably best considered integration tests — they’re long sequences of browser activity and I expect it all to work. I find this to be a better fit for JUnit (run a long sequence of mixed actions and validations) than Spock (break up the tests into small given/when/then type groups). So for my purposes I chose to write JUnit tests.
Installing Geb
I chose to set up a Maven project so I could run my tests via mvn test as well as generating an IntelliJ project and using IntelliJ to write my test code. The POM has several entries to include Geb, Selenium, JUnit, and Groovy, and to enable the Groovy compiler:
Geb JUnit Maven 3 POM
You should be able to run mvn install on that and at least confirm that the plumbing works.
Learning Curve:
Geb makes heavy use of Groovy closures, and the ability to invoke properties and methods that weren’t explicitly defined. At first, I just followed the examples in the Book of Geb. Eventually things started to make a little more sense.
I’m not going to describe all the Geb syntax here, as the manual does a decent job of it. I’m just going to hit the things the manual didn’t cover for me.
Some of my issues were:
- The manual does clarify how to get Geb to work with single-page applications. The issue for me is that page transitions are animated and therefore are not instantaneous, so the “at checker” ran before the new page was actually visible (and therefore failed).
- I had to piece together the config file since I didn’t see a complete example
- I wasn’t quite clear on how to spread my tests and page object across Groovy source files
- I wasn’t quite clear on whether to use inheritance or modules to reflect that fact that the Canvas is visible for every “page” (meaning a JQuery Mobile page) of the application
- It wasn’t obvious how to handle a Canvas click that caused a page change, since the basic click operation in WebDriver is broken for the Canvas (as described in the previous article)
- I wanted to automate the recording of tests like I did for WebDriver directly
Config File
Here’s my Geb config file for Firefox. The last two lines are necessary if you don’t want to configure them as system properties for all the different ways you might run the tests. Note the code within the “driver” block is running against the WebDriver Java API.
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxProfile
import org.openqa.selenium.Dimension
driver = {
def profile = new FirefoxProfile()
profile.setEnableNativeEvents(true)
def driver = new FirefoxDriver(profile)
driver.manage().window().setSize(new Dimension(1040, 850))
return driver
}
baseUrl = "http://localhost:9393/"
reportsDir = "target/test-reports/geb"
Single Page Applications
Since the browser never requests a new URL, there is no need to use “to” or “via” or set the URL for a page. You can simply point the browser to the base URL with go() and then navigate around within that.
It doesn’t mean you don’t use multiple “pages” within the Geb tests, though. I created a Page object for every JQuery Mobile screen. I just handle the page transitions via “at SomePage” rather than “to SomePage” to tell the test that I now expect a different JQuery Mobile screen to be showing even though the URL hasn’t changed.
My first stab at an “at checker” looked like this:
class SomePage extends Page {
static at = { $("#pageID").displayed }
}
That doesn’t account for the page animation, though. The fix turns out to be easy:
class SomePage extends Page {
static at = { waitFor { $("#pageID").displayed } }
}
That gives the page 5 seconds (by default) to show up, more than enough. Putting the wait logic in the “at checker” is important because some other things call the at checker as a side effect, and that would have to be avoided if the at checker itself didn’t handle the waiting.
Many Pages Sharing a Canvas
Since my Canvas is always visible, no matter what JQuery Mobile page you’re on, I ended up creating a base Page class with all the Canvas logic (clicking on the Canvas, clicking various widgets within the canvas such as the menu button or toolbar buttons shown there, etc.). All my other Page objects inherited from that one (except the dialog pages, because you can’t interact with the Canvas while a dialog is showing on the screen).
Handling Canvas Clicks
I put my same JavaScript workaround for Canvas clicks in my base Page class:
class BasePage extends Page {
def clickOnDiagram(x, y) {
interact {
moveToElement($('#diagramCanvas'), x, y)
}
js.exec x, y, """
var evt = $.Event('click',
{ pageX: "+arguments[0]+", pageY: "+(arguments[1]+55)+" } );
$('#diagramCanvas').trigger(evt);
"""
}
}
Then I wanted my toolbar clicks to automatically change the current Page and run the new page’s at checker, just like you can do with regular clicks in Geb. I ended up invoking the Canvas click manually, and then returning some innocuous page element that Geb could use its click-and-change-page logic on.
Normal link or button that forwards to a new page:
static content = {
someLink(to: NewPage) { $("#someLink") }
}
Toolbar button in the Canvas:
static content = {
toolbarButtonFoo(to: NewPage) {
clickToolbarFoo()
$("#someBoringElement")
}
}
def clickToolbarFoo() {
clickOnDiagram(50, 600);
}
Since the content entry returns a legitimate Geb Navigator object (after clicking the Canvas), you can call click() on the content entry. When you call click on it, it changes the page per the to: NewPage directive (including calling the at checker on the new page), even though the click on #someBoringElement did nothing at all.
And the test ends up looking like this:
// On OldPage someLink.click() // Now on NewPage // Automatically ran the NewPage at checker at OldPage toolbarButtonFoo.click() // Now on NewPage again // Automatically ran the NewPage at checker
That’s not strictly necessary — you could put in all the at checks explicitly, and I suppose it could make the test clearer. But I prefer to have them all run as a convenient side effect of navigating.
Source Code Organization
I ended up putting all my Page definitions in a single TestSupport.groovy file. They were all pretty short and closely related, and I didn’t want to clutter up my project with lots of tiny files. I think a consequence of this is that I needed to do a “full build” more often since the dependencies across files weren’t as obvious, but builds were fast and that didn’t bother me.
import geb.Page
class BasePage extends Page {
static content = {
... // Common Canvas-related content
}
... // JavaScript model lookups, toolbar logic, etc.
}
class PageOne extends BasePage {
static at = { waitFor { $("#pageOne").displayed } }
static content = {
... // Content specific to this page
}
}
class PageTwo extends BasePage {
static at = { waitFor { $("#pageTwo").displayed } }
static content = {
... // Content specific to this page
}
}
For my tests, I created several XYZTest.groovy files, since Maven by default runs tests with the *Test.* naming convention. It was easy to put several related tests in each file with the JUnit @Test annotation:
import org.junit.runner.RunWith
import org.junit.runners.JUnit4
import geb.junit4.GebReportingTest
import org.junit.Test
@RunWith(JUnit4)
class SomeTest extends GebReportingTest {
@Test
public void doSomething() {
go()
at DefaultPage
mainMenuButton.click()
...
}
}
Recording Tests
There’s an up side and a down side here. The up side is, I was able to record tests by including a custom “QA” JavaScript file in my app, which added listeners to buttons and links and pages and so on. It emits the appropriate Geb code to the Web console, which then I copy and paste into a test class. The down side is, it relies a lot on referencing page elements by their ID. That works fine, but it sort of misses the point of the PageObjects pattern. It’s very easy in Geb to define all the page widgets in a content block, but when recording the tests, I know that an element with a particular ID was clicked, but I don’t know what you decided to call that element in the content block of the page element.
It’s possible I could settle on a convention of always naming elements in the content block the same as their ID, but even that avoids some of the nice syntax available with Geb.
For instance, I have a screen with four items in a glorified list, but every time you go to the screen, the specific content of the four items might be different (one screen tries to pick the best four for you, another is the same screen no matter which of many categories you’re viewing, etc.). So the IDs are pretty generic, like ListItem0, ListItem1, ListItem2, ListItem3. It makes it easy to loop over them and so on.
I can set up a content entry in Geb like this:
ListPage extends BasePage {
static content = {
listItems(to: ItemDetailPage) {id -> $("#ListItem${id}") }
}
}
Then my test can click an item and cause an implicit navigation to the detail page and run the at checker for the detail page all very easily:
listItems(2).click()
But again, the test recorder doesn’t know about that, so it can only generate code like this:
$("#ListItem0").click()
at ItemDetailPage
Does it do the same thing? Yes. But it’s more lines, uglier lines, and the test would break if the list IDs ever changed — exactly the reasons the PageObjects pattern exists.
So I end up with a mix of somewhat uglier tests that I recorded, and somewhat prettier ones that I wrote by hand.
Here’s what the recorder script looks like now:
$('#diagramCanvas').click(function(e) {
if(e.pageX >= 0 && e.pageX <= 10 &&
e.pageY >= 55+590 && e.pageY <= 55+600) {
QA.recording = !QA.recording;
console.log("Recording: "+(QA.recording ? "on" : "off"));
if(QA.recording) {
console.log("go()")
console.log("at DefaultPage")
}
} else if(QA.recording) {
console.log('clickOnDiagram('+
e.pageX+','+(e.pageY-55)+');');
QA.lastClickToolbar = false;
}
});
$('a').on('click', function() {
if(QA.recording && $(this).attr('id')) {
var id = $(this).attr('id');
console.log('$("#'+id+'").click()');
QA.lastClickToolbar = false;
}
});
$('body').on('change', 'input', function() {
if(QA.recording) {
var type = $(this).attr('type');
if(type === 'checkbox' || type === 'radio')
console.log('$("label[for='"+
$(this).attr('id')+'']").click()')
else if(type === 'text' || type === 'number')
console.log('$("#'+$(this).attr('id')+
'").value("'+$(this).val()+'")');
}
});
$('#page1').on('pageshow', function() {
if(QA.recording && !QA.lastClickToolbar)
console.log('at OnePage');
});
$('#page2').on('pageshow', function() {
if(QA.recording && !QA.lastClickToolbar)
console.log('at TwoPage');
});
And a generated test:
go()
at DefaultPage
mainMenuButton.click()
$("#MainMenuNew").click()
clickOnDiagram(393, 412)
at XYZPage
$("#SomeTextFieldID").value("MyNewText")
$("#SomeEditWidgetID").click()
assert $("#SomeSpanID").text() == "MyNewText"
...
Whereas a handcrafted test might look a little cleaner:
go()
at DefaultPage
mainMenuButton.click()
newItem.click()
clickOnDiagram(393,412)
at XYZPage
nameField.value("MyNewText")
saveButton.click()
assert visibleName == "MyNewText"
...
Running Tests
Running tests is as easy as mvn test or right-clicking on a test file or test method in IntelliJ (or however else you prefer to run JUnit tests). I get a summary of tests run, passed, and failed, green or red bars, etc. I find that IntelliJ is a little better at putting the specific error in front of me for a test failure, whereas I have to hunt a little more with Maven, but either way works. It's a big improvement over plain WebDriver code.