Yesterday I did some major studying on Gherkin and trying to figure out how it would integrate into our current workflow at work. In a meeting we had on Thursday, all the players involved agreed that test-cases at the beginning of a story would be beneficial. I've been through this before when I was a QA lead, and yes, Developers tend to do a lot better job coding when they know what QA is going to be testing for. Now I'm on the Development side, looking back at the QA side and trying to help out. Here are our discoveries.
Gherkin
So our developers are pushing really hard for Gherkin and it has some major win qualities. It's simple, free-formed, flexible, and you can pretty much copy and paste it into anything. It has a form that is like:
Scenario: Searching for Selenium should yeild seleniumhq.org as a result.
Given I am at google.com
When I type 'Selenium' in the search box
And I click the search button
Then I should see seleniumhq.org in the results below.
This is wonderful, because in a fixed format like this we can then use something like Behat to parse it and run automated tests that are "functional," meaning, we can piece them together. We already have the concept of functional in our Selenium tests, but that's not the point. Gherkin can go anywhere and can be read by anyone. Selenium-rc looks like this:
<?php
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("http://www.google.com/");
}
public function testMyTestCase()
{
$this->open("/");
$this->type("id=gbqfq", "Selenium");
$this->click("id=gbqfb");
$this->assertTrue($this->isTextPresent("seleniumhq.org/"));
}
}
?>
The majority of QA people are going to take one look at that and go WTF MATE! So it's a universal language everyone can write. Works great for us lazy developers!
TestLink
However, then we throw TestLink into the mix and we have a whole new set of problems. The QA people are now going, Where do I put this freeform peice of text that I just wrote? See TestLink was designed to get into the nitty gritty detail of testing a feature. It works very well with another tool called Case Complete because they both go by the same model. That is "For every action, you must have a corresponding reaction." For instance, lets take the test case above and translate it in such a way you would enter it into TestLink.
Title: My Test-Case Summary: Searches Google for Selenium Preconditions: None Steps:
- Navigate to http://google.com.
- Type "Selenium" into the 'search' field
- Click the 'Search' button.
Expected Results:
- You should see a page with a Google Logo and a search text box.
- "Selenium" should appear in the 'search' field
- A search result for 'seleniumhq.org' should be in the results list below.
This is the defacto way that most test-cases are written in our company today. The reason we don't want to take TestLink out of the mix is because it's somewhat robust reporting. You can see what tests passed and failed in an instance and you can see who passed and who failed it, when, and for what reason. You also can see trends. These are very powerful tools, not only in the QA world, but for the business as a whole.
Translation
As you can see, there can be a translation effort from Gherkin to TestLink's standard. We see the similarities. We would have to make "Given" the "Preconditions", each of the "When" a single step, and only the final step would have an "Expected Result". Now all of this translation can be done through code because Gherkin is a language specification. The problem that lies here is that QA is now editing test-cases outside of the tool they are using to test in, Making it very odd to say the least. This also means that we should be able to translate it in reverse as well, but because of the ridged style of TestLink's entry, the test-cases may not translate well. Also you see I would have to tweak the way I write my test-cases in TestLink so there is always a precondition.
Final Thoughts
I really don't know which way to go yet. As a Developer I can see the advantages to Gherkin and all it's glory and help. We could package it with the stories themselves and not necessarily off in it's own realm like test-cases today in TestLink. On the other hand, if it causes disruption in the flow and logic of QA's everyday task... is it really worth it to benefit of not having to "translate" the tests between departments. Look for a follow-up post in the future on this subject.