Test Automation Frameworks

1. Linear
2. Structured
3. Modular
4. Data-Driven
5. Keyword-Driven

Test Automation Framework (TAF) is required in order to achieve later three automation levels. TAF provides a set of reusable supporting functions both for the framework and the Application Under Test (AUT). Supporting functions for framework are main application-independent, can be easily used in other products’ automation scripts/programs, e.g. string handling, file handling, variable handling and so on. The supporting functions for AUT should be framework-independent, hidden application specific implementaions to provide abstract level interfaces which the TAF recognizes as library routines or scripts.

These supporting functions can be provided in two formats : library routine or keywords. Library Routine – implementes those common-used basic functionalities which can be called by different automated testcases; Keywords – keywords are literal strings which can be recognized and processed by the TAF, its under-layer is the supporting functions for AUT or in other words component functions.

Reusability performs an important role in TAF, it’s better that we can reuse functions and even automated testcases. A concept for Keyword-Driven (so called Table-Driven sometimes) TAF abstracts automated testcases into three levels – Step Tables, Suite Tables, Cycle Tables.

• Step Tables contain steps which uses a single keyword and arguments to execute certain task. One step table can represent one testcases, or a part of bigger testcases (represented by Suite Table or Cycle Table) which have same combination of operations.
• Suite Tables mainly contain Step Tables as its operation steps, and also can be a part of bigger testcases (represented by Cycle Table).
• Cycle table may contain Suite Tables or Step Tables as its operationi steps.

This kind of division is only one alternative, basically it’s a way that tables can be reused no matter it’s Step or Suite or Cycle table.

TAF can not be designed despite of other aspects in software development lifecycle, especially it should be considered together with testing strategy, there are some suggested essential guiding principles we should follow when developing our overall test strategy :

• Test automation is a fulltime effort, not a sideline.
• The test design and the test framework are totally separate entities.
• The test framework should be application-independent.
• The test framework must be easy to expand, maintain, and perpetuate.
• The test strategy/design vocabulary should be framework independent.
• The test strategy/design should remove most testers from the complexities of the test framework.

We can not consider TAF separately out of the Testing scope, like we can not consider Testing separately out of the Software Developing Lifecycle. Different TAFs represent different views on TA, have different abstract levels and corresponding responsibilities, also have different suitable circumstances.

Modular type TAF may require the engineer has comprehensive abilities, include framework development and supporting library function implement and testcase implementation ability, they should be familiar or master some programming languages, experienced in software development practices, and also should know the product features. In order to ensure that our automation build the code (automation code) right, and also build the right code.

While we use Data-Driven TAF, at least we can separately have two responsibles, one takes care of test datas to be verified, one takes the rest of above described responsibilities. But the goal does not change, we should provide reliable automated testcases.

Keyword-Driven have more detailed divisions between responsiblities. Somebody develop and maintain the framework itself, someone takes care of supporting functions or in other words component functions, also some engineer should implement material automated testcases, e.g. Step/Suite/Cycle Tables and so on.

Reference Articles :

• Test Automation Frameworks by Carl Nagle
• Choosing a Test Automation Framework by Mike Kelly
• Automated Acceptance Tests: Building the Right Code by Bob Martin

More References Going-On :

• Test Automation is Software Development by Jonathan Kohl
• Keyword-Driven Testing by Danny Faught
• Keyword-Driven Testing Involves New Constituencies In Automated Testing: Tiny Forrester article on keyword driven testing
• Deconstructing GUI Test Automation by Bret Pettichord
• A Framework for Data-Driven and Keyword-Driven Test Automation by Pekka Laukkanen (Jan 2006)
• IBMAutomation.pdf IBM Automation presentation (STAF, STAX, WebSphere^?).
  

• "Improving the Maintainability of Automated Test Suites" by Cem Kaner (paper presented at Quality Week 97)
• "Test Automation Frameworks" by Carl Nagle
• "Using Cost-Benefit Analysis to Compare Different Test Structures for Rational Robot" by Mike Kelly