Web automation is a little trickier than most other automation because there are many combinations of browsers and operating systems and they are fast evolving too.
What do you look for before you choose a tool for web automation?
The answer may actually depend on what your organization specializes in. If you are a product company which ships applications meant only for Internet Explorer, you need not consider multi-browser support or Linux support. But if you develop outward facing web applications, you may need to test the application on multiple browsers.
Here are some factors you should consider before choosing a testing tool.
The tool:
- Should be technically sound
- Should be able to identify elements/record on all browsers.
- Should handle complexities like HTTPS, Frames, IFrames, AJAX, dynamic ids.
- Should not need tinkering with source code of tool
- Should not require hard coded waits
- Should save time and effort for teams
- Ramp up time should be minimal. Users should get productive within an hour.
- Complexities like AJAX, dynamic ids, object identification etc. should be handled by the tool instead of passing it on to testers. *
- Should run reliably across browsers and operating systems to reduce re-runs and debugging effort.
- Should not be dependent on knowledge of various other tools/technologies.
- Should need minimal maintenance of scripts/code
- Should work with existing teams instead of requiring a drastic overhaul
- Should not require your teams to change from testers to developer testers, but let them easily pick up some scripting knowledge and get functional.
- Should not require expertise in various peripheral technologies like Java, Junit, TestNG, XPath, Firebug, Browser DOM etc. to just get started.
- Should require minimal stakeholders
- Should not need developer involvement for modification to application in the name of “testability”. Dynamic ids, elements without ids, etc. should be handled well by the tool. *
- Should be easy to scale testing teams
- Should be easy to hire and add more members to your testing team. This requires the tool to be simple to use.
- Should be able to move the teams across projects and products. This means that the tool needs to be sound enough to work with various technologies and frameworks.
- Should have authoritative support available
- Should be cost effective. The following need to be considered:
- Cost of acquiring the tool
- Cost of employing capable testers who can use the tool
- Cost of maintaining test infrastructure
- Cost of authoritative support
Too often, especially with open source tools, the amount of money wasted in man hours due to limitations of the tool, incompatibility with existing expertise of team, lack of support etc. far outweighs the cost of acquiring alternative commercial tools. (Developers and testers with not much business experience invariably think that their time is not a cost to their company, and do not mind spending a week on an effort which should have lasted a day, thus wasting 25% of a month’s salary for a tool which may cost 10%)
* It is possible to just use the Sahi Controller and identify various elements reliably. Because tools like Selenium cannot record across frames, iframes, the tester is forced to learn to use Firebug to figure out what ID or XPath to use, add a line of selectFrame etc. These are very tool specific. While learning to use Firebug is an awesome skill to have, it should not be required at each step of the automation process. Adding conditional waits with knowledge of DOM is an unnecessary effort put on the tester, which can be handled by intelligent tools. Same goes for making developers add custom id generators for handling dynamic ids.
