Blog

Latest Updates. News. Insights. Ideas.

Sahi Pro V6.2.0 Released!

Posted by | Uncategorized | No Comments

Download Sahi Pro v6.2.0

Sahi Pro v6.2.0 is a major release with significant fixes and enhancements.

Significant changes are:

  • Microsoft Edge Support
  • Multi Language Support: eg. record for English and playback for Japanese
  • Passing External data to Scenario files
  • Sahi Pro Runner: playback only version of Sahi Pro for CI and build systems
  • Enhancements in reports
  • License Server: for monitoring concurrent licenses

Please have a look at What’s new in Sahi Pro v6.2.0 for details.

Tyto Celebrates 5th Year Completion With 100% Increase In Sahi Pro Licenses Sold

Posted by | Uncategorized | No Comments

It is a great pleasure to share our successes with you in our 6th year of operation. We completed our 5th year last month with 350+ customers using Sahi Pro. The number of Sahi Pro licenses sold worldwide saw more than 100% increase compared to the previous year.

Our journey started 9 years back with Sahi, an open source web automation tool focused on testers. Over the years we have continued our innovative drive in Web Test Automation, and in 2014 we launched Sahi Pro 6, the latest version of the commercial product. In this new version, we added a web based editor to easily author and manage automation scripts. This intuitive user interface also allows creation of suites and triggering parallel and multi machine playback.

Sahi Pro 6 provides Image comparison APIs and better integration with Continuous Integration systems like Jenkins. Overall, we believe that the new version plugs many of those areas where we found effort being wasted. Sahi Pro 6 significantly enhances tester productivity and allows even the not-too-technical teams to adopt and reap good ROI from web test automation.

In 2014, we ramped up our marketing efforts to broaden our reach. Through case studies, we have started highlighting how our customers, especially in the enterprise software products space, have succeeded with Sahi Pro and accomplished significant cost savings. These should be helpful for existing as well as prospective customers.

We received recognition by Gartner and Forrester in their reports, including mentions in the Gartner Magic Quadrant for Integrated Software Quality Suites – 2014 and 2013 Forrester report on Navigating The Agile Testing Tool Landscape. We got covered in major media publications, including Information Week, PC Quest, Network World, The Indian Express. These details are available in our news and events page.

Sahi Pro v6.0.0 Issues and Workarounds

Posted by | Uncategorized | No Comments

This post shall list known issues and possible fixes. All changes mentioned here will be incorporated in the next version.

Issue #1: Scripts run slower because Sahi waits for uninitiated AJAX requests also
Workaround:
Open
sahi/userdata/config/user_extensions.js
and add
// XHR wait fix start
Sahi.prototype.areXHRsDone = eval("(0 || "+(""+Sahi.prototype.areXHRsDone).replace("null==h||0==h||1==h", "1==h")+")");
// XHR wait fix end

before the line:
__sahiDebug__("user_ext.js: end");

Restart Sahi, clear browser cache before re running scripts.

Sahi Pro V6.0.0 Released!

Posted by | Uncategorized | No Comments

Download Sahi Pro v6.0.0

Sahi Pro v6.0.0 is a major release and contains many new features and bugfixes. The focus of this release is tester productivity enhancement. Some of the key features added are: new Script Editor, better jenkins integration, browser and element level screenshots, APIs for comparing images, better reports, support for globally included global_include.sah file and many more.

Full details of all changes and enhancements can be found here: What’s new in Sahi Pro v6.0.0

Play with it today and let us know if you like the new features!

Is This Your Automation Story? – From the “NEED” to “SUCCESS”

Posted by | Sahi | No Comments

Your product simplifies business critical workflows for banking organizations. Your product has complicated workflows which are accomplished by navigating through various forms and screens of your web application. Currently you are doing a lot of manual testing. You have testers who understand your domain well and are good at identifying problem areas. These testers are not programmers, but use computers to get their job done.

Sometime back you moved to faster releases. The development could cope with it (or so they believed and claimed). But testing started lagging behind. You realize that a lot of time is spent verifying older functionality. So you start skipping some parts of it. One day your friendly customer calls and says, hey, I am no longer able to edit loans as a manager. A CRITICAL BUG! Your mature, smart testing manager resists the temptation to shift blame (thank god for that!). You discuss with her and realize the manual testers are overwhelmed by the speed of releases. Bugs are slipping through and they are frustrated.

The NEED:

You realize YOU NEED AUTOMATION. And that you have no clue how to go about it.

So you ask your testing head. She says she knows this tool which she has seen used in an earlier organization. You look at the cost and blanch. You still go ahead and evaluate it. You think you will just have one tester on automation then. The testers are able to understand the tool and work with it. Unfortunately it does not support some components on your web application. You are silently relieved that you did not have to justify such a cost and make the compromise of using just one automation tester!

The EXPERIMENTING:

Next you ask one of your developers. He has heard a few things and suggests that you go with product B. You ask him if he has used it. He says, he has not, but EVERYONE seems to be using it. Many of the bigwigs are using it too. It is open source and free too! Wow! – you say. Let’s try it. The testing head is a little cautious. You are mildly irritated with her and assign the developer to do a Proof of Concept. The first trivial demo goes through. The enthusiastic developer blogs about it. He then goes on to do the POC. He needs to demo it in 2 weeks.

Demo day. You see your application run automatically through a flow on Firefox. You are impressed and happy. Now your developer takes you through the code behind it. You see code. Lots of it. In Java. And cryptic long strings as identifiers. Lots of code with JavaScript expressions as strings. This seems complicated. You worry now. Can your testers do this? You ask if it runs on Internet Explorer 9 where your customers are. The developer says the tests seem to freeze up and crash and he suspects some bug in his code which he will fix. The testers want to try their hand. Install Eclipse he says. They have never used an IDE. They are excited with the new toys. The developer is surprised they don’t know Java and are staring at him blankly when he asks them to write a TestNG test. He explains and a few testers nod. He goes on to identify elements using the developer toolbar and does some copy pasting of xpaths etc. XPath? What is that asks the tester. He explains, but is starting to get frustrated. They seem to know nothing he thinks. “Can’t we use a recorder to build our tests?” asks a tester. Since the product does not have a recorder, the developer goes on to justify it that recorders are bad and you should always code. You sense that it is a cover up for a deficiency, but wait to watch how this plays out. Especially since you like the developer and don’t want to bruise his ego (little sensing how much you will regret this later). You allow the game to be played for a couple of more weeks. You think this may help the testers also scale up some skills.

A couple of weeks later, you talk to the team. The testers say they have not been able to automate much. The developer says they do not have the skillsets. You should hire automation engineers. You try out with two automation engineers. They don’t come cheap. They seem to love XPaths and adding waits and digging deep into browser developer tools. But one of them seems to not understand the business functionality. He even has a holier than thou attitude, as if testers who understand business, but cannot automate like him, are beneath him. The other one thankfully is saner. Both diligently dive in to create a “framework”. Your existing testers are getting demotivated. They fear they may be fired. You are getting frustrated too. You think maybe you should outsource this. You talk to a few vendors. They say they know the tool but you will need a framework. The framework will cost you thousands of dollars and a few months of effort. Automation can only start after that. You start budgeting for it. But you are worried. What will I do with my current testers? This all seems unnecessarily complicated. You sense that YOU ARE SOLVING TECHNICAL AND PEOPLE PROBLEMS INTRODUCED BY A TOOL RATHER THAN YOUR OWN BUSINESS PROBLEMS.

Thankfully, you talk to your friend who heads engineering in that clinical research company in the US. He tells you about another tool. A tool which has enabled his existing testers to automate. A tool which does not need XPaths or waits. A tool which has a good recorder. A tool which has inbuilt technology to ease every bottleneck you yourself have encountered so far. And a tool which works reliably across browsers. Fast. Even on Internet Explorer, which you have not been able to reliably run on yet with tool B. (Tool B’s fan boys kept blaming that IE is cr*p and is slow). And he raves about the support team which has bailed him out every time. They even shipped an entire new build in 3 days for one of his issues.

The SOLUTION:

So you change track. You ask your tester to evaluate this tool. It is called Sahi Pro. She schedules a demo from the Sahi team. She asks lots of questions. She does a screen sharing of her application and the Sahi team demonstrates how to automate it. She finds it easy. She does not know yet about running it in suites, distributing it across machines, taking screenshots, integrating with Continuous Integration systems etc. But she need not worry. Everything is inbuilt into Sahi Pro. The developer went through Sahi Pro’s documentation and found that it covers all of this. And your friend said it works too.

SahiPro-test-automation

The SUCCESS:

You are happy. You are happier to know its cost. And the fact that it allows flexible licensing. It all makes perfect business sense. Your developer who was handling automation is back to development. He was frustrated with the brittleness of the tests and having to interact with non-programmers. He is happy to go back to doing fun creative coding. His first love. The testers have automated a lot of scenarios now, especially with Sahi Pro’s inbuilt framework. One of the automation engineers who was hired is a little disappointed that he did not get to develop his own framework. He has applied for developer jobs elsewhere. The other automation tester has gone back to do what he loves – exploratory testing of complex business scenarios, while his automated tests verify known paths and regression scenarios reliably…

OpenSSL and Heartbleed clarification in Sahi Pro

Posted by | Sahi, technical | No Comments

One of our support requests asked about the version of openssl used in Sahi Pro. Since it may be of concern to others, here is some more information related to use of openssl in Sahi Pro.

The version of openssl is :
OpenSSL 0.9.8h 28 May 2008

OpenSSL 0.9.8h did not have the heartbleed bug. (“OpenSSL 0.9.8 branch is NOT vulnerable” from http://heartbleed.com/)
In Sahi, Openssl is only used to generate a signed certificate with Sahi’s CA certificate (also created via openssl).
All other HTTPS communication between browser proxy and server are done via Java’s built-in SSL implementation.
Sahi does not add its version of openssl into the system path, so it should not affect the rest of your system/application.

Sahi Pro v5.1.1 fast execution without waits on Chrome 34

Posted by | Sahi, technical | No Comments

Some of you may have noticed that Sahi Pro does not wait correctly for page loads since Chrome 34. This happens because of a change in window.document.readyState behaviour in (since?) Chrome 34.

To fix it, do the following:

Open sahi/userdata/config/user_extensions.js and add

// Chrome 33 fix start
Sahi.prototype.replace33 = function(fn) {return "("+(""+fn).replace("_isChrome", "isChrome33Minus")+")";}
Sahi.prototype.areWindowsLoaded = eval(_sahi.replace33(Sahi.prototype.areWindowsLoaded));
Sahi.prototype.ping = eval(_sahi.replace33(Sahi.prototype.ping));
Sahi.prototype.getChromeVersion = function() {
var m = window.navigator.appVersion.match(/Chrome\/(\d+)\./);
return m ? parseInt(m[1], 10) : 0;
}
Sahi.prototype.isChrome33 = function() {return this._isChrome() && this.getChromeVersion() == 33;}
Sahi.prototype.isChrome33Minus = function() {return this._isChrome() && this.getChromeVersion() <= 33;}
//Chrome 33 fix end

before the line:
__sahiDebug__("user_ext.js: end");

Restart Sahi, clear browser cache and check if it works. If you have trouble, please email support.

Sahi Pro V5.1.0.0 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro v5.1.0.0

Sahi Pro v5.1.0.0 is a major release and contains many new features and bug fixes. Some of the key features added are IE 11 support, CORS support, data driven suites, email notification, scenario editor for Excel framework, Sahi as a documentation tool and ability to launch multiple browser sessions in a single script. We have also added brand new documentation. A complete list of features incorporated into the release, bug fixes made, integrations possible etc., can be found here: What’s new in Sahi Pro v5.1.0.0

Sahi Pro V5.0.9 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.9
This is a bugfix release.
Below is the changelog:

26 August 2013
----------------
Sahi Pro V5.0.9 released.
This is a bugfix release.
* Bugfixes
	Fixed: IE renders correctly depending on X-UA-Compatible for ie meta tag
	Fixed: Cookie value containing extra trailing comma issue
	Fixed: Setting UserDefinedId from inside a script.
	To set the userDefinedId from inside a script, use the following code
	ScriptRunner.getSession().getSuiteInfo().setUserDefinedId("sampleid");
	Fixed: For Ajax requests, Sahi will return a 401 response by default, instead of returning a custom 401 page with 200 status.
	To change the behavior, add xhr.return_401_response=false in userdata.properties.
	Fixed: parser bug where switch statement immediately following a case statement was ignored
	Fixed: parser bug where function.bind was causing classcast exception
	Fixed: parser bug to handle unary operators in statements; like _click(_link($i++))
	Fixed: Missing dload.sh added
	Fixed: Correctly showing error in logs if path is wrong while running from suite from testrunner.
	Fixed: Screenshot comparison logs in IE9
 

Sahi Pro V5.0.8 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.8
This is a bugfix release.
Below is the changelog:


Sahi Pro V5.0.8 released.
------------------------
This is a bugfix release.


* Bugfixes
Fixed Flex for chrome and firefox
Fixed support for input events on IE - fixes angularjs problems
Fixed identifying elements across iframes from same domain but different from parent domain
Fixed _in so that it does not look for elements in frames outside of the reference element
Fixed slow running of drun tests when H2 database is very large. Added index.

Sahi Pro V5.0.7 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.7
This is a bugfix release.
Below is the changelog:


Sahi Pro V5.0.7 released.
------------------------
This is a bugfix release.


* Bugfixes
Fixed Excel Framework reports
Fixed Excel Framework loadSahi script using _include
Fixed _dragDrop waits for full drag drop
Fixed bug where step starting with tab in Controller Evaluate Expression throws error
Fixed code folding related bugs which caused some Object methods to be skipped or incorrectly processed
Fixed NullPointerException in PacProxySelector
Fixed Origin request header not being sent to server: Set sun.net.http.allowRestrictedHeaders=true by default
Fixed parsing of E4X code in Sahi Script
Fixed automatic scroll up bug in Controller in some IE versions

Fixed Wrong screenshot associated with previous step
Fixed AltDblClick brings controller into focus
Fixed popup prefix when identifying flex objects
Fixed data base issue for MySQL on linux
Fixed slow loading of web based testrunner

Sahi Pro V5.0.5 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.5
This is a bugfix release.
Below is the changelog:


11 June 2013
----------------
Sahi Pro V5.0.5 released.


* Bugfixes
Fixed switch case null pointer exception when there are no statements in case
Fixed _takeSnapShot. Behaves same as _takeScreenShot
Fixed: In reports, clicking on log statements points to corresponding _log step


* Features
Added _hr API to identify html HR tag.
Added browser method sendHTMLResponseAfterFileDownload(b) in java driver

Sahi Pro V5.0.4 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.4

This is yet another quick bugfix release!

Below is the changelog:


10 June 2013
----------------
Sahi Pro V5.0.4 released.

This is a bugfix release.

* Bugfixes
Fixed Code folding for functions used as constructors
Fixed switch case parsing when case has a scope ({}) instead of a single statement
Fixed Mailer: addRecipient used for multiple to addresses
Removed Mailer SSL port hard coding
Fixed recursive includes leading to StackOverflowError
Fixed setting external proxy configuration (was broken since 5.0 after adding proxy.pac support)

* Features
Added _object to identify html tag object.

Run Sahi script on iPad

Posted by | Uncategorized | No Comments

Sahi can be used for testing the fluid nature of web apps and its functionality on different devices. In this post, we will be looking into running Sahi scripts on iPad. Before I go ahead and mention how to do this, there are few important notes about this kind of execution:

  1. You can’t install Sahi on iPad. Sahi must be installed and running on a different computer in the same network.
  2. The scripts which will be executed on iPad will not be recorded on the iPad. These scripts will be recorded on the computer itself, by resizing the browser and changing the user agent to iPad.
  3. Only the playback will be done on the iPad to check if the web app renders correctly on iPad or not.
  4. The scope of this post is only related to playing back a script on iPad. A different post will come up with the steps required to record scripts for different devices.

With these points in mind, let’s move ahead and configure our iPad for the run. While following these steps, please note that you need to use the IP address of the machine on which Sahi is running (and not the IP I am using, that is of my own machine).

  1. To start configuring the iPad, click on the settings icon on the home screen.

    1

  2. This will open the general settings option panel. Click on the network tab:

    1

  3. This will show all the network options that are present for connection. In the wi-fi settings click on the network that is being used by both the iPad and the machine running Sahi. I am using Tyto_24 so I will click on the arrow button next to it.

    1

  4. This will open the settings for that particular network and show a bunch of details about the network. What we are interested is the proxy for the network. In case you are within another proxy, such as your corporate proxy, you need to make changes in the Sahi installed in your machine. Let us click on the manual button of proxy.

    1

  5. This opens up the settings panel for the proxy. The two fields which we are interested in are the proxy and port. We need to set these ports to our computer running Sahi on port 9999 so that the iPad can use the Sahi on the machine to run the scripts.

    1

  6. Set the proxy to the IP address of your machine.

    1

  7. Also set the port to 9999. This will ensure that it is going to route through Sahi installed and running on the machine.

    1

  8. Configure Safari to accept cookies always:
    Settings -> Safari -> Accept Cookies = Always
  9. Now open a Safari browser and go to the URL http://192.168.1.70:9999. Replace the IP with your own machine IP. This will open the Sahi page for running script on mobile. This is similar to the playback tab of the controller where you can provide the script name and start URL for the run.

    1

  10. Provide the name of the script at [1], the start URL at [2] and then click the run button [3].

    1

  11. As you click the run button, you can see the page navigating away to your script run and your script will be executed now.

    1

    The reports for this script will be available on the machine on which Sahi is running.

Sahi Pro V5.0.3 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.3

This is yet another quick bugfix release!

Below is the changelog:


30 May 2013
----------------
Sahi Pro V5.0.3 released.

This is a bugfix release.

* Bugfixes
Fixed recording of elements which have index as their first identification attribute
Fixed java.awt.Robot references which cause PhantomJS scripts to fail on linux machines without X Windows
Fixed handling of negative values in _xy() API
Fixed fetch API in ruby driver to handle both attributes and chained attributes (like rows.cells.length)
Fixed Assertion selection in Java Controller
Fixed code folding logs when functions are declared inside included files
Fixed Controller pause/step buttons. Removed current step, next step counters.
Fixed showing _wait statements in Controller.
Fixed display of time when time taken is more than 24 hours (prettyTimeMoreThan24Hours added and exposed in XSLUtils)
Fixed Recorded Steps not showing correctly in Controller
Fixed HTML reports not showing browser name when run via testrunner or Controller.
Fixed remembering "Record" button state
Fixed issue of Controller content opening on main window with blank Controller on some versions of IE

* Features
Added openWindow API in java driver to launch in different resolutions
Added _sahi.isReadyForStepDefault. This is called from _sahi.isReadyForStep which can be overridden to add custom waiting conditions.
Added JSDoc comments to apis.sah for auto-complete help in editors like Eclipse

Sahi Pro V5.0.2 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.2

This is another quick bugfix release. Sahi Pro V5.0 has a lot of significant changes to improve overall usability and performance. This also meant that we sneaked in a few bugs. Thanks to our customers to have reported bugs in the 5.0.1 release made just yesterday and helping us come out with v5.0.2!

Below is the changelog:


16 May 2013
----------------
Sahi Pro V5.0.2 released.

This is a bugfix release.

* Bugfixes
Fixed "return" statement parsing bug
Fixed testrunner for singleSession=true threads=1
Fixed JS error alert in Controller during Java playback
ignorable_ids applied only on ids. Added zk pattern
Ruby Sahi: Added support to identify elements using multiple attributes via hash
eg @browser.textbox({"name"=>"q","sahiIndex"=>2}).

* Features
Added ability to show/hide Base Url and User Defined Id based on flags in Suites List
Set XSL param showUserDefinedId to true in suites_list.xsl to show the User Defined Id
Set XSL param showBaseUrl to true to show the Base Url

Sahi Pro V5.0.1 – What’s New?

Posted by | Uncategorized | No Comments

Download Sahi Pro V5.0.1

This is a bugfix release with the following changelog:


15 May 2013
----------------
Sahi Pro V5.0.1 released.

This is a bugfix release.

* Bugfixes
Fixed "switch" statement parsing bug
Fixed Run button from mobile user interface (launch.htm)
Fixed browser type detection (launch.htm)
Fixed suite inside suite bug when start url is specified along suite.
Fixed web based testrunner to run scripts from custom paths configured via scripts.dir

Sahi Pro V5.0 – What’s New?

Posted by | releases, Sahi, Uncategorized | No Comments

Download Sahi Pro V5.0

Major Changes:

Detached Sahi Controller
The Sahi Controller is no longer tied to the same domain of the webpage that is being automated. This fixes two problems: 1) multiple controllers opening up when navigating between domains/popup windows and 2) Controller not responding to some parts of the web page

External Proxy Configuration using proxy.pac file
Many organizations use a separate proxy.pac file to configure their complex proxy rules. Sahi now can be configured to use a proxy.pac file instead of a hardcoded external proxy.

Added support to specify suites inside a suite
Suites can now be included inside other suites, allowing easier management of suites.

Script parsing done via Rhino AST parser
Sahi has moved from its previous string based parsing to Rhino’s AST parser. This allows formatting Sahi scripts in similar ways as normal Javascript files. For example, multiple actions can now be written on a single line.

Parallel browser launching enabled from java driver
Multiple browser instances can now be launched using new Browser() instances. Previously this would cause clashes in browser profiles, which has now been fixed.

_setValue bug fixed for firefox 20+
We feel stupid, but the problem was that Firefox 20 was being identified as Firefox 2 by Sahi, and hence falling back to Firefox 2’s event simulation mechanism. (Who anticipated Firefox 20 within 2 years?!) This has now been fixed.

Java 1.7 – Browser opening bug fixed
Browser launching was broken for people who moved to Java 1.7. We have fixed this now.

Proxy Configuration bug in IE10 x86 version Fixed
The proxy would be set and unset immediately on IE10 x86 version. This has now been fixed. Similar issue with Safari on Mac also is fixed.

Reporting Mechanism Optimized
The previous implementation stored execution steps in memory till the end of a script and then wrote to the database. In case of long running scripts, the memory requirements became large and if there were any OutOfMemoryErrors, the script would terminate without any logs, causing frustration and wastage. From this release, the logging is done directly to the database at each step itself, so there is no memory accumulation. Even if scripts abort in between, the logs are available till the last executed step.

Minor Changes:

Allow playback of scripts without refreshing page (“Use Start URL” checkbox added)
It has been a long time complaint from our users that Sahi refreshes the page before executing a script. We have now made this optional via a checkbox. If unchecked, the script can be set and played back without reloading the page.

Make Streaming configurable
To accommodate technologies like Comet and Server Push, we have added streaming_contenttypes.txt and streaming_urls.txt which specifically tell Sahi to stream the content without buffering.

Added Diagnose link to Dashboard
If a Sahi script waits for a long time without executing the next step, one can click on the Diagnose link on the Controller to dump the state of playback. It helps diagnose problems with Sahi indefinitely waiting for AJAX activity to subside.

Added Sahi version to Dashboard

Added feature to set java path automatically during installation of Sahi

Added setFile2 API to Sahi Java

Added Start URL in reports

New APIs added:

  • _verifyLayout: Allows verification of Page layout
  • _readLayout: Reads a simple csv like file of Sahi Elements and converts it into 2D array to be consumed by _verifyLayout
  • _includeOnce: For complex nested scripts, allows including library files only once.
  • _openWindow: Opens a new window at given resolution. Helps testing responsive layouts along with _verifyLayout
  • _windowAction: Used to perform maximize, minimize, focus, refresh, restore and close window on Windows

Download Sahi Pro V5.0

Complete Changelog


07 May 2013
----------------
Sahi Pro V5.0 released.

* Features
Parallel browser launching enabled from java driver.
Added Start URL in reports
Updated the API list for Notepad ++ plug-in.
scrollOffsets added to mouse events
Added setFile2 API to Sahi Java
Added diagnose link to Dashboard
Added Sahi version to Dashboard
Script parsing done via AST parser.
Added _windowAction API which can be used to perform maximize, minimize, focus, refresh, restore and close window
Added feature to set java path automatically during installation of Sahi.
User Agent added to initialized.htm
Added support to specify suites inside a suite.
Support provided for Microsoft SQL database.
_verifyLayout and _readLayout API created
Make Streaming configurable; added streaming_contenttypes.txt and streaming_urls.txt
_includeOnce API added
_openWindow API added
proxy selector with pac file
Allow playback of scripts without refreshing page ("Use Start URL" checkbox added)
Detached Controller implemented to allow easy recording via multiple domains/windows without opening multiple Controller windows
New playback mechanism using ping

* Bug fixes
Fixed identification of divs where each text is encapsulated by 2 divs
Fix for very slow performance of removeComments. Fixes slow loading.
Log Exception takes to the correct line number in script.
Script parsing error reporting fixed.
Fix _under in IE.
_under and _above fixed to take offsets on both sides
Mysql logging fixed
.js files fixed to be treated as single .sah file instead of .suite
HAR logging fixed
changes to terminate a load test, if the capacity is exhausted on all nodes
Fixed "Test->" assert of multiple lines from controller
Dsync fixed to copy scripts to nodes from master if master is not in list
Update of suite reports result in dload fixed
_setValue bug fixed for firefox 20
Suite report time taken format changed
fixed: reports dumped to FS
database table creation fixed
load runner fixed
Status and graph for drun fixed
drun logs for suites fixed
leftOf, RightOf fixed
domanifix with 2 parameters fixed
Fixed reports for csv based suites
test case reporting fixed
_takeScreenShot fixed
Linux certgen problem fixed

Download Sahi Pro V5.0

Testing responsive web design

Posted by | features | No Comments

Understanding responsive web design

When web content was primarily consumed through laptops and desktops, web sites were designed, developed and tested for handful of screen resolutions such as 800×600, 1024×768.

With advent of mobile operating systems such as iOS, Android, Windows 8 and BlackBerry 10, device manufacturers are rolling out different models of smart phones and tablets. Web content is being increasingly consumed using these smart phones and tablets. Screen size of these smart phones and tablets not only varies from one device model to other but also from one device manufacturer to another. Varying screen sizes among smart phones and tablets, create challenge for web designers, developers and testers.

Responsive web design is a user interface technique used by designers and front end developers to create adaptive web pages. These adaptive web pages will automatically change their design and layout to fit across different screen sizes (often of different devices).

Example of responsive web design

In this example we will look at a web page created using responsive web design technique. Below images show different view of same webpage across different screen size.

Webpage on Tablet Webpage on Mobile

What to test?

In responsive web design, when screen size changes, there will be an impact on following elements,

  1. Page Layout
  2. Visibility of elements
  3. Position of visible elements
  4. Size, color of visible elements

By asserting on visibility, css properties and position of elements, we could validate design and functionalities of a web page, across different screen size.

How to test?

  1. Resizing browser window
    • Load web page in a browser and resize the browser window to prefered screen size. Manually resizing browser window to specific size would be difficult and error-prone. We could use “window.open” javascript api to set window size & load the web page.
  2. Using firefox developer tools
    • Go to “Tools” > “Web developer” > “Responsive Design View”, set prefered screen size and load the web page.
  3. Device emulators/simulators.
    • Android & iOS development environment comes with device emulators/simulators. Android emulator allows us to set device resolution. Web pages can be loaded and tested on these emulators/simulators.
  4. Actual devices.

Automtaion using Sahi

Sahi pro has many capabilities that will make testing of responsive web design across different screen sizes & devices, easy & elegant. Register below for free webinar (April 2013) on testing responsive web design using Sahi pro.

This webinar has expired

Sahi Pro v 4.5.2 – What’s New?

Posted by | features, releases | No Comments

This is a bug fix release

* Features
Added ability to pass extra parameters to Web based Script Runner
* Bugfixes
Fixed memory leak in distributed playback
Fixed problem with session timeout causing “Script did not complete in 150 seconds” error if page does not load within 90 seconds

Sahi Pro v 4.5.1 – What’s New?

Posted by | features, releases | No Comments

Sahi Pro V4.5.1 released.

This is a bug fix release. Please use this version instead of Sahi Pro V4.5

Features

  • Added ability to pass extra parameters to Web based Script Runner

Bug Fixes

  • Fixed memory leak in distributed playback
  • Fixed problem with session timeout causing “Script did not complete in 150 seconds” error if page does not load within 90 seconds
  • Fixed parsing related bug which caused incompatibility with older Sahi scripts

Sahi Pro v 4.5 – What’s New?

Posted by | features, releases | 2 Comments

Sahi Pro V4.5 released.

Features

  • _maskLogs and _unmaskLogs added to mask sensitive information in logs. More details
  • _addToSession added: Allows adding different domains to a session
  • Added up down arrows to move to parentNode in Controller.
    Parent Finder

    Clicking on the Up arrow identifies the current accessor’s parent element. Clicking the down arrow brings you back.

  • Added threads field in web based script runner.




  • Added ability to copy screen shot images into output logs folder
  • Testcase based reporting enhanced. Pass/fail counts display according to testcases. Navigation improved.
  • Evaluate Expression in flex shows more details.
  • Added logging for miscellaneous functions like _focusWindow(), _takeScreenShot etc.
  • _setFile2 added. Automatically handles js validations.
  • Fixed Java APIs
  • Selenium Java API Support (beta) added.
  • Applet Support (beta) added via _applet API More details
  • Rerouter added for dynamic domain to IP mapping

Bug Fixes

  • Fixed keypress issues in various browsers. This fixes keypress enter _keyPress(el, [13,13]) not working on some browsers.
  • Test case reports show count of testcases instead of scripts.
  • Mixing file and url scripts fixed.
  • Modal from different domain fix.
  • Double commas in SFL metadata fixed.
  • Limiting controller logs length to 100 steps to fix hang issues.
  • Fixed image path of screenshots in logs for Mac
  • handle newlines and spaces in browser_types.xml
  • Flex issues fixed. Horizontal Scroll Bar, NumericStepper, MenubarItem(only using number)
  • browser_types.xml Remove duplicate (x86) from browser path. Also expand $sahiDir.
  • Fixes _takeScreenShot on mac issue.
  • Added _setFile with 3 parameters in Sahi Java.
  • NO_LOG type handled

Sahi Pro v 4.3.2 – What’s New?

Posted by | releases | One Comment

Expand Collapse State in Logs Remembered

When moving from a script log to the script line and back, the state of expand or collapse of logs is remembered.


Controller State Remembered

The state of the Controller is remembered across sessions and even after a Sahi restart. This helps reduce a lot of time spent in setting and resetting the Controller parameters. The last URL entered in the Sahi initial screen is also saved.


New APIs Added

_closeBrowser()
_openBrowser()

_closeBrowser() and _openBrowser() are used to close the browser in the middle of a script and then reopen it later. This can be useful where an application does not allow relogin unless the browser is restarted.

_setXHRReadyStatesToWaitFor($waitStates)

Sahi waits for AJAX readyStates 1,2 and 3. Some applications may have an AJAX request open at state 1 for long periods of time. Sahi should be asked to ignore readyState 1. _setXHRReadyStatesToWaitFor(“2,3″) can be called in this case. $waitStates is just a string of comma separated readyStates (“1,2,3″ or “2” or “2,3” etc.).


Store Reports into MySQL Database

Sahi’s reports are stored in a file system based H2 database. For custom reporting it may be necessary to store it into a central MySQL database or another database. Sahi now supports storing reports into a MySQL database. More details Storing Reports in MySQL Database


Upgrade or Downgrade from inside Sahi Pro

To upgrade to a newer version, we have so far had to install Sahi Pro in a different folder and copy over the userdata directory. Now it is possible to directly upgrade from Sahi Pro itself. This will enable faster patch releases for critical bugs and easier maintenance of Sahi Pro.


Bugfixes

  • Fixed bug in reporting large log files
  • Fixed _byText
  • Fixed bug in resizing Evaluate Expression box
  • Fixed IE focus issue
  • Fixed upgrade version comparison bug

Sahi Pro V 4.3 Released

Posted by | Uncategorized | No Comments

Sahi Pro V 4.3 has been released.

This release brings in significant bug fixes and features

Below are the change logs:

Features

  • _focusWindow API added – focuses windows on Windows OS
  • _takeScreenShot API added
  • Ability to automatically store and retry failed scripts during suite run
  • Added Script Explorer to trigger test cases from anywhere via web
  • Added Syntax Highlighting to Evaluate Expression box and Script Editor
  • Ability to add base64 encoded passwords in userdata.properties
  • Added missing Flex mx and spark components
  • Ability to add custom fields in suite reports
  • Ability to use Properties files when connecting to database

Bugfixes

  • Array toJSON recursive_access bug fixed
  • RegExp with modifiers handled correctly
  • onScriptError logging issue fixed
  • Sahi injects properly for conditional comments
  • _setValue($a,$b) with no space after comma handled correctly
  • Fixed Excel POI APIs
  • Fixed focusin focusout events on IE9
  • sfl files recompiled with correct versions and bugfixes
  • URL history added to Controller and Script Explorer
  • Window name is blank by default. Not null. Fix for sessvars
  • Fixed bugs in logs display and code folding
  • Fixed Excel inside Excel log steps
  • javascript-xpath.js blanked by default.
  • Drag drop significantly improved. Works across scrolling panels.
  • Firefox 13 suite run fixed
  • _confirm and _prompt wait for input
  • writeCSVFile writes double quotes correctly
  • Significant fixes for memory leaks during long suite execution
  • Fixed log reporting failures due to length of message

Sahi Pro v 4.3 – What’s New?

Posted by | releases | No Comments

New APIs

_focusWindow()
Brings the browser window into focus. This is useful while taking screenshots or using native events.
_takeScreenShot()
Takes a screenshot of the entire screen and adds it to the logs.
_focusWindow();
_takeScreenShot();


Automatically rerun failed scripts

Sahi can create a suite of failed scripts and trigger it at the end of a suite run.
Refer to “Rerunning failed scripts automatically” section here

Trigger testcases from a web interface

We have released the first version of the web based interface for running Sahi scripts.
The web based testrunner allows us to easily trigger testcases on a remote machine running Sahi.

More details here: Web based Testrunner

Syntax highlighting on Evaluate Expression box and Script Editor

Syntax highlighting helps easily see script errors and match brackets in the Evaluate Expression box and Script Editor.

Encode proxy passwords in userdata.properties

Since proxy passwords can be sensitive, we have enabled simple base64 encoding for masking passwords in userdata.properties

Usage:
ext.https.proxy.auth.encrypted=true
ext.https.proxy.auth.password=encrypted_password

You can base64 encode strings here: /demo/php/base64.php

Added missing mx and spark components in Sahi Flex Library

We added support for missing classes from the mx and spark packages

Show custom fields in suite reports

Look at Custom Fields section in Sahi Pro Reporting

Use Properties files when connecting to database

Sometimes it is necessary to pass more than just username password when making a connection to a database.
The _getDB API now can take a Properties file with extra parameters. More details: _getDB

Been a long year!

Posted by | Sahi, tyto | 2 Comments

2011 has been awesome for us. Tyto Software has grown and become a self sustaining unit with a team that has really come together. Sahi Pro is well received and we released V4.0 recently. We also added a lot of new customers; customers, who have expressed happiness at how our product works and how we support them. We added Flex capability, load testing, better reporting etc. to Sahi Pro, and the results have been very positive. A surprising hit feature was our Excel Framework, which saves a lot of time for users not very comfortable with programming.

While our support is generally well appreciated, we have been trying our best to keep improving on it. One sore point has been our lack of online payment processing methods, which we are trying to alleviate in January 2012.

Over all, we are geared well for 2012, having brought on a couple more to the team and put in processes which make the end user benefit a lot from using our product.

So where is Sahi Open Source? We have been very busy with the Pro version and we should be able to get a breather in January to port back fixes into Sahi OS.

Thank you all for your patronage, and wish you a very happy and prosperous New Year!

Sahi V3.5 2011-03-14 released

Posted by | releases, Sahi | No Comments

Sahi V3.5 2011-03-14 is now available on SourceForge.

  1. We have added support for HTML5 elements, so, for example,

    is identified by Sahi as:
    _numberbox("myId")

  2. Proxy switching on IE has become a little better
  3. _collect and _count have been added.

    Eg.

    [code]_assertEqual(5, _count(“_link”, “View Details”));[/code]

    and

    [code]var $els = _collect(“_link”, “/View/”);
    var $len = $els.length;
    _assertEqual(5, $len);
    _assertEqual(“View Details”, _getText($els[0]));
    [/code]
  4. Ignorable ids made a user modifiable property in sahi.properties/userdata.properties. So if you have a new framework which generates ids in a definite pattern, you can ask Sahi to not use them as identifiers for accessors.

    Look at recorder.ignorable_ids.pattern in sahi.properties

  5. Checkbox simulation has fluctuated a bit in the last few builds of
    Chrome. Sahi now assumes Chrome 10+ as default. This is modifiable in
    sahi.properties

    [code]simulation.chrome.explicit_checkbox_radio_toggle=true[/code]

    Change to false if the checkboxes seem to toggle back to original state.

  6. Download the latest version from here:
    https://sourceforge.net/projects/sahi/files/ and do spread the good word. You can tweet about it, or like us on Facebook or write a blog post about it.

    Wish you happy productive testing!

Stable accessors/locators using UI Relations

Posted by | Sahi, stable tests, ui relations, web automation, web testing, xpath | No Comments

Identification of elements in a web interface is one of the toughest challenges of software UI automation. First came accessing by location (x,y coordinates), which quickly faded away due to window resolutions, rendering differences etc. Then came the concept of identification using code structure. XPath became hugely popular;

Reasons being

  1. Automation was only attempted by established developers and they understood XPaths
  2. 5 years back web applications were simple enough to easily look at the DOM and identify XPaths.
  3. Some (silly!) tools adopted and hugely promoted XPaths for want of innovation on that front.

But XPaths quickly became notorious for being unmaintainable and difficult to understand. If test automation was to spread to non-programmer testers, this would be a huge barrier. Tools then tried to move on to css-selectors, which is still complex for the target test automation teams.

So what could be a possible solution? The answer is in the User Interface.

What is often forgotten in the software industry, is that applications are built to satisfy business needs. So whatever the id, XPath or css-selector is, a textbox meant for “loan amount” will always be near some label saying “Loan Amount”. An expand or collapse icon in a tree gets its meaning from the attached “User name” label for that node. These business requirements do not change as often as code or underlying HTML structure. An HTML table may get converted to an ext-js grid with scrollbars, but the cost will still be associated (and aligned) with a product and listed under the heading “Cost”. Items related to each other would be shown inside a box with an appropriate heading.

How can we use this information to identify elements? Use UI relations for identification. Sahi, which has been a pioneer in web-automation innovation, handles it as below:

A textbox meant for “loan amount” becomes

_textbox(0, _near(_label("Loan Amount")))

or

browser.textbox(0).near(browser.label("Loan Amount"))

The cost becomes
_cell(0, _near(_div("My product one")), _under(_div("Cost")))
The expand collapse icon becomes
_image("/tree-node-icon/", _near(_span("Expand me")))

  1. These accessors clearly communicate what the intention of the accessor is, in a straight forward way (within the constraints of programming language constructs).
  2. These also take away the complexity in identifying and stabilizing XPaths, or understanding complex css-selectors, or writing custom javascript loops to accomplish the same thing.
  3. They are quite stable. _near allows for more elements to come between two related elements, _under is spatially aligned. So you can move different elements around on the UI and still have your scripts working properly.

UI Relations are guaranteed to exist in any business application, irrespective of whether you have ids or not.

One argument that comes up is, what if there are two similar elements which satisfy the same condition. While it is still possible to index the similar accessors and uniquely identify them, it seldom happens in a real world application. If you have two textboxes near “Loan amount”, it is going to confuse your end user. User-interfaces do not generally have such anomalies. That said, an added index to the accessor handles that scenario too, if it arises.

There is one other simplification that Sahi does in its accessor APIs. It does not explicitly state what the identifying property is. For example what could have been
_textbox()
is simply given as
_textbox("username") in Sahi.

Why? Because there are very few cases where one textbox has name=”username” and another has id=”username”. Such naming would obviously confuse the web developer forcing him to not do something like this.

Combined with UI relations, Sahi also takes care of automatically waiting for AJAX requests and page loads. This makes for robust scripts which hardly have unexpected failures. Other features like parallel playback of suites, automatic report generation, ant integration etc. make Sahi one of the most advanced web testing tools in the industry.

If you have not tried it yet, download from http://sourceforge.net/projects/sahi/files/ now. You can also use Sahi from inside ThoughtWorks Studios Twist or PushToTest’s TestMaker 6.

October 2010 Update

Posted by | Uncategorized | No Comments

Hope you had a happy Deepavali.

October continued to be busy. We worked towards the next open source release, with lots of fixes and enhancements. Release was made on 3rd Nov. 2010, just before Deepavali. Part of the focus was on better integration with test management tools. A detailed post on integration shall follow. The new build can be downloaded here: http://sourceforge.net/projects/sahi/files/sahi-v3/2010-11-03/sahi_20101103.zip/download

We did a 2 day online training of Sahi for Advance Internet Inc. We also delivered proof of concept of flex automation to a prospective customer in the banking domain. The flex solution does not need re-compilation of flex code, so may be suitable for environments where modifying code may not be possible.

We could not conduct a webinar last month, but we shall be conducting one on 16th Nov. 2010. Click here to join our webinar: https://www3.gotomeeting.com/register/301637230

We are streamlining the process of support and sales for Sahi Pro online, which should be done by the end of November. Meanwhile, if you have queries on Sahi Pro, please email us at support@sahi.co.in

I did get invited to GTAC on the second day, and managed to talk a little about Sahi’s object recognition mechanism. But it would have been much better if I had had time for a full presentation on Sahi. We also had discussions with the WebDriver developers to see if we could adopt WebDriver underneath Sahi. Preliminary investigations reveal that we need to wait some time for WebDriver to mature before we can adopt it. Currently WebDriver supports only one instance of IE on a machine, which may work if you had a Google farm at your disposal, but will not work for Sahi’s users who are used to running multiple IE instances simultaneously on a single machine.

September Update

Posted by | Sahi, tyto, update | No Comments

The past month has been very special for us. We delivered Sahi Pro to our customers, and without even publishing about it, we got many inquiries about Sahi Pro. This was something we had not expected, but would like more of.

Given the number of support queries coming in, we had to scale to better serve our customers. So Boopathi M joins our team as a support engineer. With a background in programming and teaching mathematics online, we look forward to utilizing his tutoring capabilities to solve technical issues in Sahi for our customers.

We also managed to have a presence in Java One, thanks to PushToTest‘s generous offer to all its partners to show case themselves. Given that Sahi has started playing a good role in TestMaker’s web automation part, this turned out to be a good point of synergy.

I (Narayan) also participated as one of the panelists in the Birds Of a Feather session on open source testing tools, thanks again to Frank, CEO of PushToTest. (It is amazing what PushToTest is doing to promote open source products. If ThoughtWorks is the breeding ground for open source projects, I would say PushToTest is one of the few companies that helps them bootstrap and become commercially viable.) We realized Sahi is almost unheard of in the Java developer community! So we are coming up with ways to rectify that. If you have suggestions, do let us know.

This month was also great because we got lots and lots of praise for Sahi (read some testimonials). We had 4 instances of Sahi users reporting that they shifted from QTP to Sahi because QTP had problems identifying some web elements, which Sahi could easily identify. Some of them also had problems on QTP’s Windows 7 support.

Selenium continues to annoy us with its presence. We again had 3 different discussions with users who said “We find Sahi to be much easier to use but developers/management (in the US) have heard only of Selenium.” A stackoverflow question has considerably undermined Sahi’s capabilities and slandered DP, one of Sahi’s old and staunch supporters. It was frustrating to know we can’t do much about it. Equally unhelpful is the fact that we will not be able to participate in GTAC 2010 because “the committee reviewed each application with special attention towards contributions to the professional community, among other aspects. We regret to inform you that you have not been short-listed to attend this year’s conference.” We would have loved some discussion with the WebDriver guys.

Our website was hacked twice last month and if you find google search results showing weird titles for Sahi searches, please let us know on the forums or email us.

We have started a facebook page for Sahi, so please do spread the word.

We now have integrations with Test Management tools like XQual, Testuff and Neuvosoft Test Manager, and we will continue to support more. Coming up is integration with QMetry

Things on the roadmap (Oct-Nov):
1) Flex/Flash support(Pro)
2) Simple loadtesting through HTMLUnit (Pro)
3) Integration APIs for Test Management tools (Open and Pro)

We continue with our integrations with PushToTest‘s TestMaker and Test Object Designer, and with ThoughtWorks Studios’ Twist.

We will be starting webinars every two weeks as an introduction to Sahi. The first session is (tentatively) scheduled for 22 October 2010. If interested please email us at support@sahi.co.in

Thanks for reading through. If you have suggestions, queries or advice, please email us at feedback(at)sahi.co.in. Hope to see you next month!

Sahi V3 2010-06-10 released

Posted by | releases, Sahi | 5 Comments

Sahi V3 (2010-06-10) is now available for download. https://sourceforge.net/projects/sahi/

There have been significant improvements and bug fixes in this release.

  • API _under ( /w/browser-accessor-apis ) has been added to locate elements physically under another.
  • API _byXPath has been added to help users from Selenium and other tools move to Sahi.
  • API _row and _option have been modified to be in sync with other APIs. Existing scripts may need to be modified if you use these APIs.

Below is the changelog:

10 Jun 2010
———–


* Bugfixes
Fixed data truncation bug introduced in 30 Apr build
Fixed getText bug which returned ab on FF and a b on IE for a
b
Fixed for   in select option text
Fixed window.opener behaviour for link clicks
Fixed parsing error for $a == $b

* Features added
Added Shiretoko as variant of Firefox
_byXPath added.
For browsers without XPath support,
download the javascript file from http://coderepos.org/share/wiki/JavaScript-XPath
and save the contents in sahi/htdocs/spr /javascript-xpath.js
_under added as a positional relation
Lets identify elements under another element eg. _cell(0, _under(_tableHeader("Status")))
5xx errors are displayed on screen too.
Time taken by each test is displayed in logs.
XHR redirects are excluded from injection
Added Driver.setControllerMode. Takes sahi, java, ruby
_option brought in sync with other APIs.
NOTE: older _option(selElement, value) will not work any more
Replace with _option(value, _in(selElement))
_row brought in sync with other APIs.
NOTE: older _row(tablElement, rowNumber) will not work any more
Replace with _row(rowNumber, _in(tablElement))

Ruby Sahi with Cucumber

Posted by | BDD, Cucumber, Ruby, Sahi | 8 Comments

What is Cucumber?

Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid – all rolled into one format.
- From http://cukes.info/

Follow the steps below to get started with Ruby Sahi and Cucumber.

  1. Install Java
  2. Install Ruby
  3. Install cucumber:
    gem install cucumber
  4. Install Sahi proxy: Download Sahi from sourceforge and unzip to some location. (say D:sahi)
  5. Start Sahi:

    cd D:sahiuserdatabin
    start_sahi.bat

  6. Install Sahi Ruby client:
    gem install sahi
  7. Create a file D:testlogin.feature, add the content below and save it.

    Feature: Login
    In order to access the system
    As a user
    I want to be able to login

    Scenario: Login with valid credentials
    Given I am not logged in
    When I try to login with "test" and "secret"
    Then I should be logged in

    Scenario: Login with invalid credentials
    Given I am not logged in
    When I try to login with "test" and "wrongpassword"
    Then I should not be logged in
    And I should be shown error message "Invalid username or password"

  8. Run this feature:

    cd D:test
    cucumber login.feature

    There will be a lot of messages with hints on implementing the right steps.

  9. Implement the steps:

    Create a file D:testlogin.rb, add the content below and save it

    require 'sahi'

    def init_browser()
    #Use the correct paths from your system
    userdata_dir = "D:/sahi/userdata"
    browser_path = "C:\Program Files\Mozilla Firefox\firefox.exe"
    browser_options = "-profile #{userdata_dir}/browser/ff/profiles/sahi0 -no-remote"
    return Sahi::Browser.new(browser_path, browser_options)
    end

    #open the browser at the start
    browser = init_browser()
    browser.open

    #close the browser on exit
    at_exit do
    browser.close
    end

    Given /^I am not logged in$/ do
    browser.navigate_to("/demo/training/index.htm")
    end

    When /^I try to login with "([^"]*)" and "([^"]*)"$/ do |username, password|
    browser.textbox("user").value = username
    browser.password("password").value = password
    browser.submit("Login").click
    end

    Then /^I should be logged in$/ do
    if !browser.button("Logout").exists?()
    raise "Not logged in"
    end
    end

    Then /^I should not be logged in$/ do
    if !browser.submit("Login").exists?()
    raise "Logged in"
    end
    end

    Then /^I should be shown error message "([^"]*)"$/ do |msg|
    value = browser.div("errorMessage").text()
    if value != msg
    raise "Incorrect message: #{value}"
    end
    end

  10. Run and watch the tests complete successfully

    cd D:test
    cucumber login.feature

  11. Done!

Sahi API _under added

Posted by | Uncategorized | One Comment

Continuing with our tradition of innovation for simplicity, Tyto adds another wonderful API to Sahi.

NOTE: _under will be available in Sahi’s next release

The problem:

Let us take the example of a dynamically generated grid. The example we use here is available at http://www.zkoss.org/zkdemo/userguide/#g7

We wish to assert the value of “Received” column for “Style Guide for ZK 3.5 released”. If we bring up the controller and CTRL-hover over that element, what we see is
_div(“2008/11/14 13:23:07″)
which is not useful as a finder.
Looking at the alternatives listed, we notice that none of them can really help.

To fix this, we shall try using _near.

1) We put

_div(0, _near(_div(“Style Guide for ZK 3.5 released”)))
into the Evaluate Expression box, and click Highlight. This highlights the “Style Guide” element itself.
2) We experiment with the index passed as the first parameter, and using Highlight, pinpoint on the correct accessor as
_div(4, _near(_div(“Style Guide for ZK 3.5 released”)))

But this will not make a good accessor.
Why?
Because, this uses an index which seems like it would change when another column is added before the Received column.

What we really want, is that element, which is UNDER _div(“Received”).

Introducing the _under API

_under(element):
_under is a POSITIONAL marker. What it means is that it checks for coordinate based alignment under a particular element within a specific threshold.
So, in our case, it will look for a div which is roughly positioned underneath _div(“Received”)

Here is the final accessor:

_div(0, _near(_div(“Style Guide for ZK 3.5 released”)), _under(_div(“Received”)))

Note how this accessor is independent of the order of the rows and the columns.

And then the testers lived happily ever after …

Notes:
This grid is not a simple table but actually composed of 2 tables, one for the header and one for the contents. So we could have approaced this problem using _cell(_table(2), “Style Guide for ZK 3.5 released”, 4), but again the 4 would trip us later if the order/number of columns changes.
_under(el) can be passed as a last parameter to any Sahi API.
NOTE: _under will be available in Sahi’s next release

Choosing the right web automation tool or web testing tool

Posted by | Uncategorized | No Comments

Web automation is a little trickier than most other automation because there are many combinations of browsers and operating systems and they are fast evolving too.

What do you look for before you choose a tool for web automation?

The answer may actually depend on what your organization specializes in. If you are a product company which ships applications meant only for Internet Explorer, you need not consider multi-browser support or Linux support. But if you develop outward facing web applications, you may need to test the application on multiple browsers.

Here are some factors you should consider before choosing a testing tool.

The tool:

  1. Should be techincally sound
    1. Should be able to identify elements/record on all browsers.
    2. Should handle complexities like HTTPS, Frames, IFrames, AJAX, dynamic ids.
    3. Should not need tinkering with source code of tool
    4. Should not require hard coded waits

  2. Should save time and effort for teams
    1. Ramp up time should be minimal. Users should get productive within an hour.
    2. Complexities like AJAX, dynamic ids, object identification etc. should be handled by the tool instead of passing it on to testers. *
    3. Should run reliably across browsers and operating systems to reduce re-runs and debugging effort.
    4. Should not be dependent on knowledge of various other tools/technologies.
    5. Should need minimal maintenance of scripts/code

  3. Should work with existing teams instead of requiring a drastic overhaul
    1. Should not require your teams to change from testers to developer testers, but let them easily pick up some scripting knowledge and get functional.
    2. Should not require expertise in various peripheral technologies like Java, Junit, TestNG, XPath, Firebug, Browser DOM etc. to just get started.

  4. Should require minimal stakeholders
    1. Should not need developer involvement for modification to application in the name of “testability”. Dynamic ids, elements without ids, etc. should be handled well by the tool. *

  5. Should be easy to scale testing teams
    1. Should be easy to hire and add more members to your testing team. This requires the tool to be simple to use.
    2. Should be able to move the teams across projects and products. This means that the tool needs to be sound enough to work with various technologies and frameworks.

  6. Should have authoritative support available
  7. Should be cost effective. The following need to be considered:
    1. Cost of acquiring the tool
    2. Cost of employing capable testers who can use the tool
    3. Cost of maintaining test infrastructure
    4. Cost of authoritative support

    Too often, especially with open source tools, the amount of money wasted in man hours due to limitations of the tool, incompatibility with existing expertise of team, lack of support etc. far outweighs the cost of acquiring alternative commercial tools. (Developers and testers with not much business experience invariably think that their time is not a cost to their company, and do not mind spending a week on an effort which should have lasted a day, thus wasting 25% of a month’s salary for a tool which may cost 10%)

* It is possible to just use the Sahi Controller and identify various elements reliably. Because tools like Selenium cannot record across frames, iframes, the tester is forced to learn to use Firebug to figure out what ID or XPath to use, add a line of selectFrame etc. These are very tool specific. While learning to use Firebug is an awesome skill to have, it should not be required at each step of the automation process. Adding conditional waits with knowledge of DOM is an unnecessary effort put on the tester, which can be handled by intelligent tools. Same goes for making developers add custom id generators for handling dynamic ids.

Sahi vs. Selenium

Posted by | Uncategorized | 13 Comments

Though Sahi is aimed at non-programmer testers and Selenium at programmers, we get a lot of queries on their differences. Here is a brief document which compares Selenium and Sahi.


To those who read this:
  1. If you are a Selenium fan, be open minded and verify the claims for yourself or contact us for clarification. And again, please be open minded. You may save a lot of time for yourself and your team.
  2. This document is biased towards Sahi because we built Sahi. We believe what we state is true, but if you have evidence to prove otherwise, please do contact us at support@sahi.co.in.
  3. When we say “Not sure” or “?” it means we do not have enough information because of lack of research on our part. Please verify for yourself.
  4. If you want a comparison between other tools and Sahi, we would be happy to discuss it.
  5. If you are another tool developer/supporter, please let us know how it compares and let the world benefit from alternatives.

Sahi vs. Selenium: Comparison document

Recorder

Selenium Sahi
Works only on Firefox Works on all browsers (IE, FF, Chrome, Safari, Opera)
Has trouble recording IFrames, Frames and popup windows Can record on IFrames, Frames and popup windows
For Frames and IFrames, need to explicitly selectFrame Implicit smart identification of elements even across Frames and IFrames
Uses XPath for identification of elements if id or name not present Uses various algorithms to uniquely identify elements in a simple human recognizable way

Programming Language support

Selenium Sahi
Java, Ruby, Perl, Python, C# (and may be more). Sahi Script, Java, Ruby Sahi Script has the syntax of javascript but can interoperate with any Java code directly from script. The Java/Ruby drivers are available since Sahi V3
Needs language bridges for each new feature. For example, needs java bridge to invoke Flash via ExternalAPI. Sahi Script can directly invoke anything exposed by javascript.

Ease of use

Selenium Sahi
Easy to start with because of Selenium IDE which is a firefox plugin. Estimated start time less than 5 minutes More difficult than Selenium to start because it needs installation of Sahi. Estimated start time 10-30 minutes, depending on Java installation etc.
Deep learning curve when the need is felt to move from Selenium IDE to Selenium RC. There is only one mode of operation for Sahi. Extremely simple to learn and use for testers
Knowledge of programming language required Can achieve most automation with just functions and variables. Sahi has inbuilt APIs for most other complex tasks like data driven testing
Needs JUnit style of programming Can choose your own style
Uses XPath based identification for elements in complex html structures or those with dynamic ids. css selectors and javascript may also be used. Has nearness APIs like _in and _near which can help show nearness of elements. Eg. _image(“delete.gif”, _near(“username 4”))
Needs waits for making AJAX work No waits needed in 90% cases
Supports parallel execution Inbuilt parallel execution. Needs only one parameter change



Stability of scripts and ease of maintenance

Selenium Sahi
Smart DOM Relations resilient to UI changes No
Dependent on XPath
Difficult for testers to understand and debug
Yes
Does not use XPaths.
Uses _near and _in
Implicit waits for page load and AJAX:
1) Saves time
2) Keeps scripts simple
3) Reduces random failures
No
Explicit waits needed.
Yes
Ease of adoption by a team of testers Needs testers to know TestNG/Junit, XPaths, HTML structures, Frames IFrame knowledge, Javascript for AJAX conditional waits Sahi abstracts out all these for the tester.

Dependency on other tools

Selenium – Java (Others need something similar) Sahi
Needs JUnit (and optionally eclipse) to run tests No additional tools required. Tests run from the Sahi Controller/command line/ant
Non persistent reporting. Needs TestNG or something similar for that Persistent HTML reporting which can be shared via URL or file

Stability of product and number of releases

Selenium Sahi
Started 2004(?) in ThoughtWorks Started 2005 in ThoughtWorks
Version 1 took 5 years, Version 2 planned mid-2010. Moving away from original architecture to WebDriver based architecture Current release: Version 3 Number of stable releases in 2009: 7

Footprint

Selenium Sahi
RC: 10.5 MB, Grid 15 MB less than 2 .5 MB with source
Not sure Runtime ~ 50MB for 3 parallel threads

Reporting

Selenium Sahi
Needs external tools to create readable reports Inbuilt HTML reports with click through to relevant portion of script

Others

Selenium Sahi
Build tool integration (ant, batch files) Yes Yes
Multiple OS support Yes Yes
Version Controllable Scripts/Code Yes Yes
HTTPS support/redirects Not sure Yes
401 Authentication, Windows/NTLM Authentication dialogs Not sure Yes
External proxy tunneling Yes Yes
In built APIs for data driven testing No Yes
Works only with browsers Yes Yes
Needs privileged modes on browsers for operation. (Privileged is bad) Yes No
Extensible on future browsers Depends on finding a way to use privileged mode on that browser Yes. Very little dependency on type of browser.
Editor support Has good editors in various languages Editor support for javascript is not as good as for Java.

Support available

Selenium Sahi
Free support via Forums Yes Yes
Paid support available Yes Yes
Authoritative training available ? Yes

Presentation at XP Goa day

Posted by | Function testing, Sahi, XP Goa day | No Comments

Narayan Raman presented on Functional Testing of Web Applications using Sahi at the XP Goa Day in Goa University.

As part of a presentation , we did a small demo on record and playback of a script using Sahi, then refactored the code to be maintainable.

The site under test is available here: /demo/training
The first cut from the recorder came out to be this:


_setValue(_textbox("user"), "test");
_setValue(_password("password"), "secret");
_click(_submit("Login"));
_setValue(_textbox("q"), "2");
_setValue(_textbox("q[1]"), "1");
_setValue(_textbox("q[2]"), "1");
_click(_button("Add"));
_assertExists(_textbox("total"));
_assert(_isVisible(_textbox("total")));
_assertEqual("1150", _textbox("total").value);
_click(_button("Logout"));

This was then refactored into 2 scripts, one containing functions and the other invoking it:


// goa3_included.sah
function login($username, $password){
_setValue(_textbox("user"), $username);
_setValue(_password("password"), $password);
_click(_submit("Login"));
}

function addBooks($numJava, $numRuby, $numPython){
_setValue(_textbox("q"), $numJava);
_setValue(_textbox("q[1]"), $numRuby);
_setValue(_textbox("q[2]"), $numPython);
_click(_button("Add"));
}

function verifyTotal($total){
_assertEqual($total, _textbox("total").value);
}

function logout(){
_click(_button("Logout"));
}

// goa3.sah
_include("goa3_included.sah");

login("test", "secret");
addBooks(2, 1, 1);
verifyTotal(1150);
logout();

The next step was to modify


function addBooks($numJava, $numRuby, $numPython){
_setValue(_textbox("q"), $numJava);
_setValue(_textbox("q[1]"), $numRuby);
_setValue(_textbox("q[2]"), $numPython);
_click(_button("Add"));
}

such that identifiers “q”, “q[1]” and “q[2]” become more meaningful and are independent of their order. Using the _near API, the function becomes:


function addBooks($numJava, $numRuby, $numPython){
_setValue(_textbox("q", _near(_cell("Core Java"))), $numJava);
_setValue(_textbox("q", _near(_cell("Ruby for Rails"))), $numRuby);
_setValue(_textbox("q", _near(_cell("Python Cookbook"))), $numPython);
_click(_button("Add"));
}

We then data drive the whole test by wrapping the various steps into a single function “addAndVerify”, build a 2 dimensional array of values and then invoke “addAndVerify” for each row of values using _dataDrive


// club the functionality into a single function
function addAndVerify($numJava, $numRuby, $numPython, $total){
login("test", "secret");
addBooks($numJava, $numRuby, $numPython);
verifyTotal($total);
logout();
}

// build a 2D array
var $data = [
[2, 1, 1, 1150],
[3, 2, 1, 1650],
[1, 1, 1, 850]]
// automatically invoke addAndVerify for each row in $data.
_dataDrive(addAndVerify, $data);

We concluded the talk with an enthusiastic Q & A session.
Thank you Goa University for being a great host!

ThoughtWorks Studios’ Twist 2.0 with Sahi

Posted by | Agile, Sahi, ThoughtWorks Studios, Twist, Twist 2.0 | No Comments

ThoughtWorks announces Twist 2.0 availability from 31st March 2010:
http://www.prnewswire.com/news-releases/thoughtworks-studios-new-twist-20-provides-collaborative-agile-test-management-85809582.html

Tyto Software has been collaborating with ThoughtWorks Studios to integrate Sahi with Twist and results will be visible in Twist 2.0.

“Twist 2.0 has added Sahi as an additional option for web testing. The main benefit of Sahi is that it abstracts out most difficulties that testers face while automating web applications. Its features include an excellent recorder, platform and browser independence, no XPaths, no waits and multi-threaded playback. In addition, it allows you to identify UI components within the application as you record test scenarios.” Announcing Twist 2.0: Available for download on March 31

ZK testing with Sahi

Posted by | Sahi, ZK, ZKOSS | No Comments

Joseph Neuhaus has written a detailed article on testing ZK applications with Sahi. The article explains how to run Sahi tests headless on a linux machine.

Excerpts from the well written, thorough article:

If you have attempted to create browser-based functional tests with Selenium, or load tests using Grinder, then you will marvel at the simplicity of Sahi.

The Problem
Testing ZK Web Applications can be a challenge with Selenium and Grinder because ZK dynamically generates element IDs, and these testing frameworks identify elements within the rendered Web Page using these IDs. Therefore, once a session has been recorded, it cannot be replayed because the element IDs will change the next time the Web Application is launched. To address this, ZK provides a hook so you can generate your own IDs. This is a great feature to be sure; however, now you’re not testing the application that will be promoted into production. Also, keeping track of “special” test builds of your application increases work and complexity. To make matters worse, some testing frameworks, such as Selenium, require you to install browser plugins to create the recorded browser sessions used for playback. When browser updates occur, you can’t upgrade until the plugins are updated too. More moving parts means more issues maintaining the Test Environment. There must be a better way, and there is. It’s called Sahi.

The Solution
Sahi can record and playback sessions directly on your production ZK Web Application without having to use a custom ID generator. It requires no browser plugins to create recorded sessions to be used for playback. Sahi is pure Java, so it integrates nicely with ANT. The Sahi scripting language is simple and elegant, so there’s no need to break out the XPATH documentation to get your tests running. You can even run your Sahi tests headless using Firefox in an X-Window virtual frame buffer on Linux. In headless mode you can run real browser-based tests on your continuous integration machines without having to be logged in. If you’re testing on windows, you can run your tests on IE, Firefox, Chrome, Safari, and Opera. Despite some idiosyncrasies Sahi seems magical at times, but more importantly it makes testing productive – so productive it’s almost fun!

The full article is available here: http://docs.zkoss.org/wiki/Making_ZK_Functional_Tests_With_Sahi

Joe Neuhaus has over twenty years of experience in software development, systems design, and technical management roles.

Narayan Raman wins Safari Books Online’s Coder Challenge for Sahi

Posted by | Uncategorized | 2 Comments

Narayan Raman from Tyto Software won Safari Books Online’s Coder Challenge for his contribution to Sahi.

What followed was a day of good fun, interactions with the other winners, wine tasting at Kendell Jackson and Korbel wineries, a pleasant dinner and lots of discussions with Tim O’Reilly, O’Reilly Media, Paige Mazzoni, VP marketing, Safari Books Online, and CJ Rayhill, senior VP, product management and technology, Safari Books Online.

Narayan Raman and the other winners also won a three month subscription to Safari Books Online.

Narayan Raman, Ashley Aberneithy, Tim O’Reilly, Aral Balkan and Arturo Fernandez-Sanchez

CJ Rayhill, Narayan Raman, Ashley Aberneithy, Tim O’Reilly, Aral Balkan, Arturo Fernandez-Sanchez and Paige Mazzoni

Press release: http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20100202005735&newsLang=en

Thank you Safari Books Online for the award and the excellent hospitality.

More pictures on Flickr

Web automation does not need XPaths

Posted by | Uncategorized | 3 Comments

Really.

Learn this web automation nursery rhyme today!

XPaths are evil,
XPaths are fickle,
Developers touch code,
And the testers are in a pickle!

Have you ever used XPaths and found that it needs non-trivial amount of effort in maintenance?

Especially testers, who do not have the time or energy to get XPaths right, stay away from XPaths. Use ids or names or any other attribute the web element provides.

Sahi, since it is aimed at testers rather than developers, does not encourage use of XPath, which means that you can use it if you want, but the controller will never show you XPaths.

So how does Sahi handle something like this?

My name Edit
Your name Edit
His name Edit

Simple, Sahi uses the _in and _near APIs.

So to access the edit link of Your name, use

_link(“Edit”, _near(_cell(“Your name”)))

Like wise

_link(“Edit”, _near(_cell(“My name”)))
_link(“Edit”, _near(_cell(“His name”)))

Visit us again or subscribe to this blog for more tips on web automation …

Sahi Nightly Release 2009-07-15

Posted by | releases | No Comments

A new nightly build is available at https://sourceforge.net/projects/sahi/files/
This fixes a file upload issue and adds support for 401 Authentication and HTTPS Client Certificates.

* Features added
Support for 401 authentication. A dialog box is shown on the browser for authentication.
Support for HTTPS client certificates. Look at ssl.client.* settings in sahi.properties
_hidden(identifier) added
_byClassName(className, tagName [, domRelation]) added

* Bugfixes
checkbox onchange triggered for IE.
Content-Length removed from parts of MultiPartRequest

Parsing XML in Sahi scripts

Posted by | features, Sahi, tutorials, XML | One Comment
Sahi uses Rhino as its javascript engine and Rhino has excellent support for handling XML.
Below is a script which reads and asserts XML nodes and attributes. The example has been picked from http://www.ibm.com/developerworks/webservices/library/ws-ajax1/ so that it is easy to experiment with the ibm examples in this script.


var xmlStr = '' +
'' +
' ' +
' Ant' +
' Shaggy' +
' Blue' +
' 176' +
'
' +
' ' +
' Paul' +
' Spiky' +
' Grey' +
' 178' +
'
' +
'
';

var $x = new XML(xmlStr);
_assertEqual("Ant", $x.person[0].name.toString());
_assertEqual("Grey", $x.person[1].eyes.toString());

for each (var $p in $x.person){
var $measure = $p.height.@measure.toString();
_assert($measure == "metric");
_assert($p.height > 170);
}
Two points to note:
1) All nodes that you access are of type xml. You will need to use toString() on them before you assert them.
2) Using @ from inside a Browser Action Function (like _click, _assert etc.) causes the script to fail because of a parsing error in Sahi’s code. So first assign it to a variable and then use it, like it has been used for $measure. This bug will be fixed in the coming release.
There is a lot more that can be done with the XML object. Have a look at these links:

Configuring Eclipse for Sahi

Posted by | eclipse, IDE, Sahi, tutorials | No Comments
Eclipse’s JSDT plugin provides syntax highlighting and verification for Javascript. It can be configured to work well with Sahi scripts too. This video goes through the different steps to configure Eclipse to work with Sahi scripts.

JSDT looks at all functions in the given source folder and can list them in content-assist. Taking advantage of this, we use a dummy definitions file called apis.sah which has all the Sahi APIs. This file, apis.sah, can be downloaded from here and needs to copied into the “scripts” folder.
JSDT is a part of the Web Platforms Toolkit.
It can be installed as a plugin or can be downloaded as a single bundle in the “Eclipse IDE for Java EE Developers (163 MB)” (For windows: eclipse-jee-ganymede-SR2-win32.zip). This is downloadable from http://www.eclipse.org/downloads/
Use fully-loaded Sahi Pro FREE for a month. Download Now Request a Demo