Sahi Pro - Excel Framework
abstract
The Sahi Excel Framework allows testers to write their testcases in Excel and run it from Sahi. Often a testing team consists of a mix of subject matter experts, some manual testers and testers with some automation experience. Writing tests in the language of the business allows all stake holders to participate and derive value out of the automation process.
Sample Excel Sheet
For example, a simple test written as an Excel sheet looks like this:TestCase | Key Word | Argument1 | Argument2 | Argument3 |
loadSahi | "books_lib.sah" | |||
Check shopping cart total | [Documentation] | Smoke test for add books | ||
login | "test" | "secret" | ||
addBooks | 3 | 2 | 1 | |
verifyTotal | 1640 | |||
logout | ||||
Test login error message | [Documentation] | Checks Invalid login message | ||
login | "test" | "bad password" | ||
verifyNotLoggedIn | ||||
verifyErrorMessage | "Invalid username or password" |
These tests talk mostly in the language of the business (also called a Domain Specific Language or DSL for that business), and hide away all the implementation details of clicking buttons and populating textboxes.
Starting from Sahi Pro 6.1.0, a new format as shown below is introduced. This allows for adding additional columns. The example below shows two additional columns Comments and Tags added to the file.
Comments | Tags | TestCase | Key Word | Argument1 | Argument2 | Argument3 |
Load the function library file | loadSahi | "books_lib.sah" | ||||
First Testcase | smoke, admin | Check shopping cart total | [Documentation] | Smoke test for add books | ||
login | "test" | "secret" | ||||
addBooks | 3 | 2 | 1 | |||
verifyTotal | 1640 | |||||
logout | ||||||
Second Testcase | all, smoke, user | Test login error message | [Documentation] | Checks Invalid login message | ||
login | "test" | "bad password" | ||||
verifyNotLoggedIn | ||||||
verifyErrorMessage | "Invalid username or password" |
At the time of execution from Script Editor or commandline, you can provide the expression for evaluating tags using binary operators. In above case, if you want to execute testcases for admin and user, the tag can be "admin || user".
info
Additional columns can only be added before TestCase column.
A column with heading as "Tags" have a special meaning and will allow you to select the tags during execution.
A column with heading as "Tags" have a special meaning and will allow you to select the tags during execution.
Loading Sahi script libraries
The implementation details are moved into an included Sahi script, which is linked to this Excel sheet via the initial
loadSahi | "books_lib.sah" |
The code in books_lib.sah is given below:
function login($username, $password){
_setValue(_textbox("user"), $username);
_setValue(_password("password"), $password);
_click(_submit("Login"));
}
function addBooks($numJava, $numRuby, $numPython){
_setValue(_textbox("q", _near(_cell("Core Java"))), $numJava);
_setValue(_textbox("q", _near(_cell("Ruby for Rails"))), $numRuby);
_setValue(_textbox("q", _near(_cell("Python Cookbook"))), $numPython);
_click(_button("Add"));
}
function verifyTotal($total){
_assertEqual($total, _textbox("total").value);
}
function logout(){
_click(_button("Logout"));
}
function verifyNotLoggedIn(){
_assertExists(_textbox("user"));
}
function verifyErrorMessage($msg){
_assert(_isVisible(_div("errorMessage")));
_assertEqual($msg, _getText(_div("errorMessage")));
}
Executing the Excel Sheet
Executing the Excel sheet is no different from executing a Sahi script.Just copy the Excel sheet into sahi/userdata/scripts directory (or inside a sub-directory) and the xls file will be visible in the drop down in Controller Playback tab. Choose the file, Click "Set" and then "Play" to playback.
Play back reports/logs
On execution, Sahi generates logs showing success or failure. Logs are visible from the "Logs" link in Playback tab.
Logs can also be accessed via http://localhost:9999/logs
A sample log is shown below. Clicking any step expands to show the underlying Sahi steps. You can click on the log statements below.
Expand All | Collapse All | |||||||||
loadSahi | "sample_lib.sah" | ||||||||
Check shopping cart total | [Documentation] | Smoke test for add books | |||||||
login | "test" | "secret" | |||||||
_setValue(_textbox("user"), "test"); at Mar 9, 2011 11:37:33 AM
_setValue(_password("password"), "secret"); at Mar 9, 2011 11:37:33 AM
_click(_submit("Login")); at Mar 9, 2011 11:37:33 AM
| |||||||||
addBooks | 3 | 2 | 1 | ||||||
_setValue(_textbox("q", _near(_cell("Core Java"))), 3); at Mar 9, 2011 11:37:35 AM
_setValue(_textbox("q", _near(_cell("Ruby for Rails"))), 2); at Mar 9, 2011 11:37:35 AM
_setValue(_textbox("q", _near(_cell("Python Cookbook"))), 1); at Mar 9, 2011 11:37:35 AM
_click(_button("Add")); at Mar 9, 2011 11:37:35 AM
| |||||||||
verifyTotal | 1640 | ||||||||
_assertEqual(1640, _textbox("total").value);
Assertion Failed. Expected:[1640] Actual:[1650] at Mar 9, 2011 11:37:46 AM | |||||||||
logout | |||||||||
_click(_button("Logout")); at Mar 9, 2011 11:37:48 AM
| |||||||||
Test login error message | [Documentation] | Checks Invalid login message | |||||||
login | "test" | "bad password" | |||||||
_setValue(_textbox("user"), "test"); at Mar 9, 2011 11:37:49 AM
_setValue(_password("password"), "bad password"); at Mar 9, 2011 11:37:49 AM
_click(_submit("Login")); at Mar 9, 2011 11:37:49 AM
| |||||||||
verifyNotLoggedIn | |||||||||
_assertExists(_textbox("user")); at Mar 9, 2011 11:37:50 AM
| |||||||||
verifyErrorMessage | "Invalid username or password" | ||||||||
_assert(_isVisible(_div("errorMessage"))); at Mar 9, 2011 11:37:50 AM
_assertEqual("Invalid username or password", _getText(_div("errorMessage"))); at Mar 9, 2011 11:37:50 AM
|
Please refer to sahi/userdata/scripts/demo/framework folder for some examples.
info
- Excel files need to be saved as .xls files in Excel 97-2003 Workbook format.
- You may need to close the Excel sheet before running it
- Only the first sheet will be executed.
- For running excel based scripts via Sahi Pro, MS Excel NEED NOT to be installed on the system. You can even edit these excel files via the WebRunner UI.
- Due to a bug, numbers need to be formatted as text in Excel. One can do this by entering a single quote before entering the number
For example type '23 instead of 23.
Syntax
Test Case | Key word | Argument 1 | Argument 2 | Argument 3 | Very first line of sheet |
Blank lines are ignored | |||||
loadSahi | "custom_lib.sah" | Loads a Sahi script with required function definitions | |||
Test Case One | Step One | Param1 | Param2 | New test case started.StepOne(Param1, Param2) is called. | |
Step Two | Param3 | StepTwo(Param3) is called | |||
Test Case Two | Step One | 25 | "age" | New test case started.String values are quoted | |
Step Two | Param5 | ||||
Test Case Three | [Documentation] | Some description about the test case | [Documentation] is useful for, well, documentation | ||
Step One | 25 | "age" | New test case started.String values are quoted | ||
// | Step Two | Param5 | Commented step using // | ||
Step Two | Param6 | ||||
Spaces will be removed from keywords and corresponding functions invoked.
The rules for writing the Excel sheet are as follows
The first line should be populated with
Test Case | Key word | Argument 1 | Argument 2 | Argument 3
The names of the columns are not important, but they should not be left blank
If the first column is populated, a new test case is started.
The second column holds keywords. Keywords are mapped to functions in the included Sahi script.
They can be user defined functions or Sahi APIs themselves
For example,
login | "test" | "secret" |
login("test", "secret");
_assertEqual | _getText(_cell("msg")) | "abcd" |
_assertEqual(_getText(_cell("msg")), "abcd");
Variables
The Excel framework also supports variables, eg.
Simple variables
$amount= | 1000 | |||
verifyAmount | $amount |
Assigning value returned by function
Eg. To get the value returned by functioncreateUserInGroup
:Using [ReturnValue]
createUserInGroup | "My name" | "My group" | ||
$userId= | [ReturnValue] | |||
verifyUserCreated | $userId | "My name" | "My group" |
info
[ReturnValue]
is a keyword to access return value of function executed in previous step. Added since Sahi Pro 6.1.0Inline declaration
$userId=createUserInGroup | "My name" | "My group" | ||
verifyUserCreated | $userId | "My name" | "My group" |
Inline as code
$msg= | _getText(_cell("msg")) | |||
_assertEqual | $msg | "abcd" |
SetUp and TearDown
Different test cases may need the same steps to be executed before and after.
For example, one may need to login before and logout after each test case.
This can be accomplished through global SetUp and TearDown blocks.
TearDown will be called inspite of any errors or failures in the testcase.
infoThe [Global] keyword is mandatory and defines these [Setup] and [Teardown] methods for all testcase blocks.
[Global] | [SetUp] | |||
_log | "In Global Setup" | |||
login | "test" | "secret" | ||
[TearDown] | ||||
_click | _button("Logout") | |||
_log | "In Global Teardown" | |||
Verify books total | [Documentation] | Check once | ||
addBooks | 3 | 2 | 1 | |
verifyTotal | 1650 | |||
Verify books again | [Documentation] | Check again | ||
addBooks | 3 | 2 | 2 | |
verifyTotal | 2000 |
This will execute as:
_log | "In Global Setup" | |||
login | "test" | "secret" | ||
addBooks | 3 | 2 | 1 | |
verifyTotal | 1650 | |||
_click | _button("Logout") | |||
_log | "In Global Teardown" | |||
_log | "In Global Setup" | |||
login | "test" | "secret" | ||
addBooks | 3 | 2 | 2 | |
verifyTotal | 2000 | |||
_click | _button("Logout") | |||
_log | "In Global Teardown" |
Creating Keywords on the fly in Excel
Keywords are normally created as functions in the included Sahi scripts, so that the details are hidden away and one gets good programmatic control in the scripts.But sometimes it may be convenient to club together different steps and create a keyword in the Excel sheet itself.
Create AddBooksCheck Key Word | [CreateKeyword] | Add Books Check | [CreateKeyword] command creates a keyword called "Add Books Check" | |||
[Arguments] | $java | $ruby | $python | $total | These are the arguments or parameters that "Add Books Check" takes as input | |
[Documentation] | Data drivable add books check | |||||
addBooks | $java | $ruby | $python | |||
verifyTotal | $total |
So now, a call to "Add Books Check" like this:
Add Books Check | 3 | 2 | 1 | 1650 |
is equivalent to
addBooks | 3 | 2 | 1 | |
verifyTotal | 1650 |
Data driven testing
Data Driven Example | [Keyword] | Add Books Check | Starts a testcase "Data Drivern Example" AddBooksCheck function will be called for the data set below. | |||
[SetUp] | This will be called BEFORE each row of data | |||||
login | "test" | "secret" | ||||
[TearDown] | This will be called AFTER each row of data | |||||
_click | _button("Logout") | |||||
[Documentation] | java | ruby | python | total | Ignored by Sahi. Makes test case more readable | |
[Data] | 3 | 2 | 1 | 1650 | [Data] denotes start of data | |
4 | 5 | 0 | 2100 | |||
0 | 1 | 9 | 3350 |
This roughly translates to:
login | "test" | "secret" | |||
Add Books Check | 3 | 2 | 1 | 1650 | |
_click | _button("Logout") | ||||
login | "test" | "secret" | |||
Add Books Check | 4 | 5 | 0 | 2100 | |
_click | _button("Logout") | ||||
login | "test" | "secret" | |||
Add Books Check | 0 | 1 | 9 | 3350 | |
_click | _button("Logout") | ||||
Using External Data
Normally, parameter data is passed inline to keywords/functions in scenario files.However, one may want to keep the parameter data in a separate file for easier maintenance.
Sahi Pro 6.2 adds the ability to represent data in external files or database and allows an easy way of accessing such externalized data.
There can be three different ways of accessing external data in Sahi's excel framework:
1) Through CSV file by using _readCSVFile api
D1=_readCSVFile | sample_data.csv |
2) Through excel file by using _readExcelFile api
D1=_readExcelFile | sample_data.xls |
3) Through database by using _getDB api
$db= | _getDB($jdbcDriver, $jdbcURL, "", "") | ||
$sql= | "SELECT * FROM EXTERNALDATA" | ||
D2= | $db.selectWithHeader($sql) |
To understand how this data can be used in scenario files.
Consider a situation where a new user is to be created and added to the database. Each user has four properties lets say firstname, lastname, age, gender.
One wants to pass these parameters as data in a function called "addUser()", which takes these four parameters in the same sequence as mentioned above.
function addUser(firstname, lastname, age, gender) {
...
}
Now, lets have a look at how the external data can look like.Case 1 | data1 | Shyam | Sundar | 11 | male |
Case 2 | data2 | firstname | lastname | age | gender |
Shyam | Sundar | 11 | male | ||
Jack | Sparrow | 21 | male | ||
Case 3 | data3 | age | gender | firstname | lastname |
11 | male | Shyam | Sundar | ||
21 | male | Jack | Sparrow | ||
Case 4 | data4 | firstname | lastname | age | gender |
Jack | Sparrow | 21 | male | ||
Shyam | Sundar | 11 | male | ||
From database:
Case 5 | firstname | lastname | age | gender | |
Jack | Sparrow | 21 | male | ||
Shyam | Sundar | 11 | male |
When data has a single row
Case1
Here data1 is a 1-dimensional array, which can be directly accessed as [D1:data1] and passed to the addUser function.TC:addUser | [Documentation] | Add User to the database |
addUser | [D1:data1] |
Or
TC:addUser | [Keyword] | addUser |
[Data] | [D1:data1] |
When data has multiple rows
Case2
Here data2 is a 2-dimensional array and column sequence is same as that required by the function. So it can either be passed directly as [D1:data2] or each parameter in the same sequence as required.TC:addUser | [Keyword] | addUser | |||
[Data] | [D1:data2] | ||||
// or | [Data] | [D1:data2:firstname] | [D1:data2:lastname] | [D1:data2:age] | [D1:data2:gender] |
Case3
Here also data3 is a 2-dimensional array but sequence of the columns is not the same as the function. So here wecan not
pass it directly as [D1:data3].TC:addUser | [Keyword] | addUser | |||
[Data] | [D1:data3:firstname] | [D1:data3:lastname] | [D1:data3:age] | [D1:data3:gender] |
Case4
This is same as the Case2. Empty rows will be ignored.Case5
This is the case when we fetch data from any database using _getDB. There is no key like "data1" here so each column can be accessed as [D2::columnName].TC:addUser | [Keyword] | addUser | |||
[Data] | [D2::firstname] | [D2::lastname] | [D2::age] | [D2::gender] |