Playback via Command Line

abstract During development, scripts will be played back from the Controller or Editor. For Continuous Integration systems, create an ANT target and trigger from there. This section describes a way of triggering suites/scripts from the command line via bat files or shell scripts.
warning We recommend using the ANT target over using the command line. This section is for backward compatibility.

Playback via testrunner

testrunner.bat and testrunner.sh files allow running scripts from the command line.
  1. Open a command prompt and cd to sahi/userdata/bin (or click on the "Bin" link on the Sahi Dashboard)
  2. Run testrunner.bat to see the syntax
    D:\sahi_pro\userdata\bin>testrunner.bat -- Usage: testrunner <sah file|suite file> <startURL> <browserType> <tags> File path is relative to userdata/scripts Multiple browsers can be specified using +. Eg. ie+firefox Modes are available in Sahi Pro Desktop Add-on only START_MODE can be : WINDOWS, JAVA, JAVA_APPLET tags are used only if the input file is a .csv file, .s.csv file or a .dd.csv file -- Example: testrunner demo/demo.suite http://sahitest.com/demo/ firefox testrunner demo/sahi_demo.sah http://sahitest.com/demo/ ie testrunner demo/sahi_demo.sah http://sahitest.com/demo/ ie+firefox testrunner demo/demo.dd.csv http://sahitest.com/demo/ firefox "win||all" testrunner demo/testcases/testcases_sample.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" testrunner demo/ddcsv/test.dd.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" testrunner demo/framework/sample.xls http://sahitest.com/demo/training/ ie testrunner demo/framework/scenario_new.s.csv http://sahitest.com/demo/training/ ie "all" testrunner script_name.sah http://sahitest.com/demo/ "chrome;windows" D:\sahi_pro\userdata\bin>
  3. Parameters other than suite, startURL and browserType can be modified by editing testrunner.bat
    1. To set the number of parallel browsers to execute the script in, modify
      SET THREADS=5
      To run in one thread only, ie. to run scripts one after the other sequentially, use
      SET THREADS=1
    2. Sahi normally opens and closes the browser for each script. To execute all scripts sequentially without closing and reopening browsers, use
      SET THREADS=1
      SET SINGLE_SESSION=true
    3. Sahi, by default, writes logs to the database only. To force it to write logs to the filesystem, use
      SET LOGS_INFO=html,xml
      html and xml logs will be created in the sahi/userdata/logs/playback folder

      To force it to write logs to specific file locations, use,
      SET LOGS_INFO=html:D:/reports/htmllogs/,xml:D:/reports/xmllogs/


      Output formats available are html,xml,junit,excel

      The format to specify LOGS_INFO is
      reportType1:filePath1,reportType2,reportType3:filePath3,...
    4. When a Sahi suite is run, some scripts may fail because of environment reasons or due to browser crashes. A natural next step for testers is to verify the failure by running the script again. Sahi now automatically creates a suite of all failed scripts. The name of the failed suite has _failed_<browsertype> appended to the filename. For example if you ran a suite admin/admin.suite on Firefox, failed suite will be created as admin/admin_failed_firefox.suite For admin/admin.dd.csv, it will be admin/admin_failed_firefox.dd.csv

      This failed suite file can then be run to verify only the failed testcases.
    5. Sahi can send out emails at the end of a suite run. Set the following properties in testrunner.bat file.
      SET SEND_EMAIL_REPORT=true
      SET EMAIL_TRIGGER=success,failure
      SET EMAIL_PROPERTIES=email.properties
      SET EMAIL_PASSWORD_HIDDEN=true
      Email specific parameters like username, password etc. are defined in sahi/userdata/bin/email.properties file.
    6. For a long running suite, Sahi can send periodic emails indicating progress. To enable periodic emails, set the following property in testrunner.bat file.

      SET SEND_EMAIL_REPORT_PERIODICALLY=true


      To set time interval (in minutes), set the following property in testrunner.bat file.

      SET SEND_EMAIL_REPORT_PERIODICALLY_TIME=30
    7. Sahi Pro v6.3.0 onwards, a summary of the script playback status is written to the Testrunner console while the script is running. You can read more on Periodic Summary here.

      To disable these messages in Windows, set the following property in testrunner.bat file.
      SET SHOW_PERIODIC_SUMMARY=false
      For Mac and Linux, you need to set the following property in testrunner.sh
      export SHOW_PERIODIC_SUMMARY=false
info Browser names are defined in sahi/userdata/browser-types-xml testrunner.bat and testrunner.sh use the TestRunner java class internally.

Distributed playback via drun

Sahi Pro can distribute and run tests on multiple machines (nodes). The machine which distributes the scripts and manages the distributed run is referred to as the Master. All Dashboard logs are visible from the Master. The other machines are called Slaves. Any machine in the distributed environment can serve as the Master. The machine which launches the distributed run is referred to as the Initiator. Sahi provides a couple of batch files - drun.bat and drun_different_master.bat (and equivalent shell scripts). They allow running test scripts on multiple machines from the command line.
danger Please ensure that the same SahiPro version has been installed on all the machines participating in the distributed run.

drun

  1. Sahi distributes the scripts to different machines based on their capacity and availability of browsers. Please note that Sahi should be running on each of the nodes. Also, on each machine, the browser paths must be correctly specified.
  2. Open sahi_pro/userdata/bin/drun.bat (drun.sh on linux) using any editor.
  3. SOURCE_SCRIPTS_PATH points to the folder that contains ALL the scripts to be run. It should be relative to sahi_pro/userdata folder OR it should be an absolute path.

    For Windows:
    SET SOURCE_SCRIPTS_PATH=scripts
    For Linux:
    export SOURCE_SCRIPTS_PATH=scripts
  4. Configure the NODES variable to contain all the machines on which the tests should run. The command to set the nodes in Windows and Linux respectively are: For Windows:
    SET NODES=localhost:9999,othermachine:9999,thirdmachine:9999
    For Linux:
    export NODES=localhost:9999,othermachine:9999,thirdmachine:9999
    info The nodes may or may not include the Master machine (localhost). If the Master machine is not included, scripts will not be run on the Master.
    warning "othermachine" and "thirdmachine" are placeholders to illustrate usage. Replace them with actual machine names. The nodes can contain 1 or more machines.
  5. From SahiPro 6.1.0 onwards, it is possible to specify the number of threads that the scripts of the suite would run in. The number of maximum parallel browsers would be the minimum of two values - threads, and capacity of the browser specified in browser_types.xml.

    To set the number of parallel browsers to execute the script in, modify
    SET THREADS=5
    To run in one thread only, ie. to run scripts one after the other sequentially, use
    SET THREADS=1


    Please note that this would affect the number of parallel browsers on each Node.
  6. When a Sahi suite is run, some scripts may fail because of environment reasons or due to browser crashes. A natural next step for testers is to verify the failure by running the script again. Sahi now automatically creates a suite of all failed scripts. The name of the failed suite has _failed_<browsertype> appended to the filename. For example if you ran a suite admin/admin.suite in Chrome, failed suite will be created as admin/admin_failed_chrome.suite For admin/admin.dd.csv, it will be admin/admin_failed_chrome.dd.csv

    This failed suite file can then be run to verify only the failed scripts.
  7. Sahi can send a mail summarizing the playback. The variable SEND_EMAIL_REPORT can be set as true or false. If true, Sahi sends the mail.

    For Windows:
    SET SEND_EMAIL_REPORT=true
    For Linux:
    export SEND_EMAIL_REPORT=true
  8. Sahi can send a mail based on success or failure or in both the cases. This condition can be set using the EMAIL_TRIGGER variable.

    For Windows:
    SET EMAIL_TRIGGER=success,failure
    For Linux:
    export EMAIL_TRIGGER=success,failure
  9. For a long running suite, Sahi can send periodic emails indicating progress. To enable periodic emails, set the following property.

    For Windows:
    SET SEND_EMAIL_REPORT_PERIODICALLY=true
    For Linux:
    export SEND_EMAIL_REPORT_PERIODICALLY=true


    To set time interval (in minutes), use SET SEND_EMAIL_REPORT_PERIODICALLY_TIME variable.

    For Windows:
    SET SEND_EMAIL_REPORT_PERIODICALLY_TIME=30
    For Linux:
    export SEND_EMAIL_REPORT_PERIODICALLY_TIME=30
  10. The relative path to the properties file containing information about the mail such as sender, recipient, host, port etc. can be set using the EMAIL_PROPERTIES variable.
  11. Passwords can be hidden or shown from the console and logs by toggling the EMAIL_PASSWORD_HIDDEN variable. It can be set as true or false. If true, Sahi hides the password from Sahi console and logs.
  12. Custom fields can be passed to the scripts using the CUSTOM_FIELDS variable. More than one custom field can be passed. Replace the custom field keys and values as appropriate.

    For Windows:
    SET CUSTOM_FIELDS=-customField customValue -anotherCustomField "another value"
    For Linux:
    export CUSTOM_FIELDS="-customField customValue -anotherCustomField another value"


    info Specifying custom fields is not mandatory.
  13. The end user can pass a specific id for each suite/script run using the USER_DEFINED_ID variable.

    For Windows:
    SET USER_DEFINED_ID=-userDefinedId  "Some Id"
    For Linux:
    export USER_DEFINED_ID="-userDefinedId  Some Id"


    Replace the value "Some Id" as appropriate.
    warning The key should remain as userDefinedId.
    info Specifying user defined id is not mandatory.
  14. Sahi can set offline logs to be generated in xml, html, junit, tm6 and excel types. The default type is html.

    For Windows:
    SET LOGS_INFO=xml,html
    For Linux:
    export LOGS_INFO=xml,html
  15. Sahi Pro v6.3.0 onwards, a summary of the script playback status is written to the Drun console while the script is running. You can read more on Periodic Summary here.

    To disable these messages in Windows, set the following property in drun.bat file.
    SET SHOW_PERIODIC_SUMMARY=false
    For Mac and Linux, you need to set the following property in drun.sh
    export SHOW_PERIODIC_SUMMARY=false
  16. To execute the scripts using drun.bat, open a command window and go to the <sahi_home>/userdata/bin directory. This can also be done easily by clicking the bin link on the dashboard.
  17. Run drun.bat to see the syntax.
    D:\sahi_pro\userdata\bin>drun.bat -- NOTE: Use this batch file only if you are running it on the Master machine. If you wish to use a different Master, use drun_different_master.bat instead. -- Usage: drun <sah file|suite file> <startURL> <browserType> <tags> File path is relative to userdata/scripts Multiple browsers can be specified using +. Eg. ie+firefox Modes are available in Sahi Pro Desktop Add-on only START_MODE can be : WINDOWS, JAVA, JAVA_APPLET tags are used only if the input file is a .csv file, .s.csv file or a .dd.csv file -- Example: drun demo/demo.suite http://sahitest.com/demo/ firefox drun demo/sahi_demo.sah http://sahitest.com/demo/ ie drun demo/sahi_demo.sah http://sahitest.com/demo/ ie+firefox drun demo/demo.dd.csv http://sahitest.com/demo/ firefox "win||all" drun demo/testcases/testcases_sample.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" drun demo/ddcsv/test.dd.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" drun demo/framework/sample.xls http://sahitest.com/demo/training/ ie drun demo/framework/scenario_new.s.csv http://sahitest.com/demo/training/ ie "all" drun script_name.sah http://sahitest.com/demo/ "chrome;windows" -- D:\sahi_pro\userdata\bin>


    danger NOTE:  Please refer to the Notes below for actual details.
    In the usage above,
    • The first argument is the suite or script path. This path can be relative to the path set in SOURCE_SCRIPTS_PATH, or it can be an absolute path contained in SOURCE_SCRIPTS_PATH.
    • The second argument is the Start URL.
    • The third argument is the Browser Type.

    • Tags (the fourth argument) will be used only when running a scenario file, a data driven suite (.dd.csv) or a CSV as suite with tags. Tags are specified to run specific scripts/testcases from the suite/scenario. eg. If tags are specified as "(user||admin)&&medium", all the scripts which have 'medium' tag and 'admin' or 'user' (or both) tag will be run

drun_different_master

This should be used when the Master is different from the Initiator. A typical case where you would use this or the ANT equivalent, is when you have a continuous integration build server that is not capable of launching any UI. The build server will contain all the scripts but since it cannot launch any UI, it cannot run Sahi tests. In such a case, Sahi can be run on other machines and drun_different_master can be used on the integration build server to trigger the Sahi tests on these machines.

drun_different_master does the following. Let us look at drun_different_master in more detail.
  1. Sahi (running on the Master) distributes the scripts to different machines based on their capacity and availability of browsers. Please note that Sahi should be running on each of the nodes. Also, on each machine, the browser paths must be correctly specified.
    info Sahi does not need to be installed on the Initiator machine.
  2. The following directory structure is assumed on the Initiator machine. The requisite files like ant-sahi.jar should be copied over from a SahiPro installation. The directory structure of the files is the same in the SahiPro installation.
    <TOP_LEVEL_FOLDER>
    	lib
    		ant-sahi.jar
    	logs
    	userdata
    		bin
    			drun_different_master.bat
    			setjava.bat
    		config
    			email.properties
    		scripts
    			<ALL YOUR SCRIPT FOLDERS AND SCRIPTS>
  3. Open <sahi_home>/userdata/bin/drun_different_master.bat (drun_different_master.sh on linux) using any Editor.
  4. MASTER_HOST should point to the Master machine. For Windows:
    SET MASTER_HOST=machine2
    For Linux:
    export MASTER_HOST=machine2
    danger Make sure you change "machine2" to the actual machine name
  5. INITIATOR_ORIGIN_FOLDER is the path to the base folder containing the various scripts. With the above assumed directory structure, it would point to <TOP_LEVEL_FOLDER>/userdata/scripts.
    info The default value need not be changed.
  6. MASTER_STAGING_PATH refers to the Staging folder on the Master machine to which the contents of INITIATOR_ORIGIN_FOLDER will be first synced to. Distribution of scripts will happen from MASTER_STAGING_PATH.
    info The default value need not be changed.
  7. Configure the NODES variable to contain all the machines on which the tests should run. The command to set the nodes in Windows and Linux respectively are: For Windows:
    SET NODES=machine2:9999,machine3:9999,machine4:9999
    For Linux:
    export NODES=machine2:9999,machine3:9999,machine4:9999
    info The nodes may or may not include the Master machine. If the Master machine is not included, scripts will not be run on the Master.
    danger Do not include localhost (the Initiator) here since Sahi may not be installed on the Initiator. If you want localhost as one of the nodes, run drun instead.
    warning "machine2", "machine3" and "machine4" are placeholders to illustrate usage. Replace them with actual machine names. The nodes can contain 1 or more machines.
  8. From SahiPro 6.1.0 onwards, it is possible to specify the number of threads that the scripts of the suite would run in. The number of maximum parallel browsers would be the minimum of two values - threads, and capacity of the browser specified in browser_types.xml.

    To set the number of parallel browsers to execute the script in, modify
    SET THREADS=5
    To run in one thread only, ie. to run scripts one after the other sequentially, use
    SET THREADS=1


    Please note that this would affect the number of parallel browsers on each Node.
  9. When a Sahi suite is run, some scripts may fail because of environment reasons or due to browser crashes. A natural next step for testers is to verify the failure by running the script again. Sahi now automatically creates a suite of all failed scripts. The name of the failed suite has _failed_<browsertype> appended to the filename. For example if you ran a suite admin/admin.suite in Chrome, failed suite will be created as admin/admin_failed_chrome.suite For admin/admin.dd.csv, it will be admin/admin_failed_chrome.dd.csv

    This failed suite file can then be run to verify only the failed scripts.
  10. Sahi can send a mail summarizing the playback. The variable SEND_EMAIL_REPORT can be set as true or false. If true, Sahi sends the mail. For Windows:
    SET SEND_EMAIL_REPORT=true
    For Linux:
    export SEND_EMAIL_REPORT=true
  11. Sahi can send a mail based on success or failure or in both the cases. This condition can be set using the EMAIL_TRIGGER variable. For Windows:
    SET EMAIL_TRIGGER=success,failure
    For Linux:
    export EMAIL_TRIGGER=success,failure
  12. For a long running suite, Sahi can send periodic emails indicating progress. To enable periodic emails, set the following property.

    For Windows:
    SET SEND_EMAIL_REPORT_PERIODICALLY=true
    For Linux:
    export SEND_EMAIL_REPORT_PERIODICALLY=true


    To set time interval (in minutes), use SET SEND_EMAIL_REPORT_PERIODICALLY_TIME variable.

    For Windows:
    SET SEND_EMAIL_REPORT_PERIODICALLY_TIME=30
    For Linux:
    export SEND_EMAIL_REPORT_PERIODICALLY_TIME=30
  13. The relative path to the properties file containing information about the mail such as sender, recipient, host, port etc. can be set using the EMAIL_PROPERTIES variable.
  14. Passwords can be hidden or shown from the console and logs by toggling the EMAIL_PASSWORD_HIDDEN variable. It can be set as true or false. If true, Sahi hides the password from Sahi console and logs.
  15. Custom fields can be passed to the scripts using the CUSTOM_FIELDS variable. More than one custom field can be passed. Replace the custom field keys and values as appropriate. For Windows:
    SET CUSTOM_FIELDS=-customField customValue -anotherCustomField "another value"
    For Linux:
    export CUSTOM_FIELDS=-customField customValue -anotherCustomField "another value"


    info Specifying custom fields is not mandatory.
  16. The end user can pass a specific id for each suite/script run using the USER_DEFINED_ID variable. For Windows:
    SET USER_DEFINED_ID=-userDefinedId  "Some Id"
    For Linux:
    export USER_DEFINED_ID=-userDefinedId  "Some Id"


    Replace the value "Some Id" as appropriate.
    warning The key should remain as userDefinedId.
    info Specifying user defined id is not mandatory.
  17. Sahi can set offline logs to be generated in xml, html, junit, tm6 and excel types. The default type is html. These logs will be generated on the Master and pulled back to the Initiator, since the user would want to see the logs on the Initiator. The following set of commands illustrate how HTML and XML logs can be generated.

    For Windows:
    SET UNIQUE_ID=%DATE%__%TIME%
    SET UNIQUE_ID=%UNIQUE_ID: =_%
    SET UNIQUE_ID=%UNIQUE_ID::=_%
    SET UNIQUE_ID=%UNIQUE_ID:/=_%
    
    SET MASTER_HTMLLOGS_DIR=logs/temp/html/%UNIQUE_ID%
    SET MASTER_XMLLOGS_DIR=logs/temp/xml/%UNIQUE_ID%
    
    SET LOGS_INFO=html:%MASTER_HTMLLOGS_DIR%,xml::%MASTER_XMLLOGS_DIR%
    
    SET INITIATOR_OUTPUT_HTMLLOGS_DIR=%TOP_LEVEL_FOLDER%/logs/html/%UNIQUE_ID%
    SET INITIATOR_OUTPUT_XMLLOGS_DIR=%TOP_LEVEL_FOLDER%/logs/xml/%UNIQUE_ID%
    For Linux:
    export UNIQUE_ID=$(date +"%m_%d_%Y")__$(date +"%I_%M_%S")
    
    export MASTER_HTMLLOGS_DIR=logs/temp/html/$UNIQUE_ID
    export MASTER_XMLLOGS_DIR=logs/temp/xml/$UNIQUE_ID
    
    export LOGS_INFO=html:$MASTER_HTMLLOGS_DIR,xml:$MASTER_XMLLOGS_DIR
    
    export INITIATOR_OUTPUT_HTMLLOGS_DIR=$TOP_LEVEL_FOLDER/logs/html/$UNIQUE_ID
    export INITIATOR_OUTPUT_XMLLOGS_DIR=$TOP_LEVEL_FOLDER/logs/xml/$UNIQUE_ID
    info The default values can be left as is.
  18. Sahi Pro v6.3.0 onwards, a summary of the script playback status is written to the Drun_Diff_Master console while the script is running. You can read more on Periodic Summary here.

    To disable these messages in Windows, set the following property in drun_diff_master.bat file.
    SET SHOW_PERIODIC_SUMMARY=false
    For Mac and Linux, you need to set the following property in drun_diff_master.sh
    export SHOW_PERIODIC_SUMMARY=false
  19. To execute the scripts using drun_different_master.bat, open a command window and go to the <sahi_home>/userdata/bin directory. This can also be done easily by clicking the bin link on the dashboard.
  20. Run drun_different_master.bat to see the syntax.

    D:\sahi_pro\userdata\bin>drun_different_master.bat -- NOTE: Use this batch file only if you plan to use a different Master machine. Else, use drun.bat instead. -- Usage: drun_different_master.bat <sah file|suite file> <startURL> <browserType> <tags> File path is relative to userdata/scripts Multiple browsers can be specified using +. Eg. ie+firefox Modes are available in Sahi Pro Desktop Add-on only START_MODE can be : WINDOWS, JAVA, JAVA_APPLET tags are used only if the input file is a .csv file, .s.csv file or a .dd.csv file -- Example: drun_different_master.bat demo/demo.suite http://sahitest.com/demo/ firefox drun_different_master.bat demo/sahi_demo.sah http://sahitest.com/demo/ ie drun_different_master.bat demo/sahi_demo.sah http://sahitest.com/demo/ ie+firefox drun_different_master.bat demo/testcases/testcases_sample.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" drun_different_master.bat demo/ddcsv/test.dd.csv http://sahitest.com/demo/ ie "(user||admin)&&medium" drun_different_master.bat demo/framework/sample.xls http://sahitest.com/demo/training/ ie drun_different_master.bat demo/framework/scenario_new.s.csv http://sahitest.com/demo/training/ ie "all" drun_different_master.bat script_name.sah http://sahitest.com/demo/ "chrome;windows" -- D:\sahi_pro\userdata\bin>


    danger NOTE:  Please refer to the Notes below for actual details.
    In the usage above,
    • The first argument is the suite or script path. This path can be relative to the path set in INITIATOR_ORIGIN_FOLDER, or it can be an absolute path contained in INITIATOR_ORIGIN_FOLDER.
    • The second argument is the Start URL.
    • The third argument is the Browser Type.

    • Tags (the fourth argument) will be used only when running a data driven suite (.dd.csv) or a CSV as suite with tags. Tags are specified to run specific scripts/testcases from the suite. eg. If tags are specified as "(user||admin)&&medium", all the scripts which have 'medium' tag and 'admin' or 'user' (or both) tag will be run.