Load Testing with Sahi Pro

abstract Sahi Pro lets you run load tests on your application. There are 2 Sahi scripts involved in this. One is used to generate the load. This script is called the noise and is run on a headless browser. The steps in the noise file are generally in an infinite loop so that they never terminate. The other script is called the subject. The subject is the script whose steps you would like to measure periodically at different loads.

Setting up

  1. Get the headless browser, PhantomJS. Read this article to add PhantomJS to Sahi.
  2. Start Sahi Pro. PhantomJS should be visible on the Dashboard as one of the browsers.
    warningPhantomJS is normally visible on the Dashboard even if it is not installed. Make sure sahi/ext/phantomjs/phantomjs.exe exists.
  3. On the dashboard, click on the 'Bin' link. This will open up a command prompt at userdata/bin.
  4. On the command prompt, run
    dload.bat demo/load/noise.sah demo/load/subject.sah http://sahitest.com/demo/training/ firefox
This will do the following:
  1. Run 1 (min) thread(s) of noise.sah on PhantomJS (This will not be visible)
  2. Run subject.sah on firefox browser and store the time of steps. Subject will be run 3 (subjectRepeatCount) times for averaging.
  3. Wait for 5 seconds (interval).
  4. Run 2 (incrementBy) more threads of noise.sah on PhantomJS.
  5. Run subject.sah 3 times on firefox and store results.
  6. Keep incrementing the noise threads at periodic intervals, run the subject 3 times, and store results, till it reaches max noise threads. Once done, click on 'Logs' link on the Dashboard to view results.

Reporting

Reports show the most expensive steps and URLs in the load test. Clicking on the circles in the graphs takes you to the step/URL which was expensive.

Most Expensive Steps Across Different Loads: Most Expensive URLs Across Different Loads: Steps:
info Note: The table contents are truncated in this image. You should see entries for Load: 9 as well, in the actual log.
To test your application, go to userdata/scripts/demo/load folder.
  1. Open noise.sah and add your sequence of steps. These steps should be representative of a normal user interaction. Note that your steps should be in a while(true){} loop so that the threads don't die out. (Refer to existing noise.sah)
  2. Open subject.sah and specify the sequence of steps whose reaction times you would like to measure at different loads.
  3. The parameters min, max, incrementby, interval, subjectRepeatCount etc. can be edited in dload.bat file.
  4. It is also possible to involve multiple machines in creating the load by modifying the NODES variable (Look in dload.bat)
  5. To get HAR logs and URL timing information, use _startHarLogging() in your script to start logging HAR logs. To stop the logs, use _stopHarLogging().

Running load tests through ANT

Sahi supports load testing through an ANT target.

Copy the following content to an empty file and save the file as an xml, say dload.xml in your <SAHI_INSTALLATION_FOLDER>. You can now run the default ANT target in the xml file as
ant -f dload.xml


<?xml version="1.0" encoding="ISO-8859-1"?>
<!--
/**
 * Copyright Tyto Software Pvt. Ltd.
 */
-->
<project name="demo" default="drunfirefox">
	<taskdef name="sahid" classname="in.co.sahi.ant.DAntLoadRunner">
		<classpath>
			<pathelement location="lib/ant-sahi.jar"/>
		</classpath>
	</taskdef>

	<target name="set_timestamp">
		<tstamp>
			<format property="ts" pattern="yyyy_MM_dd_HH_mm_ss" locale="en, IN"/>
		</tstamp>
	</target>

	<target name="set_properties_common" depends="set_timestamp">
		<property name="scriptDir" value="scripts/"/>
		<property name="host" value="localhost"/>
		<property name="port" value="9999"/>
	</target>

	<target name="set_properties_firefox" depends="set_properties_common">
		<property name="subject" value="demo/load/subject.sah"/>
		<property name="noise" value="demo/load/noise.sah"/>
		<property name="browser" value="chrome"/>
		<property name="baseURL" value="http://sahitest.com/demo/training/"/>
		<property name="noiseBrowserType" value="phantomjs"/>
		<property name="minThread" value="1"/>
		<property name="maxThread" value="9"/>
		<property name="increment" value="2"/>
		<property name="interval" value="5"/>
		<property name="repearCount" value="3"/>
	</target>

	<target name="drunfirefox" depends="set_properties_firefox, set_timestamp">
		<sahid noise="${noise}"
			noiseBrowserType="${noiseBrowserType}"
			min="${minThread}"
			max="${maxThread}"
			incrementBy="${increment}"
			interval="${interval}"
			subject="${subject}"
			subjectRepeatCount="${repearCount}"
			sahihost="${host}"
			sahiport="${port}"
			baseURL="${baseURL}"
			browsertype="${browser}"
			scriptsPathMaster="${scriptDir}"
			ignorePattern=".*(svn|copied).*">
			<!-- Scripts will be distributed across all the nodes. The nodes may or may not include the Master machine -->
			<node host="localhost" port="9999"/>
			<!-- Offline reports -->
			<report type="html" logdir="logs/playback/html"/>
		</sahid>

		<antcall target="failsahifirefox"/>
	</target>

	<target name="failsahifirefox" depends="set_properties_firefox" if="sahi.failed.${browser}">
		<fail message="Sahi tests failed on browser ${browser}!"/>
	</target>
</project>


Attributes and their description

info taskdef classpath is relative to where ant is run from.
scriptDirSpecifies the path to the scripts folder. Can be the absolute path or a path relative to the userdata folder
hostHostname of the server where Sahi is running (Can be IP as well). Leave this as localhost
portPort on which Sahi is running
subjectPath to the subject script.
noisePath to the noise script.
browserThe browser on which the subject script is run (can be IE, Firefox etc.)
baseurlSpecifies URL which is the starting URL for the subject and noise scripts
noiseBrowserTypeThe browser on which the noise script is run (default phantomJS)
minThreadThe minimum number of noise browsers to run initially
maxThreadThe maximum number of noise browsers to run finally
incrementThe number of noise browsers to be run incrementally after the specified interval
intervalThe interval after which to increment the noise browsers by
repearCountNumber of times the subject script should be run at every noise level. It is good to run it more than 1 time to average out any random behaviour.
nodeEach node attribute specifies a machine on which the noise browsers should run. Add as many node entries as there are machines to run. The nodes may or may not include the Master machine (localhost). If the Master machine is not included, the noise browsers will not be run on the Master. There can be 1 or more nodes.
reportThis attribute is used to store offline report. type parameter specify the format in which the offline logs should be downloaded and logdir is the file location of the offline logs.

Command to execute the above ANT target

ant -f dload.xml