Authorization Service: Load and Lifetime Testing
It is important to perform load and lifetime tests on any service being put in to production. Load tests are used to determine how the service behaves, and eventually fails, under load while lifetime tests determine how the service behaves over a prolonged period of time with average load.
Note, this testing framework used here was actually developed by the
Shibboleth
team for testing their software but it allows other test packs to be installed so it can also be used in testing this service.
Testing Framework Basics
Requirements
- Unix OS based machine(s)
- Subversion
client (necessary in order to download the framework)
- Java 1.5
Downloading and Installing the Framework
- Checkout the base testing framework. The resulting
grinder-framework
directory will be known a GF_HOME
throughout the rest of this document.
- Within GF_HOME/test checkout the Argus test packs
Installation is now complete.
Running the Framework
The load testing framework is composed of two parts, an agent and a console. The agent performs the actual work while the console, a GUI application, collects the results from agents and displays some metrics based on those results
To run the console execute the
GF_HOME/bin/console.sh
command. This will start the console which will listen on port 6372 for agent results. Be sure that this port is accessible to the agents you start.
To run the console execute the
GF_HOME/bin/agent.sh test_name
command. The name of the test is dependent on which test you will run and is given in the documentation for those tests. You can install agents on more than one machine and have them report back to a single client. This is useful if one agent machine is not capable of generating enough load.
Common Configuration Options
Each test package contains a
test.properties
file that contains configuration information for the given test. The later half of these options are Grinder specific properties. In most cases you'll need to change the following properties.
Property Name |
Value |
grinder.consoleHost |
The IP address of the machine on which the console is running |
grinder.processes |
Number of grinder processes to run. Should be equal to the number of CPU cores on the agent machine. |
grinder.threads |
Number of unique clients, per process, that will connect to the service |
grinder.runs |
Number of times each client will connect to the service, 0 (zero) means to keep going until the console indicates the test should be stopped. Using a value of 1 (one) is useful for testing that your configuration is correct. |
PEPCLI Test
This test uses the
C language PEP CLI to issue basic authorization requests. It a good test to run if you are trying get measurements for applications which do not maintain state between requests (e.g. a new client is created for every request).
Test Name: pepcli-authz
Configuration Options
Property Name |
Value |
Required |
pepcli.bin |
The path to the pepcli binary. |
Yes |
pepcli.pepds |
A space separated list of PEPd endpoints |
Yes |
pepcli.certchain |
Path, relative to GF_HOME , to the certchain used to identify the subject or the request |
Yes |
pepcli.resourceid |
The resource ID used within the request |
One is required. |
pepcli.resourceid.prefix |
Prefix added to the grinder-agent-name to create the resource ID used in the request |
pepcli.actionid |
The action ID used within the request |
One is required. |
pepcli.actionid.prefix |
Prefix added to the grinder-agent-run-number to create the resource ID used in the request |
pepcli.timeout |
The connection timeout in seconds |
No, defaults to 30 |
PEPCLI Test
This test uses the
C language PEP CLI to issue multiple basic authorization requests. It a good test to run if you are trying get measurements for applications which do not maintain state between requests (e.g. a new client is created for every request).
Test Name: pepcli-multiauthz
Configuration Options
Property Name |
Value |
Required |
pepcli.bin |
The path to the pepcli binary. |
Yes |
pepcli.pepds |
A space separated list of PEPd endpoints |
Yes |
pepcli.timeout |
The connection timeout in seconds |
No, defaults to 30 |
pepcli.tests |
A set of test inputs. See below for format. |
Yes |
Test Input Format
The test import format is as follows:
[ \
['ce1', 'submit', '/home/jsmith/.globus/usercert.pem', 'Permit'], \
['ce2', 'submit', '/home/jsmith/.globus/usercert.pem', 'Deny'], \
['ce3', 'submit', '/home/jsmith/.globus/usercert.pem', 'Not Applicable'],\
]
The property value is contained in matching '[' and ']' brackets. Then each test individual test scenario is contained with matching '[' and ']'. Each testing scenario has four comma-separated values: the resource ID, action ID, and certificate chain used in the request and then the expected outcome. Each test scenario is also comma-separated. The use of the '\' character can be used to indicate that the property value continues on to the next line.
PEP-J Test
This test uses the
Java language PEP CLI to issue authorization requests. It is a good test to run if you are trying to get measurements for application which maintain state between requests (e.g. a single client is used for multiple requests).
Test Name: pepj-authz
Configuration Option