Introduction
This is a CREAM ES test-suite developed using Robot Framework and python. It provides functional testing of the CREAM Execution Service. It is a work in progress and more test cases will be included in the future. The test suite supports Scientific Linux 6. If an SL5 compliant version is required, please feel free to contact me. Also, it can be easily ported to any Linux platform.
(anyone wanting to use the test suite in a non supported environment, please feel free to contact me)
Scientific Linux 6 Installation
The test suite has to be run on a host which has the capabillities to submit jobs to a CREAM endpoint. Also, the CREAM endpoint must have the ES interface enabled (you can find out how here:
https://wiki.italiangrid.it/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI2#1_2_2_EMI_Execution_Service
). Namely, an EMI-UI is ideal for the job.
In order to install the test-suite, execute as root:
wget http://yum.gridctb.uoa.gr/repository/robot_testing_sl6.repo -O /etc/yum.repos.d/robot_testing_sl6.repo
yum install cream_es_test
(note that the dependencies are automatically taken care of)
You can also install the RIDE editor. RIDE is a wysiwyg editor for Robot Framework's test case files. The available packages are for Scientific Linux only, but it can be easily installed in any Linux environment. In the case of SL, in order to install it, you first need to install wxPython with unicode support:
yum install wxPython
yum install robotframework-ride
If you want to use it by X11 forwarding, you must also have the xorg-x11-xauth package installed:
yum install xorg-x11-xauth
An alternative would be, to install it on your local machine and edit the remote test case files by mounting the remote host's file system through ssh for example.
If you only want to install robotframework,:
yum install robotframework
Scientific Linux 6 Dependencies
Package |
Dependency |
cream_es_test |
glite-ce-cream-cli |
|
uberftp |
|
pexpect |
|
robotframework |
robotframework |
python >= 2.6 & < 3 |
Paramiko: |
python-crypto |
RIDE |
python >= 2.6 & <3 |
|
wxPython >= 2.8.6 with Unicode support |
The packages needed by the test suite and their dependencies are provided in the aforementioned repository. Since the default Python for SL6 is version 2.6, there is no need for any external packages. All dependencies are either installed by default, or available in the standard repos (SL, EPEL etc)
Installed files
The CREAM ES testsuite installs the following files:
File |
Description |
/opt/cream_es_test/testsuite/cream_es_test_suite.html |
The html testing module for Robot Framework |
/opt/cream_es_test/lib/cream_es_testing.py |
The python module with the methods/keywords needed by the test |
/opt/cream_es_test/lib/vars_es.py |
Python module, containing the variables needing to be set for the test |
/opt/cream_es_test/docs/cream_es_testing_keywords.html |
Html documentation of the keywords provided by the test library |
/opt/cream_es_test/docs/cream_es_testing_libdoc.html |
Html Python module documentation for the test library |
/opt/cream_es_test/docs/Cream_Es_Test_Suite-doc.html |
Html test suite documentation |
/opt/cream_es_test/docs/COPYING |
Copyright document |
/opt/cream_es_test/docs/cream_es_test.7.gz and /usr/share/man/man7/cream_es_test.7.gz |
The test suite's man page |
/opt/cream_es_test/docs/CHANGELOG |
The test suite's changelog |
Also, once installed, the man page "cream_es_test" is available.
Deployment
Once the cream test package group is installed, you have to deploy the test to your environment. First of all,you can do a:
man cream_es_test
Standard Deployment Method
In order to do so, edit the file
/opt/cream_es_test/lib/vars_es.py
with the desired values. Examples are provided in the file itself, in the man page and in later parts of this document. Then you can execute the test suite.
Deploy with RIDE
If you have a graphical environment available or the ability to X-forward GUI applications from the testing host to a graphical environment, you can install the RIDE editor.
RIDE is a WYSIWYG editor for Robot Framework test cases. This might be the most intuitive way to create a new test, since it exposes the test-suite logic to the user.
- Create a copy of the file /opt/cream_es_test/lib/vars_es.py
- Edit it, assigning the desired values
- Create a copy of the file /opt/cream_es_test/testsuite/cream_es_test_suite.html
- Edit it with the RIDE editor, changing the path pointing to the variables file (by selecting the test suite's name in the list at the left part of the screen)
- Save the produced file
Test Execution
In the directory in which the test-suite HTML file resides,run:
pybot -W $COLUMNS /relative/or/absolute/path/to/test/html/file
The -W flag is used to tell pybot how many columns of text to print. The variable COLUMNS contains the number of columns in the current shell by default, so it is used for convenience.
Unless changed from the original deployment scenario, the test suite file is
/opt/cream_es_test/testsuite/cream_es_test_suite.html.
Advanced Test Execution
1. If you want to execute a certain test case in the test suite,run:
pybot -i test_case_tag testsuite.html
Multiple test cases can be included in the same way:
pybot -i tag1 -i tag2 testsuite.html
2. If you want to exclude a test case from execution,run:
pybot -e test_case_tag testsuite.html
Multiple test cases can be excluded in the same way:
pybot -e tag1 -e tag2 testsuite.html
3. You can set the log level with the -L flag. Possible values are (from lowest to highest): WARN, INFO, DEBUG, TRACE. Example:
pybot -L TRACE testsuite.html
4. You can set whether a test case is critical or not. This means that if a test case isn't critical (by default all are), if it fails, the test-suite will not fail. If a critical test case fails, the test-suite fails. This doesn't affect whether the rest of the test cases will be executed, it affects only the final test report. Examples:
pybot --critical tag testsuite.html
pybot --noncritical tag testsuite.html
Variables
The test's variables have explanations and examples in both the test-suite's HTML file and the variables.txt file. Here follows the same info:
Variable Name |
Description |
ce_endpoint |
The cream endpoint to be used (e.g.: ctb04.gridctb.uoa.gr:8443 ) |
queue |
The cream queue to be used (e.g.: cream-pbs-see ) |
vo |
The user’s submitting the jobs virtual organisation (e.g.: see ) |
proxy_pass |
The user’s submitting the jobs proxy password (e.g.: p4sSw0rD ) |
log_level |
The log level used during the test.Default is INFO.For extra output,set to DEBUG or TRACE. (possible values: NONE FAIL WARN INFO DEBUG TRACE) |
tmp_dir |
The path in which temporary files will reside. They will be automatically cleaned up unless you set the variable delete_files to "False" or explicitely don’t run the cleanup test case. The path will be created -with its parents-, it doesn’t have to exist. You can leave it empty and a temporary directory will be created for you. In order to know which temp random directory is used, it is printed in standard output and in the final test suite report. Warning: any parent directories created, are not removed! All in all, unless needed for specific reasons, you should leave this variable empty. |
delete_files |
Delete temporary files (jdl and script files created during the test) or not. Possible values: True False. Defaults to "True" |
Test Cases
Documentation for Cream Es Test Suite
Library Reference
cream_es_testing
Shortcuts
Keywords
Keyword |
Arguments |
Documentation |
Check Proxy |
time_left=None |
Description: |
Check whether the proxy exists and if it has any time left. |
Arguments: |
Without any arguments,it checks if the proxy exists and has any time left |
|
With one argument,it checks if the proxy exists and has greater than or equal to the requested time left. |
|
Create Proxy |
password, vo, cert=None, key=None, time=None |
Description: |
Create a user proxy. |
Arguments: |
password |
the user's proxy password |
|
vo |
for the voms extention. |
|
cert |
non standard certificate path |
|
key |
non standard key path |
|
time |
the validity period of the proxy. Form: HH:MM |
|
Destroy Proxy |
|
Description: |
Delete a user's proxy. |
|
Diff Seconds |
t1, t2, tolerance=0 |
Description: |
calculate difference of t1 from t2, in seconds |
Arguments: |
t1 |
a timestamp in seconds |
|
t2 |
a timestamp in seconds |
|
tolerance |
the maximum difference between the two timestamps |
|
Es Activity Cancel |
ep, aid |
Description: |
Execute the glite-es-activity-cancel command on the given endpoint for this aid |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
nothing or error |
|
Es Activity Create |
ep, adl, retOut=False |
Description: |
Submit the activity described in the file adl to the endpoint ep |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
adl |
path to adl file |
|
retOut |
if set to True, the output will also be returned |
Returns: |
the submitted aid and possibly the command's output |
|
Es Activity Current Status |
ep, aid |
Description: |
Execute the glite-es-activity-status command on the given endpoint |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
the activity's status and attributes |
|
Es Activity Final Status |
ep, aid |
Description: |
Execute the glite-es-activity-info command on the given endpoint and wait until |
|
the job has reached TERMINAL status |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
the activity's status (normaly TERMINAL) and attributes |
|
Es Activity Info |
ep, aid |
Description: |
Execute the glite-es-activity-info command on the given endpoint for this aid |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
the activity's info |
|
Es Activity List |
ep, fMax=None, fFrom=None, fTo=None, fState=None |
Description: |
Execute the glite-es-activity-list command on the given endpoint, |
|
using the specified filters. |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
fMax |
the maximum returned entries filter |
|
fFrom |
FROM YYYY-MM-DD HH:mm:ss date filter |
|
fTo |
TO YYYY-MM-DD HH:mm:ss date filter |
|
fState |
Sets the status/status-attrs filter: each status/status-attr is |
|
|
a string in this format <STATUS>[:<STATUS_ATTRIBUTE>]; multiple |
|
|
status/status-attr items can be separated by ',' and are OR'ed by |
|
|
the service when it tries to selects activities to return. |
|
|
Example: TERMINAL,PREPROCESSING:VALIDATING,TERMINAL:APP_FAILURE |
|
|
Check corresponding man page for relevant statuses and attributes. |
|
Es Activity Pause |
ep, aid |
Description: |
Execute the glite-es-activity-pause command on the given endpoint for this aid |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
nothing or error |
|
Es Activity Resume |
ep, aid |
Description: |
Execute the glite-es-activity-resume command on the given endpoint for this aid |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
nothing or error |
|
Es Activity Wipe |
ep, aid |
Description: |
Execute the glite-es-activity-wipe command on the given endpoint for this aid |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aid |
the activity id |
Returns: |
nothing or error |
|
Es Delegate Proxy |
ep |
Description: |
Execute the glite-es-delegate-proxy command on the given endpoint |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
Returns: |
the delegation id or raises error |
|
Es Delegation Info |
ep, did, time=False |
Description: |
Execute the glite-es-delegation-info command on the given endpoint |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
did |
the delegation id |
|
time |
if set to True, the time remaining is instead returned |
Returns: |
the command's output or the time remaining (in seconds) |
|
Es Delegation Renew |
ep, did |
Description: |
Renew a delegation with glite-es-delegation-renew |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
did |
the delegation id |
Returns: |
the command's output |
|
Es Multi Activity Create |
ep, adl |
Description: |
Submit a multiple activity described in the file adl to the endpoint ep |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
adl |
path to adl file |
|
Es Multi Activity Current Status |
ep, aids |
Description: |
Execute the glite-es-activity-status command on the given endpoint for multiple aids |
Arguments: |
ep |
the endpoint on which to run the command (format: host:port ) |
|
aids |
a list of aids |
Returns: |
two lists with statuses and attributes, in correct order |
|
Execute Command |
com |
Description: |
Runs the given command through the subprocess module |
Arguments: |
com |
a string with the command and its arguments |
Returns: |
(int,str) tuple, containing the returnValue and the output of the command |
|
Execute Uberftp Command |
uberftp_command, gridftp_server, gridftp_path |
Description: |
Execute an uberftp command on a gridftp url |
Arguments: |
uberftp_command |
one of cat,chgrp,chmod,dir,ls,mkdir, |
|
|
rename,rm,rmdir,size,stage |
|
gridftp_server |
the gridftp server hostname |
|
gridftp_path |
the path in the gridftp server |
Returns: |
The output of the command |
|
Extract Error From Activity Submit |
s |
Description: |
calculate difference of t1 from t2, in seconds |
Arguments: |
s |
activity submit output |
Returns: |
error message (may be empty) |
|
Get Activity Sb |
ep, aid |
Description: |
Find the gridftp url of the ISB of the given job |
Arguments: |
aid |
activity id returned by submit operation |
|
ep |
enpoint |
Returns: |
(gridftp server, gridftp path) |
|
|
Multi Adl |
queue, secs, output_dir |
Description: |
Multi adl file.Executes /bin/sleep for the defined number of seconds. |
Arguments: |
queue |
the queue to submit the activity |
|
secs |
seconds to sleep |
|
output_dir |
the directory to put the file in |
Returns: |
Temporary file name. |
|
Sleep Adl |
queue, secs, output_dir |
Description: |
Simple adl file.Executes /bin/sleep for the defined number of seconds. |
Arguments: |
queue |
the queue to submit the activity |
|
secs |
seconds to sleep |
|
output_dir |
the directory to put the file in |
Returns: |
Temporary file name. |
|
Sleep Failif Ne Adl |
queue, secs, output_dir, failif_exitcode_ne=None |
Description: |
Simple adl file, using the FailIfExitCodeNotEqualTo tag. |
|
Executes /bin/sleep for the defined number of seconds. |
Arguments: |
queue |
the queue to submit the activity |
|
secs |
seconds to sleep |
|
output_dir |
the directory to put the file in |
|
failif_exitcode_ne |
the FailIfExitCodeNotEqualTo argument (ignore if None) |
Returns: |
Temporary file name. |
|
Unsupported Capabillity Adl |
queue, output_dir, optional=None |
Description: |
Simple adl file, specifying an unsupported attribute. |
Arguments: |
queue |
the queue to submit the activity |
|
output_dir |
the directory to put the file in |
|
optional |
the optional value for the capabillity being tested |
|
|
(either true, false or none(not present) |
Returns: |
Temporary file name. |
|
Changelog
First version, no changelog available.
Check Also
Robot Framework Quick Start Guide
https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkQuickstartGuide
Robot Framework Advanced Guide
https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkAdvancedGuide
Robot Framework
http://code.google.com/p/robotframework/
RIDE Editor for Robot Framework test suites
http://code.google.com/p/robotframework-ride/
pybot --help
man cream_es_test