Introduction

This is a CREAM ES test-suite developed using Robot Framework and python. It provides functional testing of the CREAM Execution Service. It is a work in progress and more test cases will be included in the future. The test suite supports Scientific Linux 6. If an SL5 compliant version is required, please feel free to contact me. Also, it can be easily ported to any Linux platform.

(anyone wanting to use the test suite in a non supported environment, please feel free to contact me)


Scientific Linux 6 Installation

The test suite has to be run on a host which has the capabillities to submit jobs to a CREAM endpoint. Also, the CREAM endpoint must have the ES interface enabled (you can find out how here: https://wiki.italiangrid.it/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI2#1_2_2_EMI_Execution_Service). Namely, an EMI-UI is ideal for the job.

In order to install the test-suite, execute as root:

wget http://yum.gridctb.uoa.gr/repository/robot_testing_sl6.repo -O /etc/yum.repos.d/robot_testing_sl6.repo
yum install cream_es_test
(note that the dependencies are automatically taken care of)

You can also install the RIDE editor. RIDE is a wysiwyg editor for Robot Framework's test case files. The available packages are for Scientific Linux only, but it can be easily installed in any Linux environment. In the case of SL, in order to install it, you first need to install wxPython with unicode support:

yum install wxPython
yum install robotframework-ride

If you want to use it by X11 forwarding, you must also have the xorg-x11-xauth package installed:

yum install xorg-x11-xauth

An alternative would be, to install it on your local machine and edit the remote test case files by mounting the remote host's file system through ssh for example.
If you only want to install robotframework,:

yum install robotframework

Scientific Linux 6 Dependencies

Package Dependency
cream_es_test glite-ce-cream-cli
  uberftp
  pexpect
  robotframework
robotframework python >= 2.6 & < 3
Paramiko: python-crypto
RIDE python >= 2.6 & <3
  wxPython >= 2.8.6 with Unicode support

The packages needed by the test suite and their dependencies are provided in the aforementioned repository. Since the default Python for SL6 is version 2.6, there is no need for any external packages. All dependencies are either installed by default, or available in the standard repos (SL, EPEL etc)

Installed files

The CREAM ES testsuite installs the following files:
File Description
/opt/cream_es_test/testsuite/cream_es_test_suite.html The html testing module for Robot Framework
/opt/cream_es_test/lib/cream_es_testing.py The python module with the methods/keywords needed by the test
/opt/cream_es_test/lib/vars_es.py Python module, containing the variables needing to be set for the test
/opt/cream_es_test/docs/cream_es_testing_keywords.html Html documentation of the keywords provided by the test library
/opt/cream_es_test/docs/cream_es_testing_libdoc.html Html Python module documentation for the test library
/opt/cream_es_test/docs/Cream_Es_Test_Suite-doc.html Html test suite documentation
/opt/cream_es_test/docs/COPYING Copyright document
/opt/cream_es_test/docs/cream_es_test.7.gz and /usr/share/man/man7/cream_es_test.7.gz The test suite's man page
/opt/cream_es_test/docs/CHANGELOG The test suite's changelog
Also, once installed, the man page "cream_es_test" is available.


Deployment

Once the cream test package group is installed, you have to deploy the test to your environment. First of all,you can do a:
man cream_es_test

Standard Deployment Method

In order to do so, edit the file
 /opt/cream_es_test/lib/vars_es.py 
with the desired values. Examples are provided in the file itself, in the man page and in later parts of this document. Then you can execute the test suite.

Deploy with RIDE

If you have a graphical environment available or the ability to X-forward GUI applications from the testing host to a graphical environment, you can install the RIDE editor. RIDE is a WYSIWYG editor for Robot Framework test cases. This might be the most intuitive way to create a new test, since it exposes the test-suite logic to the user.

  1. Create a copy of the file /opt/cream_es_test/lib/vars_es.py
  2. Edit it, assigning the desired values
  3. Create a copy of the file /opt/cream_es_test/testsuite/cream_es_test_suite.html
  4. Edit it with the RIDE editor, changing the path pointing to the variables file (by selecting the test suite's name in the list at the left part of the screen)
  5. Save the produced file

Test Execution

In the directory in which the test-suite HTML file resides,run:
pybot -W $COLUMNS /relative/or/absolute/path/to/test/html/file
The -W flag is used to tell pybot how many columns of text to print. The variable COLUMNS contains the number of columns in the current shell by default, so it is used for convenience. Unless changed from the original deployment scenario, the test suite file is /opt/cream_es_test/testsuite/cream_es_test_suite.html.

Advanced Test Execution

1. If you want to execute a certain test case in the test suite,run:
pybot -i test_case_tag testsuite.html
Multiple test cases can be included in the same way:
pybot -i tag1 -i tag2 testsuite.html

2. If you want to exclude a test case from execution,run:

pybot -e test_case_tag testsuite.html
Multiple test cases can be excluded in the same way:
pybot -e tag1 -e tag2 testsuite.html

3. You can set the log level with the -L flag. Possible values are (from lowest to highest): WARN, INFO, DEBUG, TRACE. Example:

pybot -L TRACE testsuite.html

4. You can set whether a test case is critical or not. This means that if a test case isn't critical (by default all are), if it fails, the test-suite will not fail. If a critical test case fails, the test-suite fails. This doesn't affect whether the rest of the test cases will be executed, it affects only the final test report. Examples:

pybot --critical tag testsuite.html
pybot --noncritical tag testsuite.html

Variables

The test's variables have explanations and examples in both the test-suite's HTML file and the variables.txt file. Here follows the same info:
Variable Name Description
ce_endpoint The cream endpoint to be used (e.g.: ctb04.gridctb.uoa.gr:8443 )
queue The cream queue to be used (e.g.: cream-pbs-see )
vo The userís submitting the jobs virtual organisation (e.g.: see )
proxy_pass The userís submitting the jobs proxy password (e.g.: p4sSw0rD )
log_level The log level used during the test.Default is INFO.For extra output,set to DEBUG or TRACE. (possible values: NONE FAIL WARN INFO DEBUG TRACE)
tmp_dir The path in which temporary files will reside. They will be automatically cleaned up unless you set the variable delete_files to "False" or explicitely donít run the cleanup test case. The path will be created -with its parents-, it doesnít have to exist. You can leave it empty and a temporary directory will be created for you. In order to know which temp random directory is used, it is printed in standard output and in the final test suite report. Warning: any parent directories created, are not removed! All in all, unless needed for specific reasons, you should leave this variable empty.
delete_files Delete temporary files (jdl and script files created during the test) or not. Possible values: True False. Defaults to "True"

Test Cases

Documentation for Cream Es Test Suite

Documentation for Cream Es Test Suite

 
Expand All TEST SUITE: Cream Es Test Suite
TEST CASE: Set Log Level
TEST CASE: Check Executables
TEST CASE: Delegation
TEST CASE: Proxy Renewal
TEST CASE: Activity without Data Staging
TEST CASE: Activity with FailIfExitCodeNotEqualTo
TEST CASE: Multiple Activities
TEST CASE: Unsupported Capabillity
TEST CASE: List
TEST CASE: Activity Info
TEST CASE: Pause and Resume
TEST CASE: Cancel
TEST CASE: Wipe

Library Reference

cream_es_testing

Shortcuts

Keywords

Keyword Arguments Documentation
Check Proxy time_left=None
Description: Check whether the proxy exists and if it has any time left.

Arguments: Without any arguments,it checks if the proxy exists and has any time left
With one argument,it checks if the proxy exists and has greater than or equal to the requested time left.

Returns: nothing
Create Proxy password, vo, cert=None, key=None, time=None
Description: Create a user proxy.

Arguments: password the user's proxy password
vo for the voms extention.
cert non standard certificate path
key non standard key path
time the validity period of the proxy. Form: HH:MM

Returns: nothing.
Destroy Proxy
Description: Delete a user's proxy.

Arguments: none.

Returns: nothing.
Diff Seconds t1, t2, tolerance=0
Description: calculate difference of t1 from t2, in seconds

Arguments: t1 a timestamp in seconds
t2 a timestamp in seconds
tolerance the maximum difference between the two timestamps

Returns: diff in seconds
Es Activity Cancel ep, aid
Description: Execute the glite-es-activity-cancel command on the given endpoint for this aid

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: nothing or error
Es Activity Create ep, adl, retOut=False
Description: Submit the activity described in the file adl to the endpoint ep

Arguments: ep the endpoint on which to run the command  (format: host:port )
adl path to adl file
retOut if set to True, the output will also be returned

Returns: the submitted aid  and possibly the command's output
Es Activity Current Status ep, aid
Description: Execute the glite-es-activity-status command on the given endpoint

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: the activity's status and attributes
Es Activity Final Status ep, aid
Description: Execute the glite-es-activity-info command on the given endpoint and wait until
the job has reached TERMINAL status

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: the activity's status (normaly TERMINAL) and attributes
Es Activity Info ep, aid
Description: Execute the glite-es-activity-info command on the given endpoint for this aid

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: the activity's info
Es Activity List ep, fMax=None, fFrom=None, fTo=None, fState=None
Description: Execute the glite-es-activity-list command on the given endpoint,
using the specified filters.

Arguments: ep the endpoint on which to run the command  (format: host:port )
fMax the maximum returned entries filter
fFrom FROM YYYY-MM-DD HH:mm:ss date filter
fTo TO YYYY-MM-DD HH:mm:ss date filter
fState Sets the status/status-attrs filter: each status/status-attr is
a string in this format <STATUS>[:<STATUS_ATTRIBUTE>]; multiple
status/status-attr items can be separated by ',' and are OR'ed by
the service when it tries to selects activities to return.
Example: TERMINAL,PREPROCESSING:VALIDATING,TERMINAL:APP_FAILURE
Check corresponding man page for relevant statuses and attributes.

Returns: list of aid(s)
Es Activity Pause ep, aid
Description: Execute the glite-es-activity-pause command on the given endpoint for this aid

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: nothing or error
Es Activity Resume ep, aid
Description: Execute the glite-es-activity-resume command on the given endpoint for this aid

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: nothing or error
Es Activity Wipe ep, aid
Description: Execute the glite-es-activity-wipe command on the given endpoint for this aid

Arguments: ep the endpoint on which to run the command  (format: host:port )
aid the activity id

Returns: nothing or error
Es Delegate Proxy ep
Description: Execute the glite-es-delegate-proxy command on the given endpoint

Arguments: ep the endpoint on which to run the command  (format: host:port )

Returns: the delegation id or raises error
Es Delegation Info ep, did, time=False
Description: Execute the glite-es-delegation-info command on the given endpoint

Arguments: ep the endpoint on which to run the command  (format: host:port )
did the delegation id
time if set to True, the time remaining is instead returned

Returns: the command's output or the time remaining (in seconds)
Es Delegation Renew ep, did
Description: Renew a delegation with glite-es-delegation-renew

Arguments: ep the endpoint on which to run the command  (format: host:port )
did the delegation id

Returns: the command's output
Es Multi Activity Create ep, adl
Description: Submit a multiple activity described in the file adl to the endpoint ep

Arguments: ep the endpoint on which to run the command  (format: host:port )
adl path to adl file

Returns: list of aids
Es Multi Activity Current Status ep, aids
Description: Execute the glite-es-activity-status command on the given endpoint for multiple aids

Arguments: ep the endpoint on which to run the command  (format: host:port )
aids a list of aids

Returns: two lists with statuses and attributes, in correct order
Execute Command com
Description: Runs the given command through the subprocess module

Arguments: com a string with the command and its arguments

Returns: (int,str) tuple, containing the returnValue and the output of the command
Execute Uberftp Command uberftp_command, gridftp_server, gridftp_path
Description: Execute an uberftp command on a gridftp url

Arguments: uberftp_command one of cat,chgrp,chmod,dir,ls,mkdir,
rename,rm,rmdir,size,stage
gridftp_server the gridftp server hostname
gridftp_path the path in the gridftp server

Returns: The output of the command
Extract Error From Activity Submit s
Description: calculate difference of t1 from t2, in seconds

Arguments: s activity submit output

Returns: error message (may be empty)
Get Activity Sb ep, aid
Description: Find the gridftp url of the ISB of the given job

Arguments: aid activity id returned by submit operation

ep enpoint
Returns: (gridftp server, gridftp path)
Multi Adl queue, secs, output_dir
Description: Multi adl file.Executes /bin/sleep for the defined number of seconds.

Arguments: queue the queue to submit the activity
secs seconds to sleep
output_dir the directory to put the file in

Returns: Temporary file name.
Sleep Adl queue, secs, output_dir
Description: Simple adl file.Executes /bin/sleep for the defined number of seconds.

Arguments: queue the queue to submit the activity
secs seconds to sleep
output_dir the directory to put the file in

Returns: Temporary file name.
Sleep Failif Ne Adl queue, secs, output_dir, failif_exitcode_ne=None
Description: Simple adl file, using the FailIfExitCodeNotEqualTo tag.
Executes /bin/sleep for the defined number of seconds.

Arguments: queue the queue to submit the activity
secs seconds to sleep
output_dir the directory to put the file in
failif_exitcode_ne the FailIfExitCodeNotEqualTo argument (ignore if None)

Returns: Temporary file name.
Unsupported Capabillity Adl queue, output_dir, optional=None
Description: Simple adl file, specifying an unsupported attribute.

Arguments: queue the queue to submit the activity
output_dir the directory to put the file in
optional the optional value for the capabillity being tested
(either true, false or none(not present)

Returns: Temporary file name.



Changelog

First version, no changelog available.

Check Also

Robot Framework Quick Start Guide https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkQuickstartGuide

Robot Framework Advanced Guide https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkAdvancedGuide

Robot Framework http://code.google.com/p/robotframework/

RIDE Editor for Robot Framework test suites http://code.google.com/p/robotframework-ride/

pybot --help

man cream_es_test

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2012-11-20 - DimosthenesFioretosExCern
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback