FTS Tests Guide

Introduction

• Every module should have unit test.
• When there is a new functionality we write some new unit test and then you add the functionality.
• Once a bug is fixed a test must be created for this bug (Regression tests).

How to get tests and configure them

• cp test-config.sh $WORKSPACE
• vi test-config.sh. Configured:
  • Workspace
  • Path to the repository
  • lib_dir set it to lib64 for 64 bit machines
  • Authentication
  • Oracle configuration
• source test-config.sh
• Run the unit test througth some bash scripts, located under the folder org.glite.data.transfer-url-copy/test/unit-test.sh with the option -t you get the targets.

Description of the scripts in test-utils

• test-config.sh
  • Initialize the configuration for the tests
  • Define the WORKSPACE
  • Certificate configuration
  • Configure the DB parameters

• tests-proxy-utils
  • Configure globus location,
  • Create the user and host certificates
  • Start the proxy server
  • Get the proxy certificate
  • Create fake certificates

• test-common
  • Add path
  • Get path
  • Get external path
  • Get glite path
  • Get VDT path
  • Add libraries path (gSOAP, globus, log4cpp,glite, python,oracle)

Types of tests

• Types of tests:
  • Unit tests
  • Transfer Tests
  • Stress Tests
  • Integration Tests

Unit test

 
source org.glite.data/bin/e-env
source ./test-config.sh
# Generate fake certificates, needed for the tests - but it will work with your real certificates as well
cd org.glite.data.test-utils
./glite-test-certs 
# or: ./glite-test-certs  [--certdir=<directory>] [--env] [--verbose]
cd ../org.glite.data.srm-util-cpp/test
./unit-test.sh

The test logs are under $WS/stage/share/test/\<modulename\>/var/log.

Executing the whole unit test suite can be long, especially during development when we are interested in a particular test case only. Therefore, we can select the desired test classes in the manner as follows. We use the transfer-url-copy test in the example. We assume that the test binaries have already been built and stage-d.

Display the available test classes:

~/stage/share/test/glite-data-transfer-url-copy/bin/glite-data-transfer-urlcopy-test -h

The list of names corresponds to the CppUnit tests class names (the unit tests are bases on CppUnit). To select a test class, you need to set the TEST_CLASSES environment variable. Let's assume that we are interested in two test cases, then:

TEST_CLASSES="UrlCopyBug_47879 UrlCopyBug_37018"

UrlCopyBug_47879 and UrlCopyBug_37018 are the test class names, coming from the previous list.

Testing with the mock SRM servers

Setting up the environment

There are mock SRM servers, to test if the client-side logic works well. The mock SRM servers accept SRM requests and generate specific SRM responses, controlled by the special data in the request.

On both the client and server side, you need to set up your environment in the following way:

cd $WORKSPACE
source org.glite.data/bin/e-env
source ./test-config.sh
cd org.glite.data.test-utils
./glite-test-certs
cd ../org.glite.data.srm-util-cpp/test
source stub_common.sh

Starting/stopping the servers

To start/stop the mock servers (after setting up your environment):

cd $WORKSPACE/org.glite.data.srm-util-cpp/test
./stub.sh start|stop

The command starts two servers for both SRM 1.1 and SRM 2.2. The endpoints are:

• SRM 1.1: httpg://localhost:8043/srm/managerv1
• SRM 2.2: httpg://localhost:8044/srm/managerv2

Testing on the client side

Set up your environment like earlier, then:

cd $WORKSPACE/org.glite.data.srm-util-cpp/test
init_unit_test

After that, you may issue your test commands/clients. For instance, ping the servers:

cd $WORKSPACE/org.glite.data.srm-util-cpp/build/src/tools
./glite-srm-util-ping -v -s httpg://localhost:8043/srm/managerv1
./glite-srm-util-ping -v -s httpg://localhost:8044/srm/managerv2

Mock server naming rules

There are hundreeds of rules, used by the FTS unit tests. If you would like to use the mock servers for your own tests, then please contact the FTS developers, and we discuss it together.

Transfer tests

The transfer tests uses glite-srm-copy and glite-url-copy. These binaries are executed via a python script under org.glite.data.transfer-url-copy/test/scripts/python.

• What do they do?
  • Reads list of Endpoints
  • creates a temporary file
  • transfers the file to each endpoint in the list
  • When the transfer is successful creates an html report and copies it to afs so it's accessible through the test site

• Why are they necessary?
  • The unit tests are not covering all the cases as the stub servers are not a real gridftp servers.
  • They public html reports: reports are publish on the web http://test-fts-srm2.web.cern.ch/test-fts-srm2/ . (Check if you have permissions in (/afs/cern.ch/project/gd/www/ftssrm2/) before running the tests or the reports won't be published.

• Requirements
  • Python 2.5
  • Python-Yaml
  • Valid real proxy certificate
  • permissions on the endpoints

Usefull commands

The binary running the transfers are :

• glite-srm-copy
• glite-url-copy

There are some usefull commands when testing functionality on the pilot

• Show the content of the memory mapfile):
  • glite-url-copy-status. Takes a transfer id and goesinto /var/tmp/glite.url.copy/username/ and searchs for the channelname completed folder, the failed one and the lost one. So it's useful if you run transfers as yourself in your machine.
  • glite-print-status status. It takes the path from the memory map file. It does the same thing as the previous command but if you are root user then it will look for the status files in the path not in the username folder. This command is the one to use if you want to look to other user transfers.

• Commands to send signals to interrupt transfers.
  • glite-uc-stop sends a signal to interrupt the transfer.If the transfer process receives a signal set a flag and check it periodically. When it received a signal it does a graceful termination.
  • glite-uc-continue which tells the transfer to resume execution if they have been paused. There are 2 options that allow you to block in the preparation phase and the transfer process. ONLY 4 TESTING

Stress Test

See Load generator here GliteDataTransferLoadGenerator.

Deployment Test

See GliteDataTransferDeploymentTests.

-- Main.RosaGarciaRioja - 03 Mar 2009