ETICS multinode testing exercises

This page collects some preliminary tests to use the ETICS multinode testing to certify Data Management services.

Sequence diagram for DPM test automation

The general idea is shown in the following sequence diagram: MultinodeDPM.png

Yaimgen and test scripts review

Mutlinode testing in ETICS will be used here as a driver and comunication infrastructure built on top of existing script for test automation ([https://svnweb.cern.ch/trac/yaimgen/][Yaimgen]].

Automatic deployment of services: - glite-SE_dpm_mysql:

./vnode-ygen.sh -i sl5-64-dh -n glite-SE_dpm_mysql
[...]

- glite-UI

./vnode-ygen.sh -i sl5-64-dh -n glite-UI

The vnode-ygen.sh script uses the Vnode CLI to create a virtual machine, and then Yaimgen to deploy the service.

The UI deployment also creates and prepare a testuser account generating a voms proxy, so that tests can be immediately run.

Launching DPM client tests using tests-wrapper.py:

 ./tests-wrapper.py -d ../tests/functional lcg
This command will launch the lcgutil functional test using the CTB resources specified in lcgutil-config.

ISSUE #1: How to override the default hostnames in order to use the one just deployed?

SOLUTION #1: The lcgutil-config has been modified in order not to override existing environment variables.

Therefore, before calling tests-wrapper.py one has to set SE_HOST to point the tests to the DPM server just deployed:

export SE_HOST="vtb-generic-91.cern.ch"

ISSUE #2_ The just deployed DPM will not be published in the BDII, causing lcg-* command to fail:

[BDII][][] lxbra2305.cern.ch: No entries for host: vtb-generic-91.cern.ch

SOLUTION #1_A: Do not use the BDII, run all the lcg-* command with the option '--nobdii' and specifying the endpoint type, e.g:

[testuser@vtb-generic-61 scripts]$ lcg-cr -v --nobdii --defaultsetype srmv2 -d vtb-generic-91.cern.ch file:/etc/hosts
Using grid catalog type: lfc
Using grid catalog : lxbra2506v3.cern.ch
Checksum type: None
[GFAL][gfal_init][EINVAL] Invalid request: When BDII checks are disabled, you must provide SURLs and endpoint type
lcg_cr: Invalid argument
Moreover we need to provide the SURL instead of the hostname, like this:
[testuser@vtb-generic-61 scripts]$ lcg-cr --nobdii --defaultsetype srmv2 -d srm://vtb-generic-91.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/gpfile4 file:/etc/hosts
guid:b15c58f8-83a8-432f-bc21-0c7315c9e94b

In order to make this solution work we need to modify modify the script lcg-common in order to use the --nobdii option, which is not working because it relies on a BDII query

[testuser@vtb-generic-61 lcg]$ ./lcg-cr --nobdii --usefullsurl vtb-generic-91.cern.ch

    === lcg-cr, -del test ===    

 ===> --nobdii option will be passed to commands sensitive to it
 ===> Full SURLs will be used whenever possible
 ===> Full SURLs will be used whenever possible
 ===> SE host      : vtb-generic-91.cern.ch
 ===> second storage element (for lcg-rep): 
 ===> [for information]: LCG_GFAL_INFOSYS=
 ===> [for information]: LCG_GFAL_VO=
 ===> [for information]: LFC_HOST=lxbra2506v3.cern.ch
 ===> Trying to determine endpoint for vtb-generic-91.cern.ch
ldap_bind: Can't contact LDAP server (-1)
ldap_bind: Can't contact LDAP server (-1)
ldap_bind: Can't contact LDAP server (-1)
 ===> SE endpoint : 
 ===> SRM endpoint:   
 ===> Could not determine srm endpoint for vtb-generic-91.cern.ch

This solution also has the disadvantage of not testing the BDII interaction.

SOLUTION #2_B: Add a testing BDII that include CTB resources, and where we can publish unstable resources.

Possible interactions among unstable services? Should not be a problem.

The testing BDII is already available, but publishing information automatically (on demand) looks tricky.

SOLUTION #2_C: Create an additional node with an ad-hoc BDII

This way we have control over the machine and would be easier to publish information. However for every test there is going to be an additional node needed.

I will investigate this solution for now.

The new sequence diagram including the BDII node and additional communication messages is: MultinodeDPM2.png

In order to update the BDII with the newly created DPM host we need to add to /opt/glite/etc/gip/site-urls.conf the string:

DPM  ldap://<dpm hostname>:2170/mds-vo-name=resource,o=grid

BDII node commands

TBD

UI node commands

TBD

DPM node commands

TBD

ETICS integration

TBD

-- GianniPucciani - 08-Dec-2010

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatdia MultinodeDPM.dia r1 manage 2.2 K 2010-12-08 - 14:32 GianniPucciani High level sequene diagram DIA
PNGpng MultinodeDPM.png r1 manage 22.4 K 2010-12-08 - 14:31 GianniPucciani High level sequence diagram
Unknown file formatdia MultinodeDPM2.dia r1 manage 3.1 K 2010-12-09 - 16:43 GianniPucciani Sequnce with BDII .dia
PNGpng MultinodeDPM2.png r1 manage 46.0 K 2010-12-09 - 16:42 GianniPucciani Sequence with BDII
Edit | Attach | Watch | Print version | History: r8 | r6 < r5 < r4 < r3 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r4 - 2010-12-09 - GianniPucciani
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback