EMI Integration Testbed: EMI gLite Job Management /Authentication/authorization/Data Management Tests

Testbed Used

  1. EMI CREAMCE: cert-07.cnaf.infn.it + Torque + MPI + ARGUS Enabled
  2. EMI ARGUS: emitestbed10.cnaf.infn.it
  3. EMI VOMS: emitestbed07.cnaf.infn.it
  4. EMI WNs: emitestbed[15-17].cnaf.infn.it + Torque + MPI + ARGUS Enabled + glexec
  5. EMI WMS: cert-08.cnaf.infn.it
  6. EMI LB: emitb1.ics.muni.cz
  7. EMI BDII: certtbrc-bdii-site.cern.ch
  8. EMI UI: emitestbed08.cnaf.infn.it

Performed Tests

Test 1: UI basics

VOMS OPERATION: PROXY CREATION and INFO

[dongiovanni@emitestbed08 ~]$ voms-proxy-init -voms testers.eu-emi.eu 
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni
Creating temporary proxy ............................................... Done
Contacting  emitestbed07.cnaf.infn.it:15002 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed07.cnaf.infn.it] "testers.eu-emi.eu" Done
Creating proxy .......................................................................................................................................................................................... Done

[dongiovanni@emitestbed08 ~]$ voms-proxy-info  --all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni
type      : proxy
strength  : 1024 bits
path      : /tmp/x509up_u500
timeleft  : 11:58:22
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO testers.eu-emi.eu extension information ===
VO        : testers.eu-emi.eu
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni
issuer    : /C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed07.cnaf.infn.it
attribute : /testers.eu-emi.eu/Role=NULL/Capability=NULL
timeleft  : 11:58:10
uri       : emitestbed07.cnaf.infn.it:15002

BDII QUERIES: check env, infosites, ldapsearch to BDII, ldapsearch to CE

[dongiovanni@emitestbed08 ~]$ env |grep LCG
LCG_LOCATION=/usr
LCG_GFAL_INFOSYS=certtbrc-bdii.cern.ch:2170                          <---- BDII USED IN INFOSITES



[dongiovanni@emitestbed08 ~]$ lcg-infosites --vo testers.eu-emi.eu all
#   CPU      Free   Total Jobs   Running   Waiting   ComputingElement
----------------------------------------------------------------
      4         4           16         0        16   cert-07.cnaf.infn.it:8443/cream-pbs-demo
     16        16            0         0         0   cert-09.cnaf.infn.it:8443/cream-lsf-demo
 Avail Space(kB)  Used Space(kB)  Type    SE
------------------------------------------
         5353730            4493  SRM     cvitbrc11.cern.ch


[dongiovanni@emitestbed08 ~]$  ldapsearch -x -H ldap://certtbrc-bdii-top.cern.ch:2170 -b mds-vo-name=local,o=grid
...........
.......
GlueCESEBindWeight: 0
GlueCESEBindCEUniqueID: cert-09.cnaf.infn.it:8443/cream-lsf-demo
GlueSchemaVersionMajor: 1

# search result
search: 2
result: 0 Success

# numResponses: 73
# numEntries: 72




[dongiovanni@emitestbed08 ~]$  ldapsearch -x -H ldap://cert-07.cnaf.infn.it:2170 -b mds-vo-name=resource,o=grid
....


Test 2: CREAM CE BASICS

LDAPQUERY TO CHECK CE PUBLISHES AND GET THE QUEUE TO USE


[dongiovanni@emitestbed08 ~]$ ldapsearch -x -H ldap://cert-07.cnaf.infn.it:2170 -b mds-vo-name=resource,o=grid |grep CEUniqueID
GlueForeignKey: GlueCEUniqueID: cert-07.cnaf.infn.it:8443/cream-pbs-demo
....

DIRECT SUBMISSION TO CREAM CE


[dongiovanni@emitestbed08 ~]$ ldapsearch -x -H ldap://cert-07.cnaf.infn.it:2170 -b mds-vo-name=resource,o=grid |grep CEUniqueID
(reverse-i-search)`ce-su': glite-ce-job-submit -d -r cert-13.cnaf.infn.it:8443/demo -a test.jdl
(reverse-i-search)`ce-': glite-ce-job-submit -d -r cert-13.cnaf.infn.it:8443/demo -a test.jdl
[dongiovanni@emitestbed08 ~]$ glite-ce-job-submit -d -r cert-07.cnaf.infn.it:8443/cream-pbs-demo -a test.jdl
2011-05-20 16:54:58,169 INFO - *************************************
2011-05-20 16:54:58,170 INFO - CREAM User Interface version 1.2.0 - Starting at Fri May 20 16:54:58 2011

2011-05-20 16:54:58,170 DEBUG - Using certificate proxy file [/tmp/x509up_u500]
2011-05-20 16:54:58,196 INFO - VO from certificate=[testers.eu-emi.eu]
2011-05-20 16:54:58,198 WARN - No configuration file suitable for loading. Using built-in configuration
2011-05-20 16:54:58,199 INFO - Logfile is [/tmp/glite_cream_cli_logs/glite-ce-job-submit_CREAM_dongiovanni_20110520-165458.log]
2011-05-20 16:54:58,200 DEBUG - Processing file [/home/dongiovanni/Test.sh]...
2011-05-20 16:54:58,201 DEBUG - Inserting mangled InputSandbox in JDL: [{"/home/dongiovanni/Test.sh"}]...
2011-05-20 16:54:58,208 INFO - Registering to [http://cert-07.cnaf.infn.it:8443/ce-cream/services/CREAM2] JDL=[ StdOutput = "test.out"; BatchSystem = "pbs"; QueueName = "demo"; ShallowRetryCount = 3; RetryCount = 3; Executable = "Test.sh"; VirtualOrganisation = "testers.eu-emi.eu"; InputSandbox = { "/home/dongiovanni/Test.sh" }; StdError = "test.err"; requirements = other.GlueCEInfoHostName == "cert-07.cnaf.infn.it" ]
2011-05-20 16:54:58,208 INFO - certUtil::generateUniqueID() - Generated DelegationID: [5dc7724ba594ddf5007d1a51f035f52111f132dd]
2011-05-20 16:55:00,473 INFO - JobID=[https://cert-07.cnaf.infn.it:8443/CREAM237281741]
2011-05-20 16:55:00,473 INFO - UploadURL=[gsiftp://cert-07.cnaf.infn.it/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_CNAF_CN_Danilo_Nicola_Dongiovanni_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst29/23/CREAM237281741/ISB]
2011-05-20 16:55:00,475 INFO - Sending file [gsiftp://cert-07.cnaf.infn.it/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_CNAF_CN_Danilo_Nicola_Dongiovanni_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst29/23/CREAM237281741/ISB/Test.sh]
2011-05-20 16:55:00,813 INFO - Now invoking JobStart for JobID [https://cert-07.cnaf.infn.it:8443/CREAM237281741]
https://cert-07.cnaf.infn.it:8443/CREAM237281741


[dongiovanni@emitestbed08 ~]$ glite-ce-job-status https://cert-07.cnaf.infn.it:8443/CREAM237281741

******  JobID=[https://cert-07.cnaf.infn.it:8443/CREAM237281741]
   Status        = [REALLY-RUNNING]


...after a while.....

[dongiovanni@emitestbed08 ~]$ glite-ce-job-status https://cert-07.cnaf.infn.it:8443/CREAM237281741

******  JobID=[https://cert-07.cnaf.infn.it:8443/CREAM237281741]
   Status        = [DONE-OK]
   ExitCode      = [0]

ACCESS TO gridFTP Server ON CREAM CE (quick test for authentication/authorization on cream)

[dongiovanni@emitestbed08 ~]$ uberftp cert-07 
220 cert-07.cnaf.infn.it GridFTP Server 3.28 (gcc64pthr, 1297437357-80) [Globus Toolkit 5.0.3] ready.
230 User tst29 logged in.
UberFTP> 

Test 3: EMI WMS

Prepare for testing

Get a voms proxy (see UI basics section) + prepare following files

[dongiovanni@emitestbed08 ~]$ cat test.jdl 
Executable    = "Test.sh";
StdOutput     = "test.out";
StdError      = "test.err";
RetryCount = 3;
ShallowRetryCount = 3;
requirements=other.GlueCEInfoHostName=="cert-07.cnaf.infn.it";
InputSandbox  = {"/home/dongiovanni/Test.sh"};
OutputSandbox = {"test.out", "test.err"};


[dongiovanni@emitestbed08 ~]$ cat Test.sh 
#!/bin/sh
clear
echo "Hi All! I'm a CMT test script, enjoy me!"
echo "I'm running on: " 
/bin/hostname
echo "that has: "
/bin/uname -a
echo "I'm running as: "
/usr/bin/whoami
echo "*****************************************"
echo "This is the enviroment that i have found:"
echo "*****************************************"
#/usr/bin/printenv
echo "*****************************************"
sleep 183
echo "That's all! Bye!"

[dongiovanni@emitestbed08 ~]$ cat WMSCONF/conf_cert-08.conf
WmsClient = [
ErrorStorage="/var/tmp";
OutputStorage=  "/tmp";
ListenerStorage="/tmp";
virtualorganisation="testers.eu-emi.eu";
RetryCount = 0;
WMProxyEndPoints = {"https://cert-08.cnaf.infn.it:7443/glite_wms_wmproxy_server"};
rank = -other.GlueCEStateEstimatedResponseTime;
];


[dongiovanni@emitestbed08 ~]$ cat test_collection_3_SR_9_0.jdl |grep -v "#"
[
Type = "collection";
InputSandbox = {"/home/dongiovanni/Test.sh"};
DefaultNodeShallowRetryCount = 0;
DefaultNodeRetryCount = 0;

nodes = {
[
    JobType = "Normal";
    Executable = "Test.sh";
    StdOutput = "test.out";
    StdError = "test.err";
    OutputSandbox = {"test.out", "test.err"};
    ],
[
    JobType = "Normal";
    Executable = "Test.sh";
    StdOutput = "test.out";
    StdError = "test.err";
    OutputSandbox = {"test.out", "test.err"};
    ],
[
    JobType = "Normal";
    Executable = "Test.sh";
    StdOutput = "test.out";
    StdError = "test.err";
    OutputSandbox = {"test.out", "test.err"};
    ]
};
]


LIST Match

[dongiovanni@emitestbed08 ~]$ glite-wms-job-list-match -c WMSCONF/conf_cert-08.conf -a test.jdl

Connecting to the service https://cert-08.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cert-07.cnaf.infn.it:8443/cream-pbs-demo

==========================================================================

Note: we had specified the ce resource in the test.jdl file

Single Job Submission / Status

1- SUBMIT

[dongiovanni@emitestbed08 ~]$ glite-wms-job-submit -c WMSCONF/conf_cert-08.conf -a test.jdl 

Connecting to the service https://cert-08.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emitb1.ics.muni.cz:9000/MPEzPmE8AJi_1a_A9pDA8A

==========================================================================

2- STATUS

[dongiovanni@emitestbed08 ~]$ glite-wms-job-status https://emitb1.ics.muni.cz:9000/TDHWmtbk1HIkV7XeRTBCPw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emitb1.ics.muni.cz:9000/TDHWmtbk1HIkV7XeRTBCPw
Current Status:     Running 
Status Reason:      unavailable
Destination:        cert-07.cnaf.infn.it:8443/cream-pbs-demo
Submitted:          Fri May 20 17:16:57 2011 CEST
==========================================================================

3- A JOB DONE SUCCESS

[dongiovanni@emitestbed08 ~]$ glite-wms-job-status https://emitb1.ics.muni.cz:9000/z3ZLoHAoO-pCIL0KB4nzYQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emitb1.ics.muni.cz:9000/z3ZLoHAoO-pCIL0KB4nzYQ
Current Status:     Done (Success)
Logged Reason(s):
    - job completed
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cert-07.cnaf.infn.it:8443/cream-pbs-demo
Submitted:          Thu May 19 20:43:14 2011 CEST
==========================================================================

4- DETAILS ON JOB EVENTS...

[dongiovanni@emitestbed08 ~]$ glite-wms-job-logging-info -v 3 https://emitb1.ics.muni.cz:9000/z3ZLoHAoO-pCIL0KB4nzYQ
---------- 
--------
Event: ReallyRunning
- Arrived                    =    Thu May 19 20:44:58 2011 CEST
- Host                       =    cert-08.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Seqcode                    =    UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000007:LRMS=000000:APP=000000:LBS=000000
- Source                     =    LogMonitor
- Timestamp                  =    Thu May 19 20:44:58 2011 CEST
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni/CN=proxy/CN=proxy
- Wn seq                     =    UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000
   ---
Event: Done
- Arrived                    =    Thu May 19 20:47:04 2011 CEST
- Exit code                  =    0
- Host                       =    cert-08.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Reason                     =    Job Terminated Successfully
- Seqcode                    =    UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000009:LRMS=000000:APP=000000:LBS=000000
- Source                     =    LogMonitor
- Status code                =    OK
- Timestamp                  =    Thu May 19 20:47:04 2011 CEST
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni/CN=proxy/CN=proxy
==========================================================================

5- JOB OUTPUT

[dongiovanni@emitestbed08 ~]$  glite-wms-job-output https://emitb1.ics.muni.cz:9000/TDHWmtbk1HIkV7XeRTBCPw

Connecting to the service https://cert-08.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - JobPurging not allowed
 (The Operation is not allowed: Unable to complete job purge)            


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emitb1.ics.muni.cz:9000/TDHWmtbk1HIkV7XeRTBCPw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/dongiovanni_TDHWmtbk1HIkV7XeRTBCPw

================================================================================

Collection Job Submission / Status

1- SUBMIT

[dongiovanni@emitestbed08 ~]$ glite-wms-job-submit -c WMSCONF/conf_cert-08.conf -a test_collection_3_SR_9_0.jdl

Connecting to the service https://cert-08.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emitb1.ics.muni.cz:9000/0IK4UBlUKb4xs_N_RnFI7w

==========================================================================

2- STATUS (logging info not available for collections)

[dongiovanni@emitestbed08 ~]$  glite-wms-job-status https://emitb1.ics.muni.cz:9000/0IK4UBlUKb4xs_N_RnFI7w


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emitb1.ics.muni.cz:9000/0IK4UBlUKb4xs_N_RnFI7w
Current Status:     Running 
Submitted:          Fri May 20 17:31:15 2011 CEST
==========================================================================

- Nodes information for: 
    Status info for the Job : https://emitb1.ics.muni.cz:9000/HhlcHRmRbQ3NJMXtDaOpNA
    Current Status:     Running 
    Status Reason:      unavailable
    Destination:        cert-07.cnaf.infn.it:8443/cream-pbs-demo
    Submitted:          Fri May 20 17:31:15 2011 CEST
==========================================================================
    
    Status info for the Job : https://emitb1.ics.muni.cz:9000/IHtDsnLU1-HYDVLRxcPYuw
    Current Status:     Running 
    Status Reason:      unavailable
    Destination:        cert-07.cnaf.infn.it:8443/cream-pbs-demo
    Submitted:          Fri May 20 17:31:15 2011 CEST
==========================================================================
    
    Status info for the Job : https://emitb1.ics.muni.cz:9000/diacy5Bp_bk0UyuVhoXFwA
    Current Status:     Running 
    Status Reason:      unavailable
    Destination:        cert-09.cnaf.infn.it:8443/cream-lsf-demo
    Submitted:          Fri May 20 17:31:15 2011 CEST
==========================================================================

Test 4: CREAM - ARGUS Interaction

Here we assume you have installed and configured your CREAM to use ARGUS (see page on cream installation /configuration) and your ARGUS

TEST 1: permit for testers VO users

1- Setting policy on ARGUS

[root@emitestbed10 ~]# cat siteinfo/policy/policy_test1.txt 
resource "http://cnaf.infn.it/cert-07" {                                                                                ------------------> NOTE: THIS RESOURCEID MUST BE SET ON THE CREAMCE
    obligation "http://glite.org/xacml/obligation/local-environment-map" {
    }
    action ".*" {
        rule permit { vo = "testers.eu-emi.eu" }
    }
}


#ADDING THE POLICY FROM FILE

[root@emitestbed10 ~]# pap-admin apf policy_test1.txt 


#then  listing policy

[root@emitestbed10 policy]# pap-admin lp

default (local):

resource "http://cnaf.infn.it/cert-07" {
    obligation "http://glite.org/xacml/obligation/local-environment-map" {
    }

    action ".*" {
        rule permit { vo="testers.eu-emi.eu" }
    }
}


#MAKING SURE THE POLICY IS IN THE CACHE !!!

[root@emitestbed10 policy]# /etc/init.d/argus-pepd clearcache
[root@emitestbed10 policy]# /etc/init.d/argus-pdp reloadpolicy


2- CE tests: again here we try to submit a job / uberftp ... (SEE TESTS ON CE ABOVE) Everything should be FINE.

3- ARGUS LOGS: to check that the CE (IP) contacts ARGUS, which makes a decision of PERMIT

less  /var/log/argus/pepd/access.log
2011-05-20T08:29:02.467Z|1305880142467|131.154.101.52|emitestbed10.cnaf.infn.it:8154|/authz| 
2011-05-20T08:31:01.318Z|1305880261317|131.154.101.52|emitestbed10.cnaf.infn.it:8154|/authz|

less  /var/log/argus/pepd/process.log
............

less  /var/log/argus/pepd/audit.log
2011-05-20T08:26:51.059Z|1305880011059|_74514a084a5cc4cfdc4a5bb9c79609c9|http://localhost:8152/authz|_df19b451dc22b200c37e7e6a885c05fd|Permit|

TEST 2: permit for testers VO users, BAN For myself

1- ADDING BAN POLICY on ARGUS


[root@emitestbed10 ~]#  pap-admin ban subject "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni"
[root@emitestbed10 ~]# pap-admin lp

default (local):

resource ".*" {

    action ".*" {
        rule deny { subject="CN=Danilo Nicola Dongiovanni,L=CNAF,OU=Personal Certificate,O=INFN,C=IT" }
    }
}

resource "http://cnaf.infn.it/cert-07" {
    obligation "http://glite.org/xacml/obligation/local-environment-map" {
    }

    action ".*" {
        rule permit { vo="testers.eu-emi.eu" }
    }
}


------------------------->>>>>>>>>>>> NOTICE THAT THE ORDER OF POLICIES MATTER: IF DENY FOR MYSELF IS AFTER PERMIT FOR TESTERS, I WOULD BE AUTHORIZED ON CE

#AGAIN MAKE SURE CACHE AND RELOAD AS BEFORE....

2- CE tests: again here we try to submit a job / uberftp ... (SEE TESTS ON CE ABOVE) JOB /UBERFTP SHOUD FAIL with Authorization error

[dongiovanni@emitestbed08 ~]$ uberftp cert-07 
220 cert-07.cnaf.infn.it GridFTP Server 3.28 (gcc64pthr, 1297437357-80) [Globus Toolkit 5.0.3] ready.
530-Login incorrect. : globus_gss_assist: Error invoking callout
530-globus_callout_module: The callout returned an error
530-an unknown error occurred
530 End.

3- RESTORE POLICY:

[root@emitestbed10 ~]#  pap-admin un-ban subject "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Danilo Nicola Dongiovanni"


Test 5: MPI ON CREAM

Take the "mpi_test.tar"

[dongiovanni@emitestbed08 MPITEST]$ ls
mpi-hooks.sh  mpi-start-wrapper.jdl  mpi-start-wrapper.sh  mpi_test.tar  name_mpi.c

where

$ [dongiovanni@emitestbed08 MPITEST]$ cat mpi-hooks.sh 
#!/bin/sh

#
# This function will be called before the MPI executable is started.
# You can, for example, compile the executable itself.
#
pre_run_hook () {

  # Compile the program.
  echo "Compiling ${I2G_MPI_APPLICATION}"

  which mpicc

  # Actually compile the program.
  cmd="mpicc ${MPI_MPICC_OPTS} -o ${I2G_MPI_APPLICATION} ${I2G_MPI_APPLICATION}.c"
  echo $cmd
  $cmd
  if [ ! $? -eq 0 ]; then
    echo "Error compiling program.  Exiting..."
    exit 1
  fi

  # Everything's OK.
  echo "Successfully compiled ${I2G_MPI_APPLICATION}"

  return 0
}

#
# This function will be called before the MPI executable is finished.
# A typical case for this is to upload the results to a storage
# element.
#
post_run_hook () {

  echo "Executing post hook."
  echo "Finished the post hook."

  return 0
}


[dongiovanni@emitestbed08 MPITEST]$ cat mpi-start-wrapper.jdl
JobType = "Normal";
CPUNumber = 2; 
Executable = "mpi-start-wrapper.sh";
Arguments = "name_mpi OPENMPI";
#Arguments = "name_mpi MPICH";
StdOutput = "std.out";
StdError = "std.err";
InputSandbox = {"mpi-start-wrapper.sh","mpi-hooks.sh","name_mpi.c"};
OutputSandbox = {"std.err","std.out"};
Requirements =
other.GlueCEInfoHostName == "cert-07.cnaf.infn.it";
  


[dongiovanni@emitestbed08 MPITEST]$ cat mpi-start-wrapper.sh 
#!/bin/bash

#/bin/ls -la
# Pull in the arguments.
MY_EXECUTABLE=`pwd`/$1
MPI_FLAVOR=$2

env | grep -i MPI

# Convert flavor to lowercase for passing to mpi-start.
MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'`

# Pull out the correct paths for the requested flavor.
eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH`

# Ensure the prefix is correctly set.  Don't rely on the defaults.
eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH
export I2G_${MPI_FLAVOR}_PREFIX

# Touch the executable.  It exist must for the shared file system check.
# If it does not, then mpi-start may try to distribute the executable
# when it shouldn't.
touch $MY_EXECUTABLE

# Setup for mpi-start.
export I2G_MPI_APPLICATION=$MY_EXECUTABLE
export I2G_MPI_APPLICATION_ARGS=
export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER
export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh
export I2G_MPI_POST_RUN_HOOK=mpi-hooks.sh

# If these are set then you will get more debugging information.
export I2G_MPI_START_VERBOSE=1
#export I2G_MPI_START_DEBUG=1

# Invoke mpi-start.
$I2G_MPI_START



[dongiovanni@emitestbed08 MPITEST]$ cat name_mpi.c 
/******************************************************************************/
/* name_mpi.c
*/
/* Test dei nomi dei processori                                               
*/
/* Fabio Spataro                                                              
*/
/******************************************************************************/

#include <stdio.h>
#include "mpi.h"

/*============================================================================*/

int main(int argc, char **argv)
{
   int        rank, np;
   char       name[MPI_MAX_PROCESSOR_NAME];
   int        resultlen;
   int        i;
   MPI_Status status;

/*----------------------------------------------------------------------------*/

   // INIZIALIZZAZIONE DI MPI
   MPI_Init(&argc, &argv);

   // OTTIENE IL RANK DEL PROCESSO NEL COMMUNICATOR STANDARD
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);

   // OTTIENE IL NUMERO DI PROCESSI NEL COMMUNICATOR STANDARD
   MPI_Comm_size(MPI_COMM_WORLD, &np);

/*----------------------------------------------------------------------------*/

   MPI_Get_processor_name(name, &resultlen);

/*----------------------------------------------------------------------------*/

   if(rank != 0)
   {
      MPI_Send(&resultlen, 1, MPI_INT, 0, 0, MPI_COMM_WORLD);
      MPI_Send(name, resultlen+1, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
   }

   if(rank == 0)
   {
      printf("[%d] Nome del processore = %s\n", rank, name);
     
      sleep(300);

      for(i = 1; i < np; i++)
      {
         MPI_Recv(&resultlen, 1, MPI_INT, i, 0, MPI_COMM_WORLD, &status);
         MPI_Recv(name, resultlen+1, MPI_CHAR, i, 0, MPI_COMM_WORLD, &status);      
         printf("[%d] Nome del processore = %s\n", i, name);
      }
   }

/*----------------------------------------------------------------------------*/

   // FINE DELLE OPERAZIONI CON MPI
   MPI_Finalize();
}

/*============================================================================*/

[dongiovanni@emitestbed08 MPITEST]$ 

  1. Then submit a job through WMS on a CREM CE with MPI enabled (on WNs too) and check it goes done success:

dongiovanni@emitestbed08 MPITEST]$ glite-wms-job-submit -c ../WMSCONF/conf_cert-13.conf -a mpi-start-wrapper.jdl




-- DaniloDongiovanni - 20-May-2011

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2011-07-08 - DaniloDongiovanniExternal
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback