Standard Compliance Testing

This page describes the available and planned test suites for standard compliance testing in EMI.

Available test suites

Testsuite Standard tested Short description
litmus HTTP and WebDAV litmus is a WebDAV server test suite, which aims to test whether a server is compliant with the WebDAV protocol as specified in RFC2518
nfs connectathon NFS4.1/pNFS Programs that can be used to test an implementation of the NFS Protocol. The tests run on a UNIX client and test server and client functions
pynfs NFS4.1/pNFS CITI maintains pynfs, a suite of several Python tools for NFS4, useful for testing purposes. Its primary components are a server tester, useful for testing server response to unusual error conditions and/or a badly behaving client
GlueValidator GLUE 2.0 The GLUE Validator is a simple python-based unit test suite that validates information against test cases. It works against both LDIF files and LDAP servers. In addition, a GLUE 2.0 LDAP schema validator is available
EMI CAR validator EMI CAR Test suite for validating EMI Compute Accounting Records
!StARval EMI StAR Test suite for validating EMI Storage Accounting Records
EMI ES Creation and Management Ports Testsuite EMI ES Management and Creation port types EMI ES Management and Creation testsuite based on the UNICORE Commandline Client (UCC)
EMI ES Delegation Port Testsuite EMI ES Delegation port type EMI ES Delegation testsuite
arcemiestest EMI ES Information port type EMI ES Information testsuite based on ARC
CANL OCSP, SHA2, others Auth validators for individual CANL packages

EMI 3 standard compliance testing

The table below list the MANDATORY standard compliance tests to be preformed by the product teams.


  • Test id: ID
  • Product: The product to be tested.
  • Test suit: The test utility to be used.
  • Fields to be filled by PTs:
    • Test page: Link to the detailed test plan or a dedicated test area, if available. If special configurations of the testbed are needed, these should be stated here.
    • Comment: Any relevant information to share. E.g. note here if the test is run automatically e.g. via a unit testing or similar framework.
    • Test outcome: For executed tests this field should be either FAILED or SUCCESS. If available, please provide URLs of problem tickets (FAILED) or URL to the test report (SUCCESS).

Test id Product Test suite Test page Comment Test outcome
S1 UNICORE CE EMI-ES delegation  

Test written by

There were 4 failures because UNICORE does NOT implement the following operations:
getInterfaceVersion(), getTerminationTime(), getVersion(), renewProxyReq()


Test report:

S2 CREAM EMI-ES delegation   Test written by


Test report:

S3 ARC CE EMI-ES delegation   Test written by


Test report:

S4 UNICORE CE EMI-ES Info Test written by SUCCESS Test report:
S5 CREAM EMI-ES Info Test written by SUCCESS Test report:
S6 ARC CE EMI-ES Info Test written by SUCCESS Test report:
S7 UNICORE CE EMI-ES Crt&Mgmt   Test written by

S8 CREAM EMI-ES Crt&Mgmt   Test written by

S9 ARC CE EMI-ES Crt&Mgmt   Test written by

S10 dCache litmus-webdav     RUN - basic tests pass, some advanced tests fail. See report for details:
S11 DPM litmus-webdav     SUCCESS:
S12 StoRM litmus-webdav   The test is considered successfull since PROPPATCH and LOCK operations have not been implemented because not of interest to StoRM users SUCCESS:
S13 dCache NFS4.1    

S15 StoRM NFS4.1   Test suite not suitable, StoRM uses GPFS NOT RUN
S16 Dcache STARval     SUCCESS
S18 StoRM STARval   Feature is not implemented yet NOT RUN
S19 UNICORE CE CAR validator Test run manually. CAR records are not stored on disk as text files normally, so the input was taken from the log dumps which can be turned on on the rus-service. SUCCESS Test report:
S20 CREAM CAR validator A 'minor' fail. If Processors or NodeCount information is not present, APEL publishes '0' instead of omitting the tag. FAIL
S21 ARC CE CAR validator Generate CAR records with the JURA from the ARC CE reports and run the emicar validator. SUCCESS Test report: ARC_CE_CAR_validation
S22 CANL JAVA auth validator All tests are run by JUnit, so are guaranteed to be passed. There is one real std. testsuite implemented: NIST (Path validation). Some optional parts which are unsupported by canl are not implemented. There are some functional tests for OCSP and SHA2 - linked too, though are not standards compliance tests. SUCCESS NIST report: OCSP: SHA2:
S23 CANL C auth validator OCSP and SHA2 use case tests run as a part of the normal caNl test Suite. The do not constitute a full standards compliance test, though. DONE
S24 CANL C++ auth validator Test written by SHA2 test is included in test suite CredentialsTest.cpp; OCSP and OCSP stapling are included in test suite CredentialsTest.cpp and OCSPStaplingTest.cpp. NIST certificates suite has not been completely supported by canl c++, so the test suite ( NIST_PKITSTest.cpp) is not supposed to succeed for all of the NIST certificates. SUCCESS (for SHA2 and OCSP/OCSP stapling), test report:
S25 UNICORE glue2 validator   4 failing tests: three because of 'open enumeration' values that are wrongly checked by the validator FAIL, report
S26 DPM glue2 validator     FAIL, report and ticket at
S27 dCache glue2 validator     RUN for glue1 and glue2 see reports:
S28 StoRM glue2 validator     RUN, report: glue_validator_storm_new.txt
S29 FTS3 glue2 validator   FTS3 is still prototype NOT RUN
S30 CREAM glue2 validator   Modified the for extending the default values of Capability_t, InterfaceName _t and Platform_t GLUE2 types. This is needed for supporting the GLUE2 extensions introduced by the EMI-ES specification (e.g.'org.ogf.glue.emies.activityinfo', 'org.ogf.glue.emies.activitycreation' ... interface names, etc). SUCCESS, report: glue2_validator_report_CREAM.txt
S31 WMS glue2 validator     SUCCESS, report: glue_validator_wms.txt
S32 ARC CE glue2 validator Test fails because of missing hardcoded open enumerations; I think this is wrong, it should just give a warning. Some open enumerations are new and are not part of the validator code FAILED but false positives - see log:
S33 VOMS glue2 validator     SUCCESS,on SL6 report: glue_validator_voms.txt
S34 ARGUS glue2 validator   The glue-validator tests failed, but the errors are in the test themself, not in Argus. Reports: argus-EMI3_RC-glue-validator-SL6.txt argus-EMI3_RC-glue-validator-SL5.txt RUN
S35 L&B glue2 validator Fixes were required in L&B (fixed immediately) and Info Providers ( All OK now. SUCCESS

-- JonKerrNilsen - 11-Nov-2011

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatext ARC_CE_CAR_validation r1 manage 13.0 K 2013-02-19 - 10:36 UnknownUser ARC CE - generated CAR record validation results
Texttxt DelegationService_report_CREAM.txt r1 manage 1.0 K 2013-02-07 - 16:59 LisaZangrando  
Unknown file formatlog S32_glue_validator_ARC_CE.log r1 manage 22.2 K 2013-02-19 - 12:24 FloridoPaganelli Test S32 - ARC CE GLUE2 Ldap validation results
Texttxt glue2_validator_report_CREAM.txt r1 manage 8.5 K 2013-02-07 - 16:20 LisaZangrando  
Texttxt glue_validator_argus.txt r1 manage 5.5 K 2013-02-15 - 17:23 DoinaCristinaAiftimiei VOMS & ARGUS glue_validator resutls
Texttxt glue_validator_storm.txt r1 manage 9.0 K 2013-02-15 - 17:32 DoinaCristinaAiftimiei StoRM glue_validator resutls
Texttxt glue_validator_storm_new.txt r1 manage 16.8 K 2013-02-19 - 23:59 DoinaCristinaAiftimiei StoRM glue_validator resutls
Texttxt glue_validator_voms.txt r1 manage 2.1 K 2013-02-15 - 17:23 DoinaCristinaAiftimiei VOMS & ARGUS glue_validator resutls
Texttxt glue_validator_wms.txt r1 manage 3.7 K 2013-02-15 - 15:14 DoinaCristinaAiftimiei WMS glue_validator resutls
Unknown file formatext ocsp_stapling_canlxx r1 manage 7.7 K 2013-02-16 - 17:24 UnknownUser Test report for canlxx (SHA2 and OCSP support)
Unknown file formatext sha2_and_ocsp_canlxx r1 manage 5.3 K 2013-02-16 - 17:18 UnknownUser Test report for canlxx (SHA2 and OCSP support)
Texttxt standards_test_emi_delegation_ARC_report.txt r1 manage 0.7 K 2013-02-20 - 12:01 LisaZangrando  
Texttxt standards_test_emi_delegation_UNICORE_report.txt r1 manage 1.3 K 2013-02-26 - 17:31 LisaZangrando  
Edit | Attach | Watch | Print version | History: r37 < r36 < r35 < r34 < r33 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r37 - 2013-06-06 - JonKerrNilsen
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback