Test Results:
==========
glite-apel-lsf
Functionality Tests:
--------------------
Automated Tests:
[root@novales apel-parsers_tests]# ./functionality.sh
F1. DELETE PROCESSOR CLEANALL=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F2. DELETE PROCESSOR CLEANALL=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F4. DBPROCESSOR INSPECTTABLES="NO"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F7. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F8. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F9. EVENTLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
F10. EVENTLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F11. EVENTLOGPROCESSOR WITH EXTRA FILES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F12. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F13. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F14. BLAHDLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F15. BLAHDLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F16. BLAHDLOGPROCESSOR WITH PREFIX=grid_jobmap
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F17. BLAHDLOGPROCESSOR WITH PREFIX=blahp.log-
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F18. RUNNING THE LSF PARSER WITH EVENTLOGPROCESSOR USING USESCALINGFACTOR="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records inserted .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F19. RUNNING THE LSF PARSER WITH EVENTLOGPROCESSOR USING USESCALINGFACTOR="NO"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records inserted .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Manual Tests:
F5. Running the parser with CPUProcessor reading from standard port 2170.
1. Emptying MySQL database.
2. Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
wipp-ce.weizmann.ac.il
3. Running the apel-publisher:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-lsf-log-parser -f /etc/glite-apel-lsf/f5.xml
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=wipp-ce.weizmann.ac.il]
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - ------ Starting the apel application ------
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Finding CPU spec values from GIIS server: wipp-ce.weizmann.ac.il
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - LDAP URL: ldap://wipp-ce.weizmann.ac.il:2170
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:46:29 UTC 2011: apel-lsf-log-parser - ------ Processing finished ------
4. Contents of SpecRecords after running the parser:
mysql> select * from SpecRecords;
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | 2011-09-12 | 08:46:29 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:46:29 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | NULL | NULL | 2011-09-12 | 08:46:29 |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
22 rows in set (0.00 sec)
Result as expected: PASS
F6. Running the parser with CPUProcessor reading from standard port 2135:
No test site available.
Regression Tests:
---------------------------
Automated Tests:
https://savannah.cern.ch/bugs/?30041
https://savannah.cern.ch/bugs/?73603
https://savannah.cern.ch/bugs/?76204
[root@novales apel-parsers_tests]# ./regression.sh
#30041. PROBLEM IN PARSING LSF ACCOUNTING LOGS
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking that the file lsb.acct has been parsed .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
#73603. LOGGING OF BLAHLOGPROCESSOR TOO VERBOSE
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking log file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
#76204. APEL LSF TO NORMALISE CPU/WCT DATA WITH HOST FACTOR VALUE PER JOB
Replaced with F18 ....
Manual Tests:
https://savannah.cern.ch/bugs/?23796
https://savannah.cern.ch/bugs/?65723
https://savannah.cern.ch/bugs/?72270
https://savannah.cern.ch/bugs/?79417
https://savannah.cern.ch/bugs/?86449
#23796. Too many open files (common)
1. Reduced ulimit to 500 (ulimit -n 500)
2. Created 1000 log files for testing (./create_files.sh lsb.acct. lsb.acct)
3. Changed configuration file to:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
testce
/root/apel-parsers_tests/23796
UTC
4. Run parser:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-lsf-log-parser -f /etc/glite-apel-lsf/23795.xml
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - Read-in configuration: [quiet, e] [CEhost=testce, DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, timeZone=UTC, DBPassword=****, site=TEST_SITE, pbslog=/root/apel-parsers_tests/23796]
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - ------ Starting the apel application ------
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - Reprocess disabled, checking new event logs only
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - **** Updating Batch Log event table (EventRecords) ****
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111849
Mon Sep 12 08:52:51 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111882
Mon Sep 12 08:52:52 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111915
Mon Sep 12 08:52:52 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111387
Mon Sep 12 08:52:52 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111014
......
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111669
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111363
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111911
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111788
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111488
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111290
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Processing /root/apel-parsers_tests/23796/lsb.acct.20111332
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - Event records inserted: 10020
Mon Sep 12 08:53:07 UTC 2011: apel-lsf-log-parser - ------ Processing finished ------
Result as expected: PASS
#65723. APEL assumes cluster and subcluster ID are the same as the CE hostname (common).
Site RAL-LCG2, GIIS: lcgce06.gridpp.rl.ac.uk. CE name different than ClusterID.
Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
lcgce06.gridpp.rl.ac.uk
Running the parser:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-lsf-log-parser -f /etc/glite-apel-lsf/65723.xml
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=lcgce06.gridpp.rl.ac.uk]
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - ------ Starting the apel application ------
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Finding CPU spec values from GIIS server: lcgce06.gridpp.rl.ac.uk
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - LDAP URL: ldap://lcgce06.gridpp.rl.ac.uk:2170
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - Found New ClusterId, now storing it
Mon Sep 12 08:56:21 UTC 2011: apel-lsf-log-parser - ------ Processing finished ------
Contents of SpecRecords:
mysql> select * from SpecRecords;
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | 2011-09-12 | 08:56:21 | 1970-01-01 | 00:00:00 |
| 2011-09-12 08:56:21 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 08:56:21 |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
18 rows in set (0.00 sec)
Result as expected: PASS
#72270: APEL Parser automatically adds a check table for the BlahdRecords table (common).
1. Enabled logging of queries in MySQL database:
Added line log=/tmp/queries.log to /etc/my.cnf and restarted mysqld
2. Run test f3:
[root@novales apel-parsers_tests]# ./functionality.sh f03
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
3. Check for CHECK TABLE in log of queries:
[root@novales apel-parsers_tests]# grep -i "CHECK TABLE" /tmp/queries.log
Result: PASS
#79417. glite-apel-lsf is not FHS compliant (glite-apel-lsf)
1. List the files in the rpm:
[root@novales apel-parsers_tests]# rpm -ql glite-apel-lsf
/etc/glite-apel-lsf
/etc/glite-apel-lsf/log4j.properties
/etc/glite-apel-lsf/parser-config.xml
/usr/bin/apel-lsf-log-parser
/usr/share/doc/glite-apel-lsf
/usr/share/doc/glite-apel-lsf/LICENSE
/usr/share/doc/glite-apel-lsf/html
/usr/share/doc/glite-apel-lsf/html/allclasses-frame.html
/usr/share/doc/glite-apel-lsf/html/allclasses-noframe.html
/usr/share/doc/glite-apel-lsf/html/constant-values.html
/usr/share/doc/glite-apel-lsf/html/deprecated-list.html
/usr/share/doc/glite-apel-lsf/html/help-doc.html
/usr/share/doc/glite-apel-lsf/html/index-all.html
/usr/share/doc/glite-apel-lsf/html/index.html
/usr/share/doc/glite-apel-lsf/html/org
/usr/share/doc/glite-apel-lsf/html/org/glite
/usr/share/doc/glite-apel-lsf/html/org/glite/apel
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/ApelLogParser.html
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/class-use
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/class-use/ApelLogParser.html
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/package-frame.html
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/package-summary.html
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/package-tree.html
/usr/share/doc/glite-apel-lsf/html/org/glite/apel/lsf/package-use.html
/usr/share/doc/glite-apel-lsf/html/overview-tree.html
/usr/share/doc/glite-apel-lsf/html/package-list
/usr/share/doc/glite-apel-lsf/html/resources
/usr/share/doc/glite-apel-lsf/html/resources/inherit.gif
/usr/share/doc/glite-apel-lsf/html/stylesheet.css
/usr/share/java/glite-apel-lsf.jar
Result: PASS
#86449: LSF parser script using wrong mysql connector jar.
[root@novales apel-parsers_tests]# grep "mysql" /usr/bin/apel-lsf-log-parser
MYSQL_DRIVER_CP="/usr/share/java/mysql-connector-java.jar"
Result: PASS
glite-apel-pbs
Functionality Tests:
--------------------
Automated Tests:
[root@novales apel-parsers_tests]# ./functionality.sh
F1. DELETE PROCESSOR CLEANALL=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F2. DELETE PROCESSOR CLEANALL=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F4. DBPROCESSOR INSPECTTABLES="NO"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F7. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F8. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F9. EVENTLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
F10. EVENTLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F11. EVENTLOGPROCESSOR WITH EXTRA FILES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F12. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F13. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F14. BLAHDLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F15. BLAHDLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F16. BLAHDLOGPROCESSOR WITH PREFIX=grid_jobmap
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F17. BLAHDLOGPROCESSOR WITH PREFIX=blahp.log-
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Manual Tests:
F5. Running the parser with CPUProcessor reading from standard port 2170.
1. Emptying MySQL database.
2. Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
wipp-ce.weizmann.ac.il
3. Running the apel-publisher:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/f5.xml
Mon Sep 12 14:03:30 UTC 2011: apel-pbs-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=wipp-ce.weizmann.ac.il]
Mon Sep 12 14:03:30 UTC 2011: apel-pbs-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Finding CPU spec values from GIIS server: wipp-ce.weizmann.ac.il
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - LDAP URL: ldap://wipp-ce.weizmann.ac.il:2170
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:03:31 UTC 2011: apel-pbs-log-parser - ------ Processing finished ------
4. Contents of SpecRecords after running the parser:
mysql> select * from SpecRecords;
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | 2011-09-12 | 14:03:31 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:03:31 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:03:31 |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
22 rows in set (0.00 sec)
Result as expected: PASS
F6. Running the parser with CPUProcessor reading from standard port 2135:
No test site available.
Regression Tests:
-----------------
Automated Tests:
https://savannah.cern.ch/bugs/?73603
https://savannah.cern.ch/bugs/?79175
[root@novales apel-parsers_tests]# ./regression.sh
#73603. LOGGING OF BLAHLOGPROCESSOR TOO VERBOSE
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking log file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
#79175. EMI APEL PATH FOR CONFIGURATION FILE WRONG
Checking config file is in /etc .... [ OK ]
Manual Tests:
https://savannah.cern.ch/bugs/?23796
https://savannah.cern.ch/bugs/?65723
https://savannah.cern.ch/bugs/?72270
https://savannah.cern.ch/bugs/?86641
#23796. Too many open files (common)
1. Reduced ulimit to 500 (ulimit -n 500)
2. Created 1000 log files for testing (./create_files.sh "" 20100816)
3. Changed configuration file to:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
testce
/root/apel-parsers_tests/23796
UTC
4. Run parser:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/23796.xml
[root@novales 23796]# env APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/23796.xml
Mon Sep 12 14:06:54 UTC 2011: apel-pbs-log-parser - Read-in configuration: [quiet, e] [CEhost=testce, DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, timeZone=UTC, DBPassword=****, site=TEST_SITE, pbslog=/root/apel-parsers_tests/23796]
Mon Sep 12 14:06:54 UTC 2011: apel-pbs-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:06:54 UTC 2011: apel-pbs-log-parser - Reprocess disabled, checking new event logs only
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - **** Updating PBS end event table (EventRecords) ****
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111492
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111821
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111531
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111555
Mon Sep 12 14:06:55 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111472
.....
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111987
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111482
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111843
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111380
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111837
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111911
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111973
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Processing batch log file: novales.esc.rl.ac.uk /root/apel-parsers_tests/23796/20111769
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - Event records inserted: 0
Mon Sep 12 14:07:01 UTC 2011: apel-pbs-log-parser - ------ Processing finished ------
Result as expected: PASS
#65723. APEL assumes cluster and subcluster ID are the same as the CE hostname (common).
Site RAL-LCG2, GIIS: lcgce06.gridpp.rl.ac.uk. CE name different than ClusterID.
Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
lcgce06.gridpp.rl.ac.uk
Running the parser:
[root@novales 23796]# env APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/65723.xml
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=lcgce06.gridpp.rl.ac.uk]
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Finding CPU spec values from GIIS server: lcgce06.gridpp.rl.ac.uk
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - LDAP URL: ldap://lcgce06.gridpp.rl.ac.uk:2170
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:08:07 UTC 2011: apel-pbs-log-parser - ------ Processing finished ------
Contents of SpecRecords:
mysql> select * from SpecRecords;
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | 2011-09-12 | 14:08:07 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:08:07 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:08:07 |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
18 rows in set (0.00 sec)
Result as expected: PASS
#72270: APEL Parser automatically adds a check table for the BlahdRecords table (common).
1. Enabled logging of queries in MySQL database:
Added line log=/tmp/queries.log to /etc/my.cnf and restarted mysqld
2. Run test f3:
[root@novales apel-parsers_tests]# ./functionality.sh f03
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
3. Check for CHECK TABLE in log of queries:
[root@novales apel-parsers_tests]# grep -i "CHECK TABLE" /tmp/queries.log
Result: PASS
#86641. PBS parser script using wrong mysql connector jar.
[root@novales apel-parsers_tests]# grep "mysql" /usr/bin/apel-sge-log-parser
MYSQL_DRIVER_CP="/usr/share/java/mysql-connector-java.jar"
Result: PASS
glite-apel-sge
Functionality Tests:
--------------------
Automated Tests:
[root@novales apel-parsers_tests]# ./functionality.sh
F1. DELETE PROCESSOR CLEANALL=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F2. DELETE PROCESSOR CLEANALL=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Inserting test data into database .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F4. DBPROCESSOR INSPECTTABLES="NO"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F7. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F8. EVENTLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F9. EVENTLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
F10. EVENTLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F11. EVENTLOGPROCESSOR WITH EXTRA FILES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F12. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F13. BLAHDLOGPROCESSOR WITH SUBDIRECTORIES=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F14. BLAHDLOGPROCESSOR WITH REPROCESS=YES
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F15. BLAHDLOGPROCESSOR WITH REPROCESS=NO
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F16. BLAHDLOGPROCESSOR WITH PREFIX=grid_jobmap
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
F17. BLAHDLOGPROCESSOR WITH PREFIX=blahp.log-
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking number of records in EventRecords .... [ OK ]
Checking number of records in BlahdRecords .... [ OK ]
Checking number of records in GkRecords .... [ OK ]
Checking number of records in MessageRecords .... [ OK ]
Checking number of records in LcgRecords .... [ OK ]
Checking number of records in SpecRecords .... [ OK ]
Checking number of records in LcgProcessedFiles .... [ OK ]
Checking number of records in RepublishInfo .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Manual Tests:
F5. Running the parser with CPUProcessor reading from standard port 2170.
1. Emptying MySQL database.
2. Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
wipp-ce.weizmann.ac.il
3. Running the apel-publisher:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-sge-log-parser -f /etc/glite-apel-sge/f5.xml
Mon Sep 12 14:31:44 UTC 2011: apel-sge-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=wipp-ce.weizmann.ac.il]
Mon Sep 12 14:31:44 UTC 2011: apel-sge-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:31:44 UTC 2011: apel-sge-log-parser - Finding CPU spec values from GIIS server: wipp-ce.weizmann.ac.il
Mon Sep 12 14:31:44 UTC 2011: apel-sge-log-parser - LDAP URL: ldap://wipp-ce.weizmann.ac.il:2170
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:31:45 UTC 2011: apel-sge-log-parser - ------ Processing finished ------
4. Contents of SpecRecords after running the parser:
mysql> select * from SpecRecords;
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-lhcb | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-see | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-alice | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-cms | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-dteam | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-geant4 | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlasil | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ilc | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-zeus | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-ops | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
| 1970-01-01 00:00:00 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | 2011-09-12 | 14:31:45 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:31:45 TEST_SITE wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | TEST_SITE | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | wipp-ce.weizmann.ac.il:2119/jobmanager-lcgpbs-atlas | 2700 | NULL | NULL | NULL | 2011-09-12 | 14:31:45 |
+-------------------------------------------------------------------------------------+-----------+-------------------------------------------------------+-------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
22 rows in set (0.00 sec)
Result as expected: PASS
F6. Running the parser with CPUProcessor reading from standard port 2135:
No test site available.
Regression Tests:
-----------------
Automated Tests:
https://savannah.cern.ch/bugs/?73603
https://savannah.cern.ch/bugs/?79175
https://savannah.cern.ch/bugs/?79517
https://savannah.cern.ch/bugs/?84078
[root@novales apel-parsers_tests]# ./regression.sh
#73603. LOGGING OF BLAHLOGPROCESSOR TOO VERBOSE
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking log file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
#79175. EMI APEL PATH FOR CONFIGURATION FILE WRONG
Checking config file is in /etc .... [ OK ]
#79517. SGE PARSER REPROCESSES EVERY LOG FILE
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
#84078. SGE PARSER READING WRONG FIELDS
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Checking output .... [ OK ]
Checking output .... [ OK ]
Checking output .... [ OK ]
Checking output .... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Manual Tests:
https://savannah.cern.ch/bugs/?23796
https://savannah.cern.ch/bugs/?65723
https://savannah.cern.ch/bugs/?72270
https://savannah.cern.ch/bugs/?86448
#23796. Too many open files (common)
1. Reduced ulimit to 500 (ulimit -n 500)
2. Created 1000 log files for testing (./create_files.sh accounting. accounting)
3. Changed configuration file to:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
testce
/root/apel-parsers_tests/23796
UTC
4. Run parser:
[root@novales 23796]# env APEL_HOME=/ /usr/bin/apel-sge-log-parser -f /etc/glite-apel-sge/23796.xml
Mon Sep 12 14:33:41 UTC 2011: apel-sge-log-parser - Read-in configuration: [quiet, e] [CEhost=testce, DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, timeZone=UTC, DBPassword=****, site=TEST_SITE, pbslog=/root/apel-parsers_tests/23796]
Mon Sep 12 14:33:41 UTC 2011: apel-sge-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:33:41 UTC 2011: apel-sge-log-parser - Reprocess disabled, checking new event logs only
Mon Sep 12 14:33:41 UTC 2011: apel-sge-log-parser - **** Updating SGE end event table (EventRecords) ****
Mon Sep 12 14:33:41 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111515
Mon Sep 12 14:33:43 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111581
Mon Sep 12 14:33:44 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111274
Mon Sep 12 14:33:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111995
Mon Sep 12 14:33:46 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111602
.....
Mon Sep 12 14:34:44 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111735
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111809
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111100
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111212
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111874
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111677
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Processing /root/apel-parsers_tests/23796/accounting.20111009
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - Event records inserted: 385770
Mon Sep 12 14:34:45 UTC 2011: apel-sge-log-parser - ------ Processing finished ------
Result as expected: PASS
#65723. APEL assumes cluster and subcluster ID are the same as the CE hostname (common).
Site RAL-LCG2, GIIS: lcgce06.gridpp.rl.ac.uk. CE name different than ClusterID.
Configuration file:
TEST_SITE
jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting
accounting
apel_password
lcgce06.gridpp.rl.ac.uk
Running the parser:
[root@novales apel-parsers_tests]# env APEL_HOME=/ /usr/bin/apel-sge-log-parser -f /etc/glite-apel-sge/65723.xml
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Read-in configuration: [quiet, c] [DBUsername=accounting, DBURL=jdbc:mysql://novales.esc.rl.ac.uk:3306/accounting, DBPassword=****, site=TEST_SITE, GIIS=lcgce06.gridpp.rl.ac.uk]
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - ------ Starting the apel application ------
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Finding CPU spec values from GIIS server: lcgce06.gridpp.rl.ac.uk
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - LDAP URL: ldap://lcgce06.gridpp.rl.ac.uk:2170
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - Found New ClusterId, now storing it
Mon Sep 12 14:35:57 UTC 2011: apel-sge-log-parser - ------ Processing finished ------
Contents of SpecRecords:
mysql> select * from SpecRecords;
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| SpecID | SiteName | ClusterID | SubClusterID | SpecInt2000 | SpecFloat2000 | EndDate | EndTime | MeasurementDate | MeasurementTime |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid500M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridWN | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-gridS | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid6000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid4000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid700M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid3000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid2000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
| 1970-01-01 00:00:00 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | 2011-09-12 | 14:35:57 | 1970-01-01 | 00:00:00 |
| 2011-09-12 14:35:57 TEST_SITE lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | TEST_SITE | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | lcgce06.gridpp.rl.ac.uk:2119/jobmanager-lcgpbs-grid1000M | 1000 | NULL | NULL | NULL | 2011-09-12 | 14:35:57 |
+----------------------------------------------------------------------------------------+-----------+----------------------------------------------------------+----------------------------------------------------------+-------------+---------------+------------+----------+-----------------+-----------------+
18 rows in set (0.00 sec)
Result as expected: PASS
#72270: APEL Parser automatically adds a check table for the BlahdRecords table (common).
1. Enabled logging of queries in MySQL database:
Added line log=/tmp/queries.log to /etc/my.cnf and restarted mysqld
2. Run test f3:
[root@novales apel-parsers_tests]# ./functionality.sh f03
F3. DBPROCESSOR INSPECTTABLES="YES"
Changing configuration file .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
Emptying the DB .... [ OK ]
Inserting test data into database .... [ OK ]
Comparing values LcgRecords.... [ OK ]
Comparing values BlahdRecords.... [ OK ]
Comparing values EventRecords.... [ OK ]
Comparing values LcgProcessedFiles.... [ OK ]
Comparing values RepublishInfo.... [ OK ]
Comparing values GkRecords.... [ OK ]
Comparing values MessageRecords.... [ OK ]
Checking output .... [ OK ]
Emptying the DB .... [ OK ]
Inserting schema .... [ OK ]
3. Check for CHECK TABLE in log of queries:
[root@novales apel-parsers_tests]# grep -i "CHECK TABLE" /tmp/queries.log
Result: PASS
#86448. SGE parser script using wrong mysql connector jar.
[root@novales apel-parsers_tests]# grep "mysql" /usr/bin/apel-sge-log-parser
MYSQL_DRIVER_CP="/usr/share/java/mysql-connector-java.jar"
Result: PASS