LCG deployment ============== - Total number of Sites (1): 263 - Status -> Num. Sites (1): ok -> 212 degraded -> 13 down -> 38 - Software -> Num. Sites (2): gLite-3_1_0 -> 235 gLite-3_0_2 -> 13 gLite-3_0_0 -> 1 - Average of concurrently running jobs during this week (3): 33.3k EGEE Pre-Production Service Coordination: ======================================== 2008-11-11: Pilot service of Cream CE: in progress 2008-12-10: Release of gLite 3.1 Update 38 to production in preparation 2008-12-10: gLite 3.1 PPS Update 41 was released to PPS and it is now going through the deployment test. Integration and Testing Report ============================== gLite 3.1 Patches scheduled for release to Production: #1579 - R3.1/SLC4/noarch: Hydra service #2017 - R3.1/SLC4/i386: Hydra client #2344 - R3.1/SLC4/x86_64: Proxy renewal 1.3.6 #2415 - First update of CREAM CE for slc4/i386 platform #2518 - MyProxy Updates, myproxy-config, yaim and info provider. #2522 - R3.1/SLC4/x86_64: Hydra client #2598 - R3.1 lcg-vomscerts-5.2.0 renames certificates #2599 - R3.1 lcg-vomscerts-5.2.0 renames certificates x86_64 #2644 - trustmanager configure.sh fix for new bouncycastle #2645 - trustmanager configure.sh fix for new bouncycastle (64bit) #2647 - Patch for glite-yaim-mon and APEL to deal with bcprov location SAM report: ========== New release of SAM Sensors to Production: https://twiki.cern.ch/twiki/bin/view/LCG/SamValidationSensors ETICS ===== The ETICS services are experiencing severe problems since last Friday. After a long investigation we have been able to find that the problems have been related mainly to two issues: 1) The ETICS repositories where packages and build/test reports are stored are hosted on AFS. The AFS server where the ETICS volumes are hosted experienced a surge of load since last Friday and the throughput suffered to the point of being unable to download any package or report. This has affected both the web and the command-line interfaces, which need to retrieve files from the repositories. The major problem has been that the connection to the server was established, but the clients were stuck waiting for a reply. The same problem has been observed even without using ETICS, simply by connecting to http://eticssoft.web.cern.ch/etics/repository which is a centrally hosted web site pointing to the repository The problem has now been fixed by the AFS team by moving the root of the ETICS project to a different server 2) In the past several days network connectivity to ETICS, but also to several other CERN hosted services or web applications, like Exchange, EDMS, AIS applications, grid monitoring applications, has been very unstable, especially when connecting from outside CERN. This problem has been reported by users from several sites in Czech Republic, United States, Italy, France and other countries. Connection to the services (usually web applications) was presenting all sorts of problems, from no connection at all, to applications hanging for very long times (or until the browser was timing them out). This problem is not yet solved at the time of writing this report, but we have been told that the Network Engineering team is investigating the issues with high priority. Operational Security ============= Further events related to the security incident affecting earlier this year academic sites (see report from 07 Aug) have been reported and are being investigated by the EGEE OSCT and the relevant academic CSIRTs. A new EUGridPMA distribution will be released next week, it includes the missing CRL for the NCSA CA, which triggered an alarm at most sites in SAM.