LCG deployment
==============

- Total number of Sites (1): 263
- Status -> Num. Sites (1):
 ok       -> 212
 degraded ->  13
 down     ->  38

- Software -> Num. Sites (2):
  gLite-3_1_0  -> 235
  gLite-3_0_2  ->  13
  gLite-3_0_0  ->   1

 - Average of concurrently running jobs during this week (3): 33.3k

 EGEE Pre-Production Service Coordination:
 ========================================
 2008-11-11: Pilot service of Cream CE: in progress
 2008-12-10: Release of gLite 3.1 Update 38 to production in preparation
 2008-12-10: gLite 3.1 PPS Update 41 was released to PPS and it is now
 going through the deployment test.
 Integration and Testing Report
 ==============================
 gLite 3.1

 Patches scheduled for release to Production:

 #1579 - R3.1/SLC4/noarch: Hydra service
 #2017 - R3.1/SLC4/i386: Hydra client
 #2344 - R3.1/SLC4/x86_64: Proxy renewal 1.3.6
 #2415 - First update of CREAM CE for slc4/i386 platform
 #2518 - MyProxy Updates, myproxy-config, yaim and info provider.
 #2522 - R3.1/SLC4/x86_64: Hydra client
 #2598 - R3.1 lcg-vomscerts-5.2.0 renames certificates
 #2599 - R3.1 lcg-vomscerts-5.2.0 renames certificates x86_64
 #2644 - trustmanager configure.sh fix for new bouncycastle
 #2645 - trustmanager configure.sh fix for new bouncycastle (64bit)
 #2647 - Patch for glite-yaim-mon and APEL to deal with bcprov location


 SAM report:
 ==========
 New release of SAM Sensors to Production:
 https://twiki.cern.ch/twiki/bin/view/LCG/SamValidationSensors


 ETICS
 =====

 The ETICS services are experiencing severe problems since last Friday.
 After a long investigation we have been able to find that the problems
 have been related mainly to two issues:

 1) The ETICS repositories where packages and build/test reports are
 stored are hosted on AFS. The AFS server where the ETICS volumes are
 hosted experienced a surge of load since last Friday and the throughput
 suffered to the point of being unable to download any package or report.
 This has affected both the web and the command-line interfaces, which
 need to retrieve files from the repositories. The major problem has been
 that the connection to the server was established, but the clients were
 stuck waiting for a reply. The same problem has been observed even
 without using ETICS, simply by connecting to

 http://eticssoft.web.cern.ch/etics/repository

 which is a centrally hosted web site pointing to the repository

 The problem has now been fixed by the AFS team by moving the root of the
 ETICS project to a different server

 2) In the past several days network connectivity to ETICS, but also to
 several other CERN hosted services or web applications, like Exchange,
 EDMS, AIS applications, grid monitoring applications, has been very
 unstable, especially when connecting from outside CERN. This problem has
 been reported by users from several sites in Czech Republic, United
 States, Italy, France and other countries. Connection to the services
 (usually web applications) was presenting all sorts of problems, from no
 connection at all, to applications hanging for very long times (or until
 the browser was timing them out).

 This problem is not yet solved at the time of writing this report, but
 we have been told that the Network Engineering team is investigating the
 issues with high priority.

Operational Security
=============

Further events related to the security incident affecting earlier this year academic sites (see report from 07 Aug) have been reported and are being investigated by the EGEE OSCT and the relevant academic CSIRTs.

A new EUGridPMA distribution will be released next week, it includes the missing CRL for the NCSA CA, which triggered an alarm at most sites in SAM.