Project Milestones

Message Brokers

• September 2008
  • Single broker set up DONE prod-grid-msg.cern.ch
• January 09
  • Network of brokers running DONE CERN/SRCE
    • COMMENT:: Some problems in getting good operations preocedures for network of brokers (e.g. restarts, config upgrades, removing problem brokers). Waiting on adding more until we have more experience. Target: June for 2 more brokers (INFN, AUTH ???)

NCG

• December 2008
  • GOCDB integration via Oracle DB – DONE
  • Integrated via XML interface to GOCDB – DONE Mid January 2009
  • Yaim modules for ROC level – DONE at start
  • Specific doc for a ROC – DONE at start
  • Full integration with MSG - DONE Mid January 2009
  • MSG → Nagios – DONE Mid January
• February 2009
  • feedback link from ROC to Site Nagios – DONE
    • PENDING - end may ?
  • Support for multiple VOs – DONE
• March 2009
  • Integration with JSON-based metric database – DONE
• April 2009
  • Integration with Metric Store (DONE) and new Topology Providers - no interface yet - still open
• May 2009
  • Implementation of NCG cache (NCG as Nagios check)

Regional Monitoring

• December 08
  • SAM ready to use Message Bus - DONE
    • SAM client on UI and WNs uses msg-publisher
    • Msg-consume2oracle used to retrieve messages from a topic and insert in SAM DB
    • Everything tested in Validation
  • Have Nagios equivalents of current SAM sensors used for WLCG/EGEE Availability calculations –
    • SRMv2; CE & CREAM CE; sBDII
    • NCG templates available
      • Dec - SRMv2, CE DONE. sBDII waiting on gstat developer
      • COMMENT: April - Site BDII tests are now ok. WN tests (CE) are now ok
• January 09
  • Test at CERN a full nagios monitoring system –
    • Pushed back to Feb due to delay in hardware request
    • (April 09) - 11 machines now setup
• April 09
  • Design of new SAM portal completed –
    • Feb - UNDER CONSIDERATION
    • April - Decision to try and use MyOSG as the DB against nagios data
  • Integration with new availability calculation at same level as current system
    • Pending on Gridview -
• June 09
  • Switch Regional Nagios metric publication to new metric store
  • Inegrated with Regionalized Dashboard alarm system
  • Adapt submission framework to use new aggregate topology provider
  • SAM portal layered on new metric store
  • 3rd party tool selected/customized for viewing NDO database

Project level repositories

• April 2009
  • 1st implementation of Metric store, aggregate topology provider and Metric Description Database
    • ATP -
      • 23 April - syncronizers nearly done. DB Schema done, history tables to be done
      • 7 May - syncronizers finished this week. Need to know the information that NCG will need.
    • MDDB - first version ready, integrated with NCG
    • Metric Store - early vrsion based on NDOUtils - under revision - may change schema.
• October 09
  • Aggregate topology provider packaged for easy deployment in regions

• December 09
  • Regional version of metric store available to function along with regional availability calculator

SLA Calculation

• March 2009
  • Test new metrics with old availability algortiihm (in old SAM SB)
    • April - PENDING
• June 2009
  • single availability algorithm calculated per VO with new metric store (e.g. functional equiv. of current system)
• August 2009
  • multiple availability calculations
  • Visualisation of Availability in web portal * December 2009
  • Regional version of availability calculation available

GOCDB

More details (including tasks dependencies) are available from https://wiki.egi.eu/wiki/GOCDB

• Feb 09
  • GOCDB-PI is available in a prototype state with certificate based authentication. DONE
  • A first test DB model is deployed DONE
  • Basic set of information to be contained in GOCDB-4 is defined DONE
  • First "region1" regional use case is defined. DONE

• Mar 09
  • GOCDB-PI read functions are in production. DONE
  • GOCDB-4 schema designed and deployed for testing purposes DONE

• April 09
  • A first API is implemented to interface new DB model. DONE
  • Data sets injected into new schema for test and validation. DONE

• May 09
  • First set of write function available in GOCDB-PI. DONE

• June 09
  • Basic APIs are available for tool developers that need to migrate to GOCDB-PI. DONE
  • A basic web interface is available in a prototype state to centrally access GOCDB new model. DONE
  • We have a "region1" use case deployed DONE

• Aug 09
  • A simple "GOCDB regional instance" test module is available for the regions that want to try it. DONE
  • GOCDB interactions with messaging system defined. DONE

• Sept 09
  • Scripts for large scale data migration from GOCDB3 to GOCDB4 are ready. DONE
  • interfaces to "region3" model are defined. DONE

• Oct 09
  • Access to GOCDB is only done through GOCDB-PI. DONE

• Nov 09
  • GOCDB4 central read-only web portal in beta version along with GOCDB3. DONE
  • GOCDB-PI is ready to access GOCDB4 data. DONE

• Dec 09
  • GOCDB4 central read/write portal in beta version. DONE
  • Beta regional package officially distributed through standard channels. DONE
  • GOCDB-PI production system accesses GOCDB4. DONE
  • Plans for GOCDB integration with messaging system finalized.

• Jan 10
  • GOCDB3 deprecated - GOCDB4 is the only production system
  • SOAP interface implemented.
  • Regional instances and central instance work in parallel in a prototype state

• Feb 10
  • Production quality regional package officially distributed through standard channels

• Mar 10
  • First regional usecases deployed as test/pilots enter production

Gstat

• March 09
  • Finish software requirements analysis and design - Done
  • Finish prototyping new model - Done
  • Regional use-cases Study - In progress
• September 09
  • GStat core components deployment ready.
    • GLUE and Topology models with associated import scripts
    • GStat validation scrips ready and initial feedback from deployment
    • GStat BDII monitoring probes ready and initial feedback from deployment
  • Prototype portal pages ready.
  • Start to investigate the migration plan and preparing for software transition
• December 09
  • GStat version 2.0 in production
  • Work well underway on enhancements.

Regional Dashboard

• July 09
  • Regionalized dashboard interfaced with Regional Nagios raising alarms based on Nagios
• Jan 10
  • Regional dashboard based on a Regional Nagios suitable for deployment at a region

Quality Metrics Portal

• Sep 09
  • Initial version of portal (infrastructure, common functionality ) - DONE - based on CESGA accounting portal
  • usage.1, usage.4, size.1 - DONE - based on GOCDB-PI
• Nov 09
  • operations.3, user-support.1, user-support.2 - In progress
• Nov 09
  • user-support.{345} - In progress
• Dec 09
  • operations.4
• Jan 10
  • operations.{12}, size.2

Accounting

• March 09
  • Create Central infrastructure for ActiveMQ
    • Install a message broker - DONE
    • Install an Accounting Archiver (consumer) - DONE
• April 09
  • Publisher written - DONE
• July 09
  • Distribute an APEL publisher that uses ActiveMQ as the transport mechanism
• April 09
  • Implement the APEL publisher
  • Create new APEL package
• June 09
  • Test version in different sites in parallel with R-GMA
  • Submit patch for g-Lite certification
  • Gather requirements/information from existing interfaces (OSG, NDGF and DGAS) configurable for different regions/sensors
• July 09
  • Implement APEL publisher configurable for different regions/sensors
• August 09
  • Implement consumer configurable for different regions/sensors
• October 09
  • Package publisher configurable for different regions/sensors for distribution
• December 09
  • Test publisher with other regions
• April 10
  • Create a package for the regionalised Accounting Servers ready for distribution

Accounting Portal

• September 2009
  • Detailed CSV in global View. -*DONE in develop portal*

• October 2009
  • Migration to new GOCDB-PI interface.
  • Create a new branch to support EELA project. - DONE in development portal
  • Create a chart for each country to show the percentage of jobs runned by foreign people.
  • Create a chart with the percentage of sites that publish their DN.

• November 2009
  • Add user DN tree.
  • Clean Global view.
  • Create report with contributed CPU.
  • Show installed capacity and resource usage by VO's at sites.

• December 2009
  • Remove unregistered sitenames like '%test%'.
  • Show RB related information.

• February 2010
  • Regionalize the EGEE accounting portal.

• Pending Date
  • Show HEP Spec related data.

Projects with no milestones attached

Operations Portal

SAMAP

Worker Node configuration (JobWrapper tests)

Site Nagios

Middleware event publication (L&B, FTS, Gridftp log parsing)