ATP: AGGREGATED TOPOLOGY PROVIDER

THIS PAGE IS NO LONGER MAINTAINED. ATP DOCUMENTATION IS NOW AVAILABLE HERE.

INTRODUCTION

In the area of tools for grid operations and monitoring, the Aggregated Topology Provider (ATP) is a service in charge of aggregating grid topology information from the different grid infrastructures (EGEE, OSG), projects, VOs, sites, services and downtimes over time. This information is consumed by different operational tools like GridView, SAM, Nagios instances, GridMap, dashboards, MyEGEE portal, MDDB, MRS and others, so they can all consume and share the same information.

The ATP authoritative sources of information are the following:

• GOCDB for EGEE sites, services and downtimes.
• RSV for OSG sites, services and downtimes.
• The CIC Portal for the list of VOs.
• The experiment VO feeds (see below) to define which are the services (among the ones defined by GOCDB and RSV) that they support.
• The BDII for any other VO, to define mapping between services and VOs supported.
• The MoU portal, to define the WLCG Site-Tier mapping.
• The GridMap interface, to extract CPU counts

COMPONENTS

• ATP is divided in two parts: ATP synchronizer with Oracle/MySQL based database schema and Django based web interface for presentation of topology information from the database back-end. Information about the configuration of these components is located here.

VO FEEDS

• The VO feeds are XML files used by the LHC VOs to define which are the services (among the ones defined by GOCDB and RSV) that they support. At the same time, they can define the name of each existing site as called by the VO.
• For more information about the VO feeds, you can check this URL: https://twiki.cern.ch/twiki/bin/view/LCG/ATPVOFeeds

ATP REPOSITORY

• The SVN repository is located here: http://svnweb.cern.ch/world/wsvn/atp/?

REQUIREMENTS DOCUMENT

• This is the ATP list of requirements

ATP ERD MODEL

• The ERD model of the ATP database can be seen here: ATP_Topology_20091127.png

DATABASE OBJECTS NAMING CONVENTIONS

The main goal is to identify the type and purpose of all database objects contained in the database. The information presented here serves as a guide to follow when naming the database objects.

• TABLES
  • Table Names
    • Must be short and unambiguous.
    • Use singular.
    • Use lowercase characters.
    • Separate words or prefixes with underlines '_'. Never use spaces.
    • Do not use numbers.
  • Columns
    • Use simple column names without any prefixes.
    • If the column is a Foreign Key, name it with the referenced table name and suffix '_id'
    • Example (MYSQL):

         create table car (
            id             INT,
            name        VARCHAR,
            description VARCHAR
         ) 

  • 'N' to 'N' Table Names
    • Use the names of the 2 tables separated by '_' and suffix with '_map', e.g. metric_profile_map
  • Unique Keys
    • Use the names of the column/s (removing '_' from the column name) and separate each column by '_'. Suffix the name with '_UN'
    • Example (see below): SERVICEID_DATAPROVIDERID_UN
  • Foreign Keys
    • As expressed above, if a column is a Foreign Key, name it as the rferenced table name and suffix '_id'
    • The convention for the Foreign Key name is: referenced table name plus the suffix '_FK'
    • Example of column: service_id
    • Example of Foreign Key name: SERVICE_FK
  • Primary Keys
    • Use a single column (autogenerated) always named 'id'
    • Create Unique Keys if you have more than 1 column which is a candidate for Primary Key.
    • Example (Oracle):

           CREATE TABLE SERVICE_LAST_SEEN (
              id               NUMBER(38),
              service_id       NUMBER(38),
              data_provider_id NUMBER(38),
              last_seen        NUMBER(38)
           ) 
      
           ALTER TABLE SERVICE_LAST_SEEN
              ADD CONSTRAINT service_fk FOREIGN KEY (service_id) REFERENCES service (id)
           ) NOT DEFERRABLE;
      
           ALTER TABLE SERVICE_LAST_SEEN
              ADD CONSTRAINT data_provider_fk FOREIGN KEY (data_provider_id) REFERENCES data_provider (id)
           ) NOT DEFERRABLE;
      
           ALTER TABLE SERVICE_LAST_SEEN 
              ADD CONSTRAINT serviceid_dataproviderid_un UNIQUE (service_id, data_provider_id);
      

• PROCEDURES and FUNCTIONS
  • Make the procedure/function name meaningful and separate words with '_'.
  • Example: DOWNTIME_UPDATE
  • Comment your code as much as possible so anyone can understand in the future what you did and why.
  • In the ATP we are prefixing the input variables with 'a_' and separating words with '_'.
  • Example:

          CREATE OR REPLACE PROCEDURE DOWNTIME_UPDATE (a_start_time IN NUMBER, a_end_time IN NUMBER, ...
       

  • Internal variables are prefixed with 'v_' and words are not separated with '_' but only capitalizing the first letter.
  • Example:

          ...
          v_downtimeId  NUMBER := 0;
          v_startTime   NUMBER;
          v_endTime     NUMBER;
          ...
       

• TRIGGERS
  • Name your triggers with the affected table name + '_' + when is the trigger executed (before insert, after insert, before delete, etc...)
  • Example:

          CREATE OR REPLACE TRIGGER VO_BEFORE_INSERT BEFORE INSERT ON VO
          FOR EACH ROW
             BEGIN
               SELECT VO_SEQ.nextval INTO :NEW.VOID FROM DUAL;
             END;
       

• SEQUENCES
  • Name your sequences using the affected table name and suffix it with '_SEQ'
  • Examples: VO_SEQ , SERVICE_SEQ