Savannah tasks

Savannah bugs

here

To do list:

  • Change the collector. Compute the total values for data transfer and job processing in the end of the loop Done on Jan 20 09 but only for the data transfer. The overall values are computed in a method of the DAO, not in the collector. And it is used only for CMS. For job processing it is not implemented, because no Vo needs it.
  • Try to modify the Gridmap, to have both data transfer in and out Done on 25 November
  • Finish the method fromAll in the AtlasAPIs class to compute the overall metrics on the basis of the data transfer metrics provided by dashboard for Atlas Done on Jan 19 09 For the total incoming traffic the URL is set to the cloud data transfer overview (for each site the corresponding cloud). Whereas for the total outgoing transfers, I just left the generic link to the general overview for all clouds. Also the time range (1 h or 4h ) is taken into account building the URL.
  • Set up a way to executes every hour the scripts which call the Atlas API to refresh the metrics about data transfer and job processing Done on Dec 11th: a class AtlasAPIs, implemented in the module atlasAPIScripts.py, updates metrics for job processing and for data transfer (also the overall values are computed, and the status). The module atlasAPIScripts.py is directly imported into the collector and it is executed at every loop of the collector. ok!
  • For the status of the data transfer activity for Atlas: modify the siteview.py file in order to set the rectangle color as the status of the metric 'success_rate'. Ok. Done on Dec 15
  • For CMS change the destination site to 'all' (unless pablo does it from his side). Done
  • For CMS: in the DAO implement a function to compute the total incoming data traffic Done on Dec. 16. Everything is done in the DAO.ComputeTotalDataTransfer. But maybe in the future it will change. Now Pablo sends the overall values for the incoming traffic, and partial values for the outgoing traffic. So I force the flow to call the ComputeTotalDataTransfer (just substituting the data_transfer activity with a fictitious total_incoming_data_transfer, so it's like if there was no overall value about data transfer provided by the experiment). This is a temporary fix
  • In the DAO: in the method: ComputeTotalDataTransfer CORRECT THE COMPUTATION OF THE STATUS FOR THE DATA TRANSFER ACTIVITY! Done on 16 Jan 09 but it's a temporary fix Currently that function is used only for CMS. I have hard coded the status on the basis of the success rate for the last hour. Possibly this is going to change, because we prefer to set the status on the basis of the 4h period values. Moreover, if the function has to be used for more VOs, the status cannot be hard coded. To be kept in mind.
  • In the database: drop the acti_metric table. And delete the old job activities of CMS which are not used any more (skimming, relval, rerecon, reprocessing) Done on 8th Dec. 2008
  • Set Alice and Atlas in capital letters Done on 26 Jan 09
  • New interaction with LHCb people: ask again about the status of the activity. Propose to compute it on the basis of the success rate of the last 24 hours Done on Dec. 16. Now Andrei provides for LHCb-job processing the metrics relative to the last hour and also to the last 24 hours and for the combination 'job processing - completed_jobs_24h' (and only for that) he provides a status
  • In the script: atlas_jobs.py modify the query to get the number of running jobs. Done on Dec 8
  • Decide if we want to store in the database the metrics about the last 24 hours and 12 hours that we use to compute the status. Outcome: yes! we decided to store the metrics used to compute the status, and also to display them in the context help, together with the metrics relative to the last hour. Done for LHCb and Atlas. To be implemented for CMS
  • Rename my collector and the logfile, and download from CVS the last version of the siteview module of Pablo. My collector has been renamed and restarted. It works fine. there where some conflicts with Pablo's code in the DAO. Now solved. Merge of the code done on Friday 3 Apr 09. My code commited in cvs and the production instance of siteview collector is deployed in production on dashboard06 (6 Apr 09)
  • Migrate the schema to the integration account. And change in the collector the coordinates of the schema to the writer account. And in the gridmap the coordinates of the schema to the reader account. DONE ON JAN 27
  • Solve the problem with GRIF: it's actually split into 2 sites TO DO
  • LHCb site status has 2 records relative to the same site! INFN-CNAF. this is in the URL, so maybe it's an error in the algorithm to write the metrics. Asked to Pablo (16 March 09)
    LCG.CNAF.it,general,all,-1,-1,n/a,1237206600,1237207200,http://dashb-lhcb-sam.cern.ch/dashboard/request.py/historicalserviceavailability?mode=serviceavl&algoId=82&sites=LCG.CNAF.it&services=all&timeRange=last24
    LCG.CNAF.it,general,all,-1,-1,up,1237206600,1237207200,http://dashb-lhcb-sam.cern.ch/dashboard/request.py/historicalserviceavailability?mode=serviceavl&algoId=82&sites=LCG.CNAF.it&services=all&timeRange=last24
    

    WHY? ASKED TO PABLO
  • Install awstats to monitor the usage of the siteview gridmap. Like here: http://lxarda18.cern.ch/awstats/awstats.pl?config=lxarda18.cern.ch Done but it does not work because cron jobs on dashboard06 don't work...
  • Fonts bigger! TO DO
  • Show all site name Done. Site names are dynamically downloaded from gridops portal. Watch out! Only Tier1 and Tier2 sites are listed, as Tier3 is a concept unknown in WLCG.
  • Create link to the specific site gridmap Done Now you can keep a bookmark of the Siteview gridmap for a specific site (before you were always redirected to the default one, CERN)
  • Links provided by LHCb are always related to running jobs. Ask for some link about the statistics of finished jobs. TO DO
  • Create historical views of metrics. to be done

-- ElisaLanciotti - 08 Jun 2009

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2010-11-10 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback