Quarterly plan for November 2012 -January 2013

Job monitoring area

Historical view

  • Import T0 ATLAS jobs statistics in Historical Views
  • Enable hourly statistics for up to a month
  • Enable statistics and plots for output files and bytes #90876
  • Implement feature requests and bug fixes tracked through Savannah

Interactive interface


Task monitoring

ATLAS Production task monitoring

  • Deploy implemented changes from June 2012 requests
    • tickets:
Ticket ID Short description
#98231 ATLAS Task monitoring: export search result
#98230 ATLAS Task monitoring: direct access to job summary for one task
#98228 ATLAS Task monitoring: integrated time per task
#98105 ProdUII -> Job Table: Show jobs which reached max attempt number
#97221 ATLAS Task monitoring -- definition of Processing time per event
#97219 ATLAS Task monitoring -- "expand job details" button on the job list page
#97112 ATLAS Task monitoring -- Pdone=int(Done/Total)
#95434 ATLAS Task monitoring -- please add a new view with summary per cloud
#95433 ATLAS Task monitoring -- please add failure pattern to the page with list of jobs, the subtable under + button
#95431 ATLAS Task monitoring -- please add filter button to show/hide finished jobs in the job list table
#95429 ATLAS Task monitoring -- please add filters to the job list table page
#95428 ATLAS Task monitoring -- please add column RunTime to job list table
#95427 ATLAS Task monitoring -- please add column MaxRunTime to task list table
#95414 ATLAS Task monitoring -- show full task name behaviour
#95412 ATLAS Task monitoring -- please add "Show all" option to the pagination setup in the task list table
#95411 ATLAS Task monitoring -- please add column "how many jobs reached #attempts and will not be auto-resubmitted"
#95402 ATLAS Task monitoring -- please change link under Task ID in the table with list of tasks
#95400 ATLAS Task monitoring -- change task ID representation on the page with list of jobs
#95398 ATLAS Task monitoring -- when filtered by taskID, task priority filter field is not disabled/greyed
#95396 ATLAS Task monitoring -- please add the "Reset" button to the filter
#95130 ATLAS Task monitoring -- please revise color legend for Graphically column
#95091 ATLAS Task monitoring -- please add filter option "show me tasks with jobs with attempt nr larger than N"
#79005 Panda Rebrokerage Monitoring
  • implement "stucked task monitoring" in the ATLAS Production task monitoring: collector, DB procedures, UI
  • implement new requests coming from the key consumers: MC production coordinators, Group production managers, ProdSys-II coordinators, ADC Management, ...

ATLAS Analysis task monitoring

  • JaroslavaSchovancova: prepare content of the ATLAS Analysis task monitoring tutorial for the ATLAS Offline SW tutorial 3rd-7th December 2012 (indico agenda)
  • Laura + JaroslavaSchovancova: participate in the tutorial
  • Make the users' page ATLAS-wide visible: I can see my jobs. I can see my colleague's jobs.
  • Proceed with job killing/job resubmission features
  • implement new requests coming from the monitoring tool consumers


DDM accounting

  • Add a new high-level grouping of the 'Projects' bug #98661
  • Enable statistics for tag related summaries bug #98662
    • Benchmark the performance of the collector and of the SQL queries
    • Come up with 3-level grouping of the tags to properly scale the up - at the moment we have 1.6 million rows per day

Transfer monitoring area

ATLAS DDM Dashboard

  • V2.0 M4 (November 2012)
    • Staging statistics and plots.

  • V2.0 M5 (January 2013)
    • Staging and Registration details view.
    • Rewrite old statistics procedures.
    • Clean-up raw tables.
    • Minor UI feature requests

WLCG Transfers Dashboard

  • Integrate ALICE traffic
  • Improve user interface (change autocomplete fields byt Tokenize input)
  • Investigate if correlated ranking plots can be integrated

ATLAS PD2P Dashboard

  • JaroslavaSchovancova's Tentative plan as of 2012-11-12
  • http://dashb-jarka.cern.ch/dashboard/request.py/pd2pmonitoring
  • Implement collector for "external" PD2P monitoring
  • Implement snapshots/history plots for "external" PD2P monitoring
    • How many datasets/files/bytes have been PD2P'd in a selected time interval with respect to site(or cloud or tier or federation)/data_project/datatype/data_stream.
  • Implement plots which were needed by Borut for ATLAS RRB report in August 2012
    • PD2P data popularity
    • PD2P data replica age distribution
  • Implement new requests from the ATLAS Computing coordinations (to be specified in November)

Google Earth

Handling of the Dashboard cluster

  • Replace out of warranty machines
  • Include http and dao configuration in sindes
  • Migrate machines to SLC6


  • V 0.2.0 (November) Possibility to modify metric data from the web page. Caching of the data on the client side
  • V 0.3.0 (December) Added summary rows in the expanded table. Creating a new collector for the CMS Production View. Organize the first SSB workshop
  • V 0.4.0 (January) More options to the metric modification, adding new time periods, modfiyfing all entries in one go, defining entries in the future


  • JaroslavaSchovancova's Tentative plan as of 2012-11-12
  • Deploy SUM adaptation to SAM framework API Update-19 to production; implement and deploy whatever changes following changes in SAM API
  • Open tickets:
Ticket ID Short description
#98565 SUM should not truncate the metric_details if it encounters a parsing error
#96735 http://dashb-sum.cern.ch/AllVOs.html fails in IE
#95988 In the SAM test details, the "summary" is not given
#95761 ATLAS-SAM : No result for CA-ALBERTA SE
#95506 sum test for frontier tooks wrong conf
#94699 When no data returned by API hide link to data and announce it to user
#93853 SUM compatibility with IE8/IE9
#92493 Dump latest output of a given test at a given site
#92048 Default values for flavour and metric
#92015 latestresultssmry-sum results -- more verbose column headers
#91972 SUM quality plots -- add availability during the whole period
#91971 SUM tooltips -- add timestamp, shorten decimal part
#91617 Clicking on host name from latest results brings to site availability history instead of test history
#91591 Download image button opens empty page in firefox 10.0
#91523 Link to download image
#91476 SUM latestresultssmry-sum: Select "Service Flavours" does not affect selection in "Instances"
#91392 SUM -- URL is too long when all sites of a group are selected
#90732 In the historical view selections should not be reset when other selections are changed

Detailed input from team members

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2013-01-25 - PabloSaiz
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ArdaGrid All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback