Squid Monitoring Task Force

Contents of this page:


  1. Make a plan for moving squid monitoring from CMS Frontier management to WLCG management
  2. Decide how to integrate squid monitoring better with WLCG common operations
  3. Produce an architecture for a common squid monitoring system that is configured the same way for all VOs

We should avoid defining how applications discover what proxies to use -- that will be a separate task force.


All members can be contacted at wlcg-ops-coord-tf-squidmon@cernNOSPAMPLEASE.ch

Task Overview

# Task Deadline Progress Affected VO Affected Siites Comment
1 Move MRTG to wlcg-squid-monitor May 2013 100% All All Done by Luis Linares
2 Move awstats to wlcg-squid-monitor August 2014 100% All All Done by Luis
3 Change frontier-awstats rpm to send to wlcg-squid-monitor December 2014 100% All Launchpad Done by Dave Dykstra
4 Update frontier-awstats on launchpads and backup proxies October 2015 100% All Launchpad, backup proxies, and stratum 1s Done by administrators.
5 Implement squid monitoring based on GOCDB/OIM November 2014 100% All All Done by Alastair Dewhurst
5b Implement page for accessing removed MRTG plots October 2019 100% All All Done by Edita #65
6 Implement exceptions list for generating MRTG plots February 2016 100% All some Done by Alastair #56 #57
7 Implement mapping to CMS site names October 2014 100% CMS All CMS Done by Luis
8 Add generating CMS MRTG page November 2019 100% CMS All CMS To be done by Edita #58
9 Add generating ATLAS MRTG page Novemer 2017 100% ATLAS All ATLAS Done by Michal Svatos
10 Update ATLAS SSB to be based on new MRTG page November 2017 100% ATLAS All ATLAS Done by Michal Svatos #111
11 Implement generalized failover monitor July 2014 100% CMS All CMS By Luis, using CMS squid list
12 Convert failover monitor to use GOCDB/OIM squid organizations list November 2019 100% All All By Michal Svatos, add ATLAS & CVMFS #102
13 Implement SAM/SUM test based on failovers September 2019 100% All All Done by Edita #236, ATLAS might not want
14 Integrate MonALISA-based squid monitor November 2014 100% All All Done by Costin Grigoras with help from Luis & Dave
15 Implement exception list to convert monitor view to worker node view September 2016 100% All All Done by Dave
16 Disable CMS audit comparing MRTG config to SITECONF November 2019 100% CMS All CMS To be done by Edita




  • Move MRTG and awstats squid monitoring from frontier.cern.ch to new pair of virtual machines more closely associated with WLCG
    • The machines will be a contribution from CMS and hosted in the vocms cluster
    • The public alias will be wlcg-squid-monitor.cern.ch
  • Squid services will be registered in GOCDB & OIM, as publicly available round-robin DNS aliases if there is more than one squid implementing the same service. Sites may have multiple independent squid services registered. GOCDB & OIM will not distinguish between different purposes/applications for squid services.
  • Additional information needed for squid monitoring beyond that stored in GOCDB & OIM will be maintained by per-VO operations personnel in per-VO files on the squid monitoring servers. These will be combined with the GOCDB & OIM information into another file as the input into a MRTG configurator, to allow ATLAS to possibly instead generate that file in the future from AGIS.
  • A SAM/SUM test of Squid services will be set up, based on non-response to MRTG monitoring probes a few periods in a row. Those Squids for which SAM tests are required must be declared in GOCDB or OIM.
  • An additional SAM/SUM test of Squid services will be set up based on hits to Frontier/CVMFS reverse-proxy Squids not going through the site's Squid services (also known as "failovers"), based on awstats on monitoring server
    • ATLAS operations is not currently planning to use this
  • Integrate the ALICE MonALISA-based squid monitor including an optional host-based data collection rpm (see 2014-08-28 meeting notes)



Production documentation

Edit | Attach | Watch | Print version | History: r56 < r55 < r54 < r53 < r52 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r56 - 2019-11-20 - DaveDykstra
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback