Squid Monitoring Task Force

Contents of this page:

Objectives

1. Make a plan for moving squid monitoring from CMS Frontier management to WLCG management
2. Decide how to integrate squid monitoring better with WLCG common operations
3. Produce an architecture for a common squid monitoring system that is configured the same way for all VOs

We should avoid defining how applications discover what proxies to use -- that will be a separate task force.

Members

• Dario Barberis (ATLAS)
• Alexandre Beche (CERN-IT)
• Doug Benjamin (ATLAS)
• Barry Blumenfeld (CMS)
• Simone Campana (CERN-IT & ATLAS)
• David Crooks (NGI_UK, added August 2014)
• Alastair Dewhurst (ATLAS)
• Alessandro Di Girolamo (CERN-IT & ATLAS)
• Dave Dykstra (CMS, OSG), Chair
• Costin Grigoras (ALICE, added July 2014)
• Luis Linares (CMS)
• Stefan Roiser (CERN-IT & LHCb)
• Scott Teige (OSG, left August 2014)
• Andrea Valassi (CERN-IT)

All members can be contacted at wlcg-ops-coord-tf-squidmon@cernNOSPAMPLEASE.ch

Task Overview

#	Task	Deadline	Progress	Affected VO	Affected Siites	Comment
1	Move MRTG to wlcg-squid-monitor	May 2013	100%	All	All	Done by Luis Linares
2	Move awstats to wlcg-squid-monitor	August 2014	100%	All	All	Done by Luis
3	Change frontier-awstats rpm to send to wlcg-squid-monitor	December 2014	100%	All	Launchpad	Done by Dave Dykstra
4	Update frontier-awstats on launchpads and backup proxies	October 2015	100%	All	Launchpad, backup proxies, and stratum 1s	Done by administrators.
5	Implement squid monitoring based on GOCDB/OIM	November 2014	100%	All	All	Done by Alastair Dewhurst
5b	Implement page for accessing removed MRTG plots	October 2019	100%	All	All	Done by Edita #65
6	Implement exceptions list for generating MRTG plots	February 2016	100%	All	some	Done by Alastair #56 #57
7	Implement mapping to CMS site names	October 2014	100%	CMS	All CMS	Done by Luis
8	Add generating CMS MRTG page	November 2019	100%	CMS	All CMS	To be done by Edita #58
9	Add generating ATLAS MRTG page	Novemer 2017	100%	ATLAS	All ATLAS	Done by Michal Svatos
10	Update ATLAS SSB to be based on new MRTG page	November 2017	100%	ATLAS	All ATLAS	Done by Michal Svatos #111
11	Implement generalized failover monitor	July 2014	100%	CMS	All CMS	By Luis, using CMS squid list
12	Convert failover monitor to use GOCDB/OIM squid organizations list	November 2019	100%	All	All	By Michal Svatos, add ATLAS & CVMFS #102
13	Implement SAM/SUM test based on failovers	September 2019	100%	All	All	Done by Edita #236, ATLAS might not want
14	Integrate MonALISA-based squid monitor	November 2014	100%	All	All	Done by Costin Grigoras with help from Luis & Dave
15	Implement exception list to convert monitor view to worker node view	September 2016	100%	All	All	Done by Dave
16	Disable CMS audit comparing MRTG config to SITECONF	November 2019	100%	CMS	All CMS	To be done by Edita

Meetings

• 2012-10-04 Notes
• 2012-11-30 Notes
• 2012-12-14 Notes
• 2013-01-17 Notes
• 2013-02-01 Notes
• 2014-08-28 Notes

Presentations

• Update for 2012-11-01 WLCG Operations Coordination meeting
• Update for 2012-12-06 WLCG Operations Coordination meeting
• Update for 2013-01-24 WLCG Operations Coordination meeting
• Update for 2013-02-07 WLCG Operations Coordination meeting

Agreements

• Move MRTG and awstats squid monitoring from frontier.cern.ch to new pair of virtual machines more closely associated with WLCG
  • The machines will be a contribution from CMS and hosted in the vocms cluster
  • The public alias will be wlcg-squid-monitor.cern.ch
• Squid services will be registered in GOCDB & OIM, as publicly available round-robin DNS aliases if there is more than one squid implementing the same service. Sites may have multiple independent squid services registered. GOCDB & OIM will not distinguish between different purposes/applications for squid services.
• Additional information needed for squid monitoring beyond that stored in GOCDB & OIM will be maintained by per-VO operations personnel in per-VO files on the squid monitoring servers. These will be combined with the GOCDB & OIM information into another file as the input into a MRTG configurator, to allow ATLAS to possibly instead generate that file in the future from AGIS.
• A SAM/SUM test of Squid services will be set up, based on non-response to MRTG monitoring probes a few periods in a row. Those Squids for which SAM tests are required must be declared in GOCDB or OIM.
• An additional SAM/SUM test of Squid services will be set up based on hits to Frontier/CVMFS reverse-proxy Squids not going through the site's Squid services (also known as "failovers"), based on awstats on monitoring server
  • ATLAS operations is not currently planning to use this
• Integrate the ALICE MonALISA-based squid monitor including an optional host-based data collection rpm (see 2014-08-28 meeting notes)

Questions

• Questions we asked and their answers are on the SquidMonitoringTaskForceQuestions page

Proposals

• Squid configuration information proposal

Production documentation

• Layout and Configuration of the Squid Monitoring Machines
• Instructions for registering Squids in GOCDB or OIM