WLCG Http Proxy Discovery Task Force

Contents of this page:

Mandate

• Define the WLCG-wide standards for grid jobs to find out what HTTP proxy or proxies to use

Rationale

• This problem has already been attempted to be solved in many different ways, none of them very completely, for example by CernVM, GlideinWMS, ATLAS site configurator, generator files for ATLAS & CMS, and now the Shoal tool for cloud proxies
• Having a standard would make it much easier for new applications to use local http proxy caches

Members

• Doug Benjamin (ATLAS, added March 2015)
• Barry Blumenfeld (CMS)
• Jakob Blomer (CERN-PH-SFT)
• Brian Bockelman (CMS, OSG)
• Dan Bradley (CMS)
• John DeStefano (ATLAS, OSG)
• Alastair Dewhurst (ATLAS)
• Dave Dykstra (CMS, OSG), Chair
• Ian Gable (ATLAS)
• Costin Grigoras (ALICE, added July 2014)
• John Hover (ATLAS, OSG)
• Stefan Roiser (LHCb)

All members can be contacted at wlcg-ops-coord-tf-httpproxydiscovery@cernNOSPAMPLEASE.ch

Task Overview

Task	Deadline	Progress	Affected VO	Affected Siites	Comment
Implement squid monitoring based on GOCDB/OIM	November 2014	100%	All	All	Done by Alastair Dewhurst
Get critical mass of sites registered in GOCDB/OIM	February 2015	100%	All	All	Done
Implement WPAD service wlcg-wpad.cern.ch/wpad.dat	September 2016	100%	All	All	Done by Dave Dykstra.
Update CVMFS client to robustly support round-robin proxies	December 2014	100%	All	All	Tracked in JIRA ticket Implemented for cvmfs 2.1.20

Meetings

• Notes from 24 April 2013 meeting

Presentations

• CHEP 2016 presentation describing the live WLCG WPAD service and published paper
• CHEP 2019 presentation describing shoal integration into WLCG WPAD and paper draft

References

• This topic was discussed some in the SquidMonitoringTaskForce before it was determined to make a separate task force, particularly in this configuration information system proposal and this WLCG operations coordination meeting presentation.
• Web Proxy Auto Discovery protocol
• Proxy Auto-Config file format
• pacparser library
• pacwget tool
• Frontier client retry strategy

Proposal

• Proposed standards for http proxy discovery

Agreement

The task force agreed to the standards in HttpProxyDiscoveryProposal, and the http://wlcg-wpad.cern.ch/wpad.dat service is now in operation. Strong negative feedback was received from the first site that attempted to implement http://wpad/wpad.dat (that is, CERN) so the task force agreed that clients would first try an additional URL http://grid-wpad/wpad.dat. In addition, a http://wlcg-wpad.fnal.gov/wpad.dat service has been made available as an identical partner to the one in cern.ch.

In summary, WLCG clients SHOULD attempt to read Proxy Auto Config files from the following URLs:

http://grid-wpad/wpad.dat; http://wpad/wpad.dat; http://wlcg-wpad.cern.ch/wpad.dat; http://wlcg-wpad.fnal.gov/wpad.dat