Machine / Job Features Task Force

MJF introduction

Machine/Job features are a means to provide information from a resource provider (batch system, IaaS) to the payload. This information can be static (eg. power of the machine, number of cores, local scratch space) or dynamic (eg. shutdown time of a VM). Use cases cover areas such as

• Provide a shutdown time for a virtual machine
• Provide number of cores available to a payload
• Provide how much work can be executed within a remaining queue
• Provide information on the disk space available for local usage by the job

Task force mandate

The mandate of this task force sets out these goals:

• Check the completeness of the proposal for machine/job features (The specification has now been published as HSF-TN-2016-02)
• Coordinate implementations used in WLCG and an interface for its usage to the VOs
• Provide means to monitor the correctness of the provided information
• Plan and execute the deployment of those implementations at all WLCG resources

The context of this task force are WLCG resource providers in both batch and virtualized environments for compute intensive data processing activities and all LHC VOs interested in using the information provided.

Members

• Manfred Alef (FZK-LCG2)
• Simone Campana
• Simon Fayer (UKI-LT2-IC-HEP)
• Dirk Hufnagel (CMS)
• Jan Just Keijser (NIKHEF-ELPROD)
• Maarten Litmaath
• Miguel Martinez Pedreira (ALICE)
• Andrew McNab (UKI-NORTHGRID-MAN-HEP)
• Victor Mendez (pic)
• Stefan Roiser
• Ulrich Schwickerath (CERN-PROD)

Discussion

An egroup wlcg-ops-coord-tf-machinejobfeatures@cernNOSPAMPLEASE.ch has been setup for technical and organisational discussions of this topic which also contains an archive

Meetings

Meetings will in irregular intervals and announced at the wlcg-ops-coord-tf-machinejobfeatures@cernNOSPAMPLEASE.ch mailing list. Agendas and meeting minutes are available at the WLCG Operations Coordination Indico Category

Presentations

• 2013-06-12, GDB, Deployment Machine/Job Features, S. Roiser
• 2013-05-30, WLCG Operations Coordination, Proposal for Deployment of Machine/Job Features , S. Roiser
• 2012-10-10, GDB, WN Environment Information, U. Schwickerath
• 2012-06-23, GDB, Environmental Information on WN, T. Cass
• 2011-04-06, GDB, WN Environment, T. Cass

Links

• Specification derived from the Hepix virtualisation working group
• Successful test to use meta data / magic IP as a communication channel in IaaS infrstructure
• Proposal for the syntax of the Machine / Job Features data structure and an interface to retrieve the information
• Instructions for setup and administering a key/value store based on couchdb
• Previous Git Repository