WN Working Group Minutes 27th Setember 2007



Items Discussed

Job Attributes to be considered.

Quick review of the attributes to be considered.
  • Operating System
  • 32bit/64bt
  • Local disk space for job.
  • Memory constraints for job.

Discussion that this group will only ever consider those attributes that can be used by the batch systems as they stand. Finding particular WNs that are not described by the the batch system internals is a much harder as yet unsolved problem. If there are shortfalls they will just be identified.

For both torque and LSF jobs can be submitted requesting amounts of diskspace, os, bitness and/or memory.

Job Attribute Passing

As for passing values to the batch system it is reliant on being able to match these values in the GlueSchema. Such values are then flattened via the mechanisms described within Francesco's HEPiX talks.

Local Disk Space

The publishing of disk space available to the job presents a current problem in that this value is not published by the Glue Schema. As a point of history it did used to be published in the old EDG schema. There was some discussion about what space should be published? It is the space the job can use but how to find the space. This is the . vs $TMPDIR vs $EGE_RB_SCRATCH or whatever it called problem. When it comes the eventual testing of match making then within a controlled environment an extra value to Glue can be published representing available space. GlueSchema 2.0 developments will be monitored to check that publishing a useful available local diskspace is considered.

Population of GlueSchema

The current population of the schema is done by where each CE node publishes a number of GlueCEs plus a single GlueCluster and GlueSubCluster. This is not how the schema was intended to be populated in particular the various GlueSubClusters published by a site now represent overlapping nodes where a GlueSubCluster was meant to be unique set of batch workers with each batch worker sitting in exactly one GlueSubCluster. This will be addressed.

In order to address the publishing the GlueCE, GlueCluster and GlueSubCluster it was suggested to split the now single EGEE node type glite-CE into 2 or perhaps 3 node types.

  • CE-ClusterPublisher Node - This would publish a sites GlueClusters and GlueSubClusters.
  • CE-GateKeeper - This would have the the actual service accepting jobs and passing them to the batch system, e.g globus-gatekeeper or Cream CE.
  • CE-GlueCEPublisher - This would publish the various GlueCE objects.

These names have just made up by me now. It is very conceivable that for small sites these would all be on the same physical node but the option should be there to allow splitting. While splitting the GateKeeper and GLueCEPublisher into two nodes types potentially has some advantages there would be some technical challenges to overcome , in particular the GlueCEPublisher would have to reflect the state of GateKeeper node and this information may not be available anywhere other than locally, e.g is the gatekeeper service actually running?

Whatever the eventual publishing would end up with a single GlueCluster and multiple GlueSubCluster for each class of machines types within a GlueCluster.

Test Rig

A quick demonstration will be set up by Steve of such a configuration within the PPS. Two GlueSubClusters with a single differing batch worker in each. The impact of deploying such a system with the current lcg-CEs where LRMS passing does not happen should be considered.

Software Tags

Steve Burke highlighted a problem of not associating a particular GlueSubCluster with a particular CEnode. In particular when a software submission job is submitted to a queue matched with some GlueSubCluster then the software tags must be added to the correct GlueSubClusters. Possible options:
  • The installation job is able to determine all the GlueSubCluster that it matched to and the tag should be added to them.
  • The installation job is able to determine which GlueSubCluster the particular batch worker is in and publish the tag just to one GlueSubCluster.
  • Others....
There are some obvious undesirables with the above two. The first requires knowledge of the underlying fabric, e.g which GlueSubClusters use the same underlying software area. The second is just a pain since it creates potentially duplication of work for multiple installations of software on what are identical GlueSubClusters as far as software installation is concerned. It depends a lot on how many GlueSubClusters a site might typically have. This area needs some more thinking, to be addressed later...

Discussion after the Meeting.

There was some discussion after the meeting.

How Many GlueSubClusters

It is quite clear that two SubClusters are needed to represent say SL3 or SL4 nodes. The same for bitness.

SteveT Don't we just publish a SubCluster for every configuration of disk size and memory size?

Ulrich There is no point, only the minimum of disk space and memory size needs to be advertised. If users select >> than the minimum memory size they will still match. The actuall correct memory will still be requested when the job is passed to the LRMS.

SteveT Do users and monitoring not need to or want to know the actual jobs running or queuing in a SubCluster for the nodes they are interested in?

Needs more discussion....

Job Classes from Experiments

Roberto highlighted that LHCb could and probably should give an outline of the job classes they have. e.g heavy CPU monte carlo vs heavy I/O reconstruction. Can the WN group consider the efficient execution of such job types. e.g mixing CPU ones and I/O ones on same nodes has some potential benefits to all. This is a good idea and should be followed up.

-- SteveTraylen - 27 Sep 2007

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2007-09-28 - SteveTraylen
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback