WLCG Multicore Deployment task force meeting, April 1st.

-Attendance: A. Filipcic, A. Lahiff, A. McNab, C. Acosta, C. Wissing, D. Crooks, J. Templon, J. Belleman, M. Alef, S. Skipsey, T. Hartmann, A. Forti, A. Perez-Calero.

-News from CMS: Multicore project discussed during the last CMS Comp & Offline week. Work is ongoing for the multithreaded application, but not yet ready, therefore CMS will test its multicore submission pilots with single core jobs. Tier1 representatives have been contacted to start these tests immediately with a two step plan: functional tests (CMS infrastructure is ready to send multicore pilots to each of the T1 sites and sites are able to allocate resources for them), followed by real scale tests (real workflows are then submitted through multicore pilots).

-News from ATLAS: Testing communication of resource request from the experiment to the sites. The key element is the translation of the request at each CE flavor and for each underlying batch system. Atlas can already pass arguments to the batch system, but it requires to create a new panda queue for each set of parameters one may want making it impractical to use widely. ATLAS is now working on making the queue creation dynamic based on the site characteristics. The current problem though is that most CREAM sites blah scripts don't pass the parameters to the batch system. Only SGE and ARC-CE sites have this capability. This needs to be looked into at WLCG level.

-Nikhef setup for multicore jobs presented by J. Templon: a model is presented where resources are dynamically allocated to single core or multicore jobs, depending on the queue content. This is performed by a custom cron job, using native Torque tools but not via Maui scheduling configuration variables. The amount of draining at any moment is constrained by thresholds on the total number of WNs and cores that are being moved, therefore the impact on the farm occupancy is minimal. Multicore jobs waiting time results depend strongly in the number of cores requested (8) with respect to the cores available in the nodes (32 vs. 8).

-Comments/Questions:

The first comment is whether this could be achieved by using Maui scheduling variables, by means of increasing the priority/share for multicore jobs wrt. single core jobs. This would however affect the fairshare results therefore having a negative impact in single core jobs from other VOs.

The discussion centers then on the use of the slots than remain free while the node is being drained to allocate multicore jobs. No backfill is performed on these slots. The debate is whether this could be profited by the site, by running some other task which could be immediately interrupted, and therefore should not be charged to the VO. The alternative view is that whenever single core and multicore jobs are forced to coexist, some level of draining is mandatory, therefore the existence of empty slots is a direct consequence of the VOs request.

The agreement and solution for this dilemma is that, as long as the impact is minimal, controlled to be of the 1-2% order, it really does not matter, and the sites would probably not include this amount of resources to the VO's bill.

The impact however depends on the relative size of the job request and WN cores, therefore has to be evaluated in a site by site case. The size of the job request could be reduced by the experiments (for example from 8 to 4 cores) in case this is needed on certain sites. ATLAS is already tailoring the number of cores per site. Most run on 8 because it is the preferred size but there are examples of sites requesting different numbers. The executable will use as many as it is told to use.

The general consensus is that there must be an agreement in the size of the job request (number of cores) by ATLAS and CMS, so that the existing multicore slots can be used equally by multicore jobs from both experiments. A general agreement should be enforced (for example, 8 cores as a default), whereas needs from some sites could also be taken into account (going for 4 cores instead), as ATLAS already does.

The main damage comes from the need of constantly draining the nodes to create multicore slots once they are destroyed as single core jobs are put back in. This is to be avoided by forcing multicore slots to be occupied by multicore jobs. A wavelike submission pattern is thus harmful, as it prevents the constant renewal of multicore jobs and imposes draining cycles on the farm. However this situation could be mitigated in sites shared by CMS and ATLAS, as both VOs will be sending multicore jobs, therefore keeping multicore slots alive longer.

-- AntonioPerezCalero - 3 Apr. 2014

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2014-04-03 - AntonioPerezCalero
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback