This page is a link to various resources about computing during CSA14, with a focus on the needs of analysis users.

General News



We strongly encourage that all analysis for CSA14 be done with CRAB3.

  • There is a tutorial session for CRAB3 on Friday, July 4, 2014 at CERN.
  • The workbook for the tutorial can be found here.
  • Help for users with CRAB3 as well as general computing problems in CMS analysis can be obtained by e-mailing the Computing Tools Hypernews.

CRAB3 Do's and Don'ts

  • Please use CRAB3, not CRAB2, for CSA14 analysis-related activities
  • Please do not use "config.Data.ignoreLocality = True" in your crab.cfg, at least until encouraged to do so by Operations. While many signal samples may have limited availability, running large numbers of analysis jobs at sites different from where the data are located is something we will want to do only later in the challenge. See the note below.

Overview of Goals and Milestones

MiniAOD Production

There will be two scenarios available at the start of the challenge, by campaign name these are PU_S14 and PU20bx25. The PU20bx25 scenario will be made into MiniAOD by central operations and one by users with CRAB3. PPD will shortly identify the centrally produced campaign and inject the requests. As all AOD samples have been distributed with the new dynamic data placement script to some Tier-2s, the proposal is to execute the AOD to MiniAOD on the hosting Tier-2 centers. At the moment operations would need to configure the destination site by hand because the AODs were distributed automatically based on available space. Computing operations agreed to provide CPU/timing performance statistics of the MiniAOD workflows once we run the at large scale centrally.

The PU_S14 scenario will be created with CRAB3. The background samples, which represent half the events but only ~50 of the ~ 400 datasets, will be converted to MiniAOD with CRAB3 jobs and elevated to official datasets using StoreResults. The signal samples will be also converted to MiniAOD using CRAB3 and published into the local scope DBS3 by CRAB3. There is some organization needed from Physics and PPD to establish the official user produced MiniAOD samples and put the dataset names on an official twiki. Users will then we able to access these samples either through direct submission to the original hosting site, or access through AAA with data locality set to false.

Accessing data through AAA

Computing expects that AAA will be used continuously for data access to analysis users throughout the entire challenge. Computing expects that this will represent in CMS about 20% of the total access, the bulk being still local. For testing purposes during CSA we identify the following 3 periods.

  1. During the first 2 weeks most AAA access will be to fall-back and to MiniAOD signal samples prepared through CRAB3, but not run through StoreResults;
  2. During the middle 4 weeks of the challenge we will aim to reach at most 20% data federation access. It is possible that users will be adviced to submit some fraction of jobs with no data locality and measure the performance and percentage coming from AAA, and measure the rate;
  3. The final 2 weeks of the challenge will be predominantly remote access through AAA as defined in the CSA14 goals document. Users should be encouraged to ignore data locality for the majority of jobs during this final period.

CRAB3 load

The HammerCloud test will be used during the challenge to maintain the desired CRAB3 load. It can be used in either local or data federation mode, and will be configured with the MiniAOD production job using at least CMSSW_7_0_6. No change in priority is expected to be needed for HammerCloud, because if there are enough users to push out the HammerCloud jobs we have reached a load that HammerCloud is unnecessary.

-- JamesLetts - 01 Jul 2014

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2014-07-14 - JamesLetts
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback