The useful twiki links are listed below:

https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookCRAB3Tutorial

https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3AdvancedTutorial

https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3ConfigurationFile

Grid UI on ntugrid3 or ntugrid5

source /cvmfs/cms.cern.ch/cmsset_default.sh
cmsrel CMSSW_7_3_5_patch2

Sets the crab environment.

source /cvmfs/cms.cern.ch/crab3/crab.sh
which crab
crab --version

Make sure certificate proxy is setup properly.

voms-proxy-init --voms cms --valid 168:00
voms-proxy-info --all

crab checkwrite --site=T3_TW_NTU_HEP

The example output:

[yuanchao@ntugrid5 src]$ crab checkwrite --site=T3_TW_NTU_HEP
Will check write permission in the default location /store/user/<username>
Retrieving DN from proxy...
DN is: /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=yuanchao/CN=596728/CN=Yuan Chao
Retrieving username from SiteDB...
Username is: yuanchao
Validating LFN /store/user/yuanchao...
LFN /store/user/yuanchao is valid.
Will use `lcg-cp`, `lcg-del` commands for checking write permissions
Will check write permission in /store/user/yuanchao on site T3_TW_NTU_HEP

Attempting to create (dummy) directory crab3checkwrite_20151127_195741 and copy (dummy) file crab3checkwrite_20151127_195741.tmp to /store/user/yuanchao

Executing command: lcg-cp -v -b -D srmv2 --connect-timeout 180 --checksum-type ADLER32 /wk_cms/yuanchao/cms/CMSSW_7_3_5_patch2/src/crab3checkwrite_20151127_195741.tmp 'srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/crab3checkwrite_20151127_195741.tmp'
Please wait...

Successfully created directory crab3checkwrite_20151127_195741 and copied file crab3checkwrite_20151127_195741.tmp to /store/user/yuanchao

Attempting to delete file srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/crab3checkwrite_20151127_195741.tmp

Executing command: lcg-del --connect-timeout 180 -b -l -D srmv2 'srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/crab3checkwrite_20151127_195741.tmp'
Please wait...

Successfully deleted file srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/crab3checkwrite_20151127_195741.tmp

Attempting to delete directory srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/

Executing command: lcg-del -d --connect-timeout 180 -b -l -D srmv2 'srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/'
Please wait...

Successfully deleted directory srm://ntugrid4.phys.ntu.edu.tw:8446/srm/managerv2?SFN=/dpm/phys.ntu.edu.tw/home/cms/store/user/yuanchao/crab3checkwrite_20151127_195741/

Checkwrite Result:
Success: Able to write in /store/user/yuanchao on site T3_TW_NTU_HEP

Better check the write permission on T3_TW_NTU_HEP and T2_TW_NCHC.

crab submit -c crabConfig_tutorial_MC_analysis.py

crab status --long

crab report

crab getlog

Running LHE to GEN-SIM MC production jobs:

Do cmsRun locally on 1000 events before submitting it with CRAB, and use the Timing service of CMSSW to get a time per event estimation. Publication should be enabled in your configuration (after test runs).

Adding Timing service to the PSet.

##### These are the lines you can add to your pset get the estimates in the FrameworkJobReport file
process.Timing = cms.Service("Timing",
    summaryOnly = cms.untracked.bool(True)
)

##### For your information CRAB3 also add the following two lines in addition to the previous three:
#process.CPU = cms.Service("CPU")

#process.SimpleMemoryCheck = cms.Service("SimpleMemoryCheck")

cmsRun -j FrameworkJobReport.xml -p pset_MC_generation_LHE.py

At the tail of the job report: on average 12.9 sec per event.

TimeReport> Time report complete in 12912.2 seconds
 Time Summary: 
 - Min event:   0.00122595
 - Max event:   152.451
 - Avg event:   12.9122
 - Total job:   12912.2
 CPU Summary: 
 - Min event:   0.001
 - Max event:   81.2956
 - Avg event:   9.57881
 - Total job:   9639.99
 - Total event: 9578.81

CRAB configuration file.

from CRABClient.UserUtilities import config
config = config()

config.General.requestName = 'CRAB3_Advanced_Tutorial_May2015_Exercise5A'

config.JobType.pluginName = 'PrivateMC'
config.JobType.generator = 'lhe'
config.JobType.psetName = 'pset_MC_generation_LHE.py'
config.JobType.inputFiles = ['dynlo.lhe']

config.Data.outputPrimaryDataset = 'TutorialMay2015Exercise5A'
config.Data.splitting = 'EventBased'
JOB_WALLTIME = 8*3600
TIME_PER_EVENT = 0.25
config.Data.unitsPerJob = int(JOB_WALLTIME / TIME_PER_EVENT)
NJOBS = 10
config.Data.totalUnits = config.Data.unitsPerJob * NJOBS
config.Data.publication = True
config.Data.outputDatasetTag = 'MC_generation_LHE'

config.Site.storageSite = <storage-site>

Get the PFN for the LHEInputSource

/afs/cern.ch/user/b/belforte/public/mybin/lfn2pfn.sh <site> [<LFN>]

Using a working URL as the pool source input for avoid the input sandbox limit of 100MB.

process.source = cms.Source("LHESource",
    fileNames = cms.untracked.vstring('root://ntugrid6.phys.ntu.edu.tw//cms/store/user/yuanchao/mc/TTbar_012jets_MG5LO/tt012j_lo_100k-1.lhe'),

config.Site.whitelist = ['T2_TW_NCHC'] or config.Site.whitelist = ['T3_TW_NTU_HEP']

For storage element of T2_TW_NCHC, the endpoint URL for xrootd is:

root://ntugrid6.phys.ntu.edu.tw//cms/store/user/[YOUR_USER_NAME]/

-- YuanChao - 2015-11-27

Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r9 - 2016-08-19 - YuanChao
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback