Difference: WorkBookPickEvents (1 vs. 23)

Revision 232019-09-04 - StefanoBelforte

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5
Line: 25 to 25
 

How to setup the environment to run edmPickEvents.py

Added:
>
>
Note that edmPickEvents in CMSSW versions before 8_3_0 is broken. Please use recent CMSSW.
 The script edmPickEvents.py is part of the PhysicsTools/Utilities package. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later and the version that uses CRAB3 instead of CRAB2 is integrated into CMSSW_5_3_29 and later (CMSSW_5_3_X release cycle) and CMSSW_7_4_7 and later.

Revision 222017-08-03 - PatriziaBarria

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5

Revision 212015-10-13 - DinkoFerencek

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5
Line: 92 to 92
 output or run edmPickEvents.py with the --runInteractive flag which will run it for you (warning, this can take a long time).
Changed:
<
<
This will created a ROOT file called pickevents.root in the same directory you executed the command from. Also note that
>
>
This will create a ROOT file called pickevents.root in the same directory you executed the command from. Also note that
 edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun.

Revision 202015-09-14 - DinkoFerencek

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5
Line: 25 to 25
 

How to setup the environment to run edmPickEvents.py

Changed:
<
<
The script edmPickEvents.py is part of the PhysicsTools/Utilities package. The version that uses DAS into CMSSW_7_4_0 and later. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later and the version that uses CRAB3 instead of CRAB2 is integrated into CMSSW_5_3_29 and later (CMSSW_5_3_X release cycle) and CMSSW_7_4_7 and later.
>
>
The script edmPickEvents.py is part of the PhysicsTools/Utilities package. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later and the version that uses CRAB3 instead of CRAB2 is integrated into CMSSW_5_3_29 and later (CMSSW_5_3_X release cycle) and CMSSW_7_4_7 and later.
 
ssh lxplus.cern.ch

Revision 192015-07-14 - FreyaBlekman

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5
Line: 98 to 98
 

Run edmPickEvents.py with CRAB

Added:
>
>
BEWARE AS THE CRAB CONFIG THAT edmPickEvents puts out might still be crab2 if you use releases before CMSSW_7_4_7!!!
 If you are running over a large number of event, if you just don't want to wait for a long job to finish, you can use edmPickEvents.py to setup a CRAB job for you.

Revision 182015-07-12 - DinkoFerencek

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events (Interactive and CRAB3)

Complete: 5
Line: 26 to 26
 

How to setup the environment to run edmPickEvents.py

The script edmPickEvents.py

Changed:
<
<
is part of the PhysicsTools/Utilities package. The version that uses DAS into CMSSW_7_4_0 and later.
>
>
is part of the PhysicsTools/Utilities package. The version that uses DAS into CMSSW_7_4_0 and later. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later and the version that uses CRAB3 instead of CRAB2 is integrated into CMSSW_5_3_29 and later (CMSSW_5_3_X release cycle) and CMSSW_7_4_7 and later.
 
Changed:
<
<
ssh lxplus.cern.ch
cmsrel CMSSW_7_4_0
cd CMSSW_7_4_0/src/
>
>
ssh lxplus.cern.ch
cmsrel CMSSW_7_4_7
cd CMSSW_7_4_7/src/
 cmsenv
Deleted:
<
<
git cms-merge-topic -u ramankhurana:edmPickEventsCrab3Updates_from-CMSSW_7_4_0 scram b
 

How to Run edmPickEvents.py

Line: 168 to 167
 

Review Status

Editor/Reviewer and date Comments
Deleted:
<
<
RamanKhurana - 8 July 2015 Added instructions for CMSSW_7_4_X compatible with CRAB3
 
DinkoFerencek - 22 June 2015 No longer necessary to merge a branch from the cms-analysis-tools GitHub area
Added:
>
>
RamanKhurana - 8 July 2015 Added instructions for CMSSW_7_4_X compatible with CRAB3
DinkoFerencek - 12 July 2015 Updated instructions indicating that the version compatible with CRAB3 is now integrated in CMSSW
  -- SudhirMalik - 09-Sep-2010

Revision 172015-07-08 - RamanKhurana

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Changed:
<
<

9.6 How to Pick Events

>
>

9.6 How to Pick Events (Interactive and CRAB3)

 Complete: 5
Detailed Review status
Line: 12 to 12
 The goal of this page is to help the users get a copy of a subset of events from a dataset. This utility is a python script called edmPickEvents.py and can run an interactive job for a small
Changed:
<
<
number of events, or can set up a CRAB job.
>
>
number of events (now user can use additional command-line option to pick more than 20 events when running interactively), or can set up a CRAB job.
  As a recent improvement, this script will allow you access events from
Changed:
<
<
data that is not local by using CRAB.
>
>
data that is not local by using xrootd and CRAB3.
  If you plan on making very large "skims' (e.g., all W candidates), please consider collaborating with others as to minimize the number of
Line: 25 to 25
 

How to setup the environment to run edmPickEvents.py

Changed:
<
<
The script edmPickEvents.py is part of the PhysicsTools/Utilities package. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later.
>
>
The script edmPickEvents.py is part of the PhysicsTools/Utilities package. The version that uses DAS into CMSSW_7_4_0 and later.
 
ssh lxplus.cern.ch
Changed:
<
<
cmsrel CMSSW_5_3_18 cd CMSSW_5_3_18/src/
>
>
cmsrel CMSSW_7_4_0 cd CMSSW_7_4_0/src/
 cmsenv
Added:
>
>
git cms-merge-topic -u ramankhurana:edmPickEventsCrab3Updates_from-CMSSW_7_4_0 scram b
 

How to Run edmPickEvents.py

The script edmPickEvents.py can be run interactively or via a CRAB

Changed:
<
<
job. If you run interactively, before you run it, make sure you have the dataset that you want to pick events from, available locally. If you have few events to pick, run it interactively. In case you do not have the dataset available locally or you have a lot of events to pick, submit a CRAB job.
>
>
job. If you have few events to pick, run it interactively. Now you can pick more than 20 events interactively using the new command line option :
edmPickEvents.py --maxEventsInteractive=30 "/MET/Run2015A-PromptReco-v1/RECO"  events.txt

In case you have a lot of events to pick, submit a CRAB job. This version of script is compatible with CRAB3 and produces a crabConfig which can be directly used by the user to submit the jobs. One can change some of the crab parameters (if needed, like site to store the output files, num of lumi etc).

 

Run edmPickEvents.py Interactively

To run the script interactively do the following, for example


Changed:
<
<
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" 176163:41:69046624
>
>
edmPickEvents.py "/MET/Run2015A-PromptReco-v1/RECO" 248038:25:12714964
 

where

Changed:
<
<
  • /Photon/Run2011B-PromptReco-v1/AOD = is the dataset you want to pick
>
>
*/MET/Run2015A-PromptReco-v1/RECO = is the dataset you want to pick
  events from. You can only do one per job.
Changed:
<
<
  • 176163:41:69046624 is one event
>
>
  • 248038:25:12714964 is one event
  you want to pick. The syntax is Run:Lumi:Event
Changed:
<
<
(one could also put more than one event by separating the different events with a comma: Run1:Lumi1:Event1, Run2:Lumi2:Event2. Be careful, for me it seems to give only the first event)
>
>
(one could also put more than one event by separating the different events with a comma: Run1:Lumi1:Event1, Run2:Lumi2:Event2.
  or


Changed:
<
<
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" events.txt
>
>
edmPickEvents.py "/MET/Run2015A-PromptReco-v1/RECO" events.txt
 

where

Line: 76 to 83
 
edmCopyPickMerge outputFile=pickevents.root \
Changed:
<
<
eventsToProcess=176163:69046624 inputFiles=/store/data/Run2011B/Photon/AOD/PromptReco-v1/000/176/163/24DD7EC1-FCDE-E011-AE6E-001D09F23A20.root
>
>
eventsToProcess=248038:12714964 inputFiles=/store/data/Run2015A/MET/RECO/PromptReco-v1/000/248/038/00000/546E74FC-5D14-E511-9DCE-02163E0145BA.root
 

In this case, the user can either paste the above edmCopyPickMerge

Line: 90 to 99
 

Run edmPickEvents.py with CRAB

Changed:
<
<
If you are running over a large number of event, if you know the data set is not available locally, or you just don't want to wait for a
>
>
If you are running over a large number of event, if you just don't want to wait for a
 long job to finish, you can use edmPickEvents.py to setup a CRAB job for you.
Line: 103 to 111
 
  • Then run the script as follows ( In this case we are running the two events via a CRAB job) :


Changed:
<
<
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" events.txt --crab
>
>
edmPickEvents.py "/MET/Run2015A-PromptReco-v1/RECO" events.txt --crab
 
Deleted:
<
<
(Note that for a large number of events , you do not need to specify the --crab option)
 When you run this it gives the following screen output
Changed:
<
<
Please visit https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideCrab
>
>
Please visit CRAB twiki for instructions on how to setup environment for CRAB: https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideCrab
 
Changed:
<
<
Setup your environment for CRAB. Then edit pickevents_crab.config to make any desired changed. The run:
>
>
Setup your environment for CRAB and edit pickevents_crab.py to make any desired changed. Then run:
 
Changed:
<
<
crab -create -cfg pickevents_crab.config crab -submit
>
>
crab submit -c pickevents_crab.py
 
Line: 126 to 132
  pickEvents_crab.config. Most people should find the defaults sufficient.
Changed:
<
<
  • Then create a Crab job
crab -create -cfg pickevents_crab.config
>
>
  • Then submit a Crab job
 
Deleted:
<
<
  • Submit a Crab job
 

Changed:
<
<
crab -submit
>
>
crab submit -c pickevents_crab.py
 
Added:
>
>
  • It will create a CRAB working directory according to current time. YYMMDD_HHMMSS.
 
  • You can check the status of your job via


Changed:
<
<
crab -status
>
>
crab status
 

Once the CRAB job finishes successfully, you can get the output ROOT file in the craboutput /res directory:

Line: 164 to 168
 

Review Status

Editor/Reviewer and date Comments
Added:
>
>
RamanKhurana - 8 July 2015 Added instructions for CMSSW_7_4_X compatible with CRAB3
 
DinkoFerencek - 22 June 2015 No longer necessary to merge a branch from the cms-analysis-tools GitHub area

-- SudhirMalik - 09-Sep-2010

Revision 162015-06-22 - DinkoFerencek

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 26 to 26
 

How to setup the environment to run edmPickEvents.py

The script edmPickEvents.py

Changed:
<
<
is part of the PhysicsTools/Utilities package.
>
>
is part of the PhysicsTools/Utilities package. The version that uses DAS instead of the outphased DBS2 is integrated into CMSSW_5_3_18 and later.
 
Changed:
<
<
Warning, important The edmPickEvents.py versions currently coming with releases do not work. Get an updated version with

ssh lxplus5.cern.ch
cmsrel CMSSW_5_3_17
cd CMSSW_5_3_17/src/
>
>
ssh lxplus.cern.ch
cmsrel CMSSW_5_3_18
cd CMSSW_5_3_18/src/
 cmsenv
Deleted:
<
<
git cms-merge-topic cms-analysis-tools:5_3_17-updateEdmPickEvents scram b -j 20 rehash
 
Deleted:
<
<
This version uses DAS instead of the outphased DBS2. Integration into nominal releases is requested.
 

How to Run edmPickEvents.py

The script edmPickEvents.py can be run interactively or via a CRAB

Line: 168 to 160
  (In cmsRun, local files must be noted with the prefix file:.)
Added:
>
>

Review Status

Editor/Reviewer and date Comments
DinkoFerencek - 22 June 2015 No longer necessary to merge a branch from the cms-analysis-tools GitHub area
 -- SudhirMalik - 09-Sep-2010

Revision 152014-04-17 - VolkerAdler

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 23 to 23
 

Contents

Changed:
<
<

How to setup the environment to run edmPickEvents.py The

The script
>
>

How to setup the environment to run edmPickEvents.py

 
Changed:
<
<
edmPickEvents.py is part of the CMS.PhysicsTools/Utilities package with tag V08-01-02 and works for CMSSW 36X or higher. For releases earlier than 3.8.0_patch1, you also need FWCore/PythonUtilities, tag V01-02-00)
>
>
The script edmPickEvents.py is part of the PhysicsTools/Utilities package.
 
Added:
>
>
Warning, important The edmPickEvents.py versions currently coming with releases do not work. Get an updated version with
 
Changed:
<
<
To setup a CMSSW release look HERE and then checkout the tag as follows:

cvs co -r V08-01-02 CMS.PhysicsTools/Utilities

Please remember to do scram b and rehash after checking out the package.

IMPORTANT: There is a bug in CMSSW that is fixed in the 3.8 release series. It may not be possible to strip or merge large numbers of events in an earlier release. One can use a 3.8 release on 3.6 or 3.7 data.

>
>
ssh lxplus5.cern.ch
cmsrel CMSSW_5_3_17
cd CMSSW_5_3_17/src/
cmsenv
git cms-merge-topic cms-analysis-tools:5_3_17-updateEdmPickEvents
scram b -j 20
rehash
 
Added:
>
>
This version uses DAS instead of the outphased DBS2. Integration into nominal releases is requested.
 

How to Run edmPickEvents.py

Line: 101 to 98
 

Run edmPickEvents.py with CRAB

Deleted:
<
<
2014 Note: Due to recent changes in CRAB, a new version of edmPickEvents.py is needed:

 If you are running over a large number of event, if you know the data set is not available locally, or you just don't want to wait for a long job to finish, you can use edmPickEvents.py to setup a CRAB

Revision 142014-03-26 - SethZenz

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 101 to 101
 

Run edmPickEvents.py with CRAB

Added:
>
>
2014 Note: Due to recent changes in CRAB, a new version of edmPickEvents.py is needed:

 If you are running over a large number of event, if you know the data set is not available locally, or you just don't want to wait for a long job to finish, you can use edmPickEvents.py to setup a CRAB

Revision 132013-05-08 - ChristopherGeorge

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 65 to 65
 where

  • /Photon/Run2011B-PromptReco-v1/AOD = is the dataset you want to pick
Changed:
<
<
events from
>
>
events from. You can only do one per job.
 
  • 176163:41:69046624 is one event you want to pick. The syntax is Run:Lumi:Event
Changed:
<
<
(one could also put more than one event by separating the different events with a coma: Run1:Lumi1:Event1, Run2:Lumi2:Event2)
>
>
(one could also put more than one event by separating the different events with a comma: Run1:Lumi1:Event1, Run2:Lumi2:Event2. Be careful, for me it seems to give only the first event)
  or

Revision 122011-12-22 - AlexandreLeonard

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 27 to 27
 The script

edmPickEvents.py

Changed:
<
<
is part of the PhysicsTools/Utilities package with tag
>
>
is part of the CMS.PhysicsTools/Utilities package with tag
 V08-01-02 and works for CMSSW 36X or higher. For releases earlier than 3.8.0_patch1, you also need FWCore/PythonUtilities, tag V01-02-00)
Line: 35 to 35
  To setup a CMSSW release look HERE and then checkout the tag as follows:
Changed:
<
<
cvs co -r V08-01-02 PhysicsTools/Utilities
>
>
cvs co -r V08-01-02 CMS.PhysicsTools/Utilities
  Please remember to do scram b and rehash after checking out the package.
Line: 59 to 59
 To run the script interactively do the following, for example


Changed:
<
<
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704
>
>
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" 176163:41:69046624
 

where

Changed:
<
<
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick
>
>
  • /Photon/Run2011B-PromptReco-v1/AOD = is the dataset you want to pick
  events from
Changed:
<
<
  • 140160:98:82878561 and 140160:101:85437704 are the two events
>
>
  • 176163:41:69046624 is one event
  you want to pick. The syntax is Run:Lumi:Event
Added:
>
>
(one could also put more than one event by separating the different events with a coma: Run1:Lumi1:Event1, Run2:Lumi2:Event2)
  or


Changed:
<
<
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" events.txt
>
>
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" events.txt
 

where

Changed:
<
<
  • events.txt is a text file that contains 140160:98:82878561 and 140160:101:85437704 on two lines
>
>
  • events.txt is a text file that contains = 176163:41:69046624 = ( and others if desired, but one by line )
  and the screen output would look like this:

edmCopyPickMerge outputFile=pickevents.root \
Changed:
<
<
eventsToProcess=140160:82878561,140160:85437704 inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
>
>
eventsToProcess=176163:69046624 inputFiles=/store/data/Run2011B/Photon/AOD/PromptReco-v1/000/176/163/24DD7EC1-FCDE-E011-AE6E-001D09F23A20.root
 

In this case, the user can either paste the above edmCopyPickMerge

Line: 113 to 114
 
  • Then run the script as follows ( In this case we are running the two events via a CRAB job) :


Changed:
<
<
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" events.txt --crab
>
>
edmPickEvents.py "/Photon/Run2011B-PromptReco-v1/AOD" events.txt --crab
 

(Note that for a large number of events , you do not need to specify the --crab option)

Revision 112010-11-03 - CharlesPlager

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

9.6 How to Pick Events

Complete: 5
Line: 28 to 28
  edmPickEvents.py is part of the PhysicsTools/Utilities package with tag
Changed:
<
<
V08-01-01 and works for CMSSW 36X or higher. For releases earlier
>
>
V08-01-02 and works for CMSSW 36X or higher. For releases earlier
 than 3.8.0_patch1, you also need FWCore/PythonUtilities, tag V01-02-00)

To setup a CMSSW release look HERE and then checkout the tag as follows:

Changed:
<
<
cvs co -r V08-01-01 PhysicsTools/Utilities
>
>
cvs co -r V08-01-02 PhysicsTools/Utilities
  Please remember to do scram b and rehash after checking out the package.

Revision 102010-09-13 - SudhirMalik

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Changed:
<
<

9.6 How to copy, pick and merge events

>
>

9.6 How to Pick Events

 Complete: 5
Detailed Review status

Revision 92010-09-13 - SudhirMalik

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Added:
>
>

9.6 How to copy, pick and merge events

 Complete: 5
Detailed Review status

Revision 82010-09-10 - CharlesPlager

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Complete: 5
Line: 27 to 27
  edmPickEvents.py is part of the PhysicsTools/Utilities package with tag
Changed:
<
<
V08-01-00 and works for CMSSW 36X or higher. For releases earlier
>
>
V08-01-01 and works for CMSSW 36X or higher. For releases earlier
 than 3.8.0_patch1, you also need FWCore/PythonUtilities, tag V01-02-00)

To setup a CMSSW release look HERE and then checkout the tag as follows:

Changed:
<
<
cvs co -r V08-01-00 PhysicsTools/Utilities
>
>
cvs co -r V08-01-01 PhysicsTools/Utilities
 
Changed:
<
<
REMEMBER to do scram b and rehash after checking out the package.
>
>
Please remember to do scram b and rehash after checking out the package.

IMPORTANT: There is a bug in CMSSW that is fixed in the 3.8 release series. It may not be possible to strip or merge large numbers of events in an earlier release. One can use a 3.8 release on 3.6 or 3.7 data.

 

How to Run edmPickEvents.py

Line: 155 to 160
 To merge the output of CRAB, you can follow the easy instructions here:


Changed:
<
<
ls -1 pick*root > myFiles.txt edmCopyPickMerge inputFiles_load=myFiles.txt output=pickevents_merged.root
>
>
ls -1 pick*root | perl -ne 'print "file:$_"' > myFiles.txt edmCopyPickMerge inputFiles_load=myFiles.txt outputFile=pickevents_merged.root maxSize=1000000
 
Added:
>
>
where maxSize=1000000 means 1 million Kb (or 1 Gb).

(In cmsRun, local files must be noted with the prefix file:.)

 -- SudhirMalik - 09-Sep-2010

Revision 72010-09-10 - CharlesPlager

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Complete: 5
Line: 8 to 8
 

Goals of this page:

Changed:
<
<
The goal of this page is to help the users get a copy of asubset of events from a dataset. This utility is a python script called edmPickEvents.py and and can run an interactive job for a small
>
>
The goal of this page is to help the users get a copy of a subset of events from a dataset. This utility is a python script called edmPickEvents.py and can run an interactive job for a small
 number of events, or can set up a CRAB job.

As a recent improvement, this script will allow you access events from

Line: 57 to 58
  where
Changed:
<
<
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from
  • 140160:98:82878561 and 140160:101:85437704 are the two events you want to pick. The syntax is Run:Lumi:Event
>
>
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from

  • 140160:98:82878561 and 140160:101:85437704 are the two events you want to pick. The syntax is Run:Lumi:Event

or

edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" events.txt

where

  • events.txt is a text file that contains 140160:98:82878561 and 140160:101:85437704 on two lines
  and the screen output would look like this:
Line: 92 to 107
 
  • Then run the script as follows ( In this case we are running the two events via a CRAB job) :


Changed:
<
<
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab
>
>
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" events.txt --crab
 

(Note that for a large number of events , you do not need to specify the --crab option)

Revision 62010-09-10 - CharlesPlager

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Complete: 5
Line: 8 to 8
 

Goals of this page:

Changed:
<
<
The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. This utility is a python script
>
>
The goal of this page is to help the users get a copy of asubset of events from a dataset. This utility is a python script
 called edmPickEvents.py and and can run an interactive job for a small number of events, or can set up a CRAB job.
Line: 23 to 21
 

Contents

Changed:
<
<

How to setup the environment to run edmPickEvents.py

The script edmPickEvents.py is part of the PhysicsTools/Utilities package with tag V08-01-00 and works for CMSSW 36X or higher.
>
>

How to setup the environment to run edmPickEvents.py The

The script

edmPickEvents.py is part of the PhysicsTools/Utilities package with tag V08-01-00 and works for CMSSW 36X or higher. For releases earlier than 3.8.0_patch1, you also need FWCore/PythonUtilities, tag V01-02-00)

 

To setup a CMSSW release look HERE and then checkout the tag as follows:

Revision 52010-09-10 - SudhirMalik

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Changed:
<
<
Complete: 4
>
>
Complete: 5
 
Detailed Review status
Line: 11 to 11
 The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. This utility is a python script
Changed:
<
<
called edmPickEvents.py and and can run an interactive job for a small number of events, or will can set up a CRAB job.
>
>
called edmPickEvents.py and and can run an interactive job for a small number of events, or can set up a CRAB job.
  As a recent improvement, this script will allow you access events from data that is not local by using CRAB.
Line: 24 to 24
 

Contents

How to setup the environment to run edmPickEvents.py

Changed:
<
<
The script edmPickEvents.py is part of the !PhysicsTools/Utilities package with tag V08-01-00 and works for CMSSW 36X or higher.
>
>
The script edmPickEvents.py is part of the PhysicsTools/Utilities package with tag V08-01-00 and works for CMSSW 36X or higher.
 

To setup a CMSSW release look HERE and then checkout the tag as follows:

Line: 40 to 40
 job. If you run interactively, before you run it, make sure you have the dataset that you want to pick events from, available locally. If you have few events to pick, run it interactively. In case you do not
Changed:
<
<
have the dataset available locally or you have lot of events to pick,
>
>
have the dataset available locally or you have a lot of events to pick,
 submit a CRAB job.

Run edmPickEvents.py Interactively

Line: 54 to 54
 where

  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from
Changed:
<
<
  • 140160:98:82878561 and 140160:101:85437704 are the two events you want to pick. The syntax is _Run:Lumi:Event*
>
>
  • 140160:98:82878561 and 140160:101:85437704 are the two events you want to pick. The syntax is Run:Lumi:Event
  and the screen output would look like this:
Line: 76 to 76
 

Run edmPickEvents.py with CRAB

If you are running over a large number of event, if you know the data

Changed:
<
<
set is not available locally, ir you just don't want to wait for a
>
>
set is not available locally, or you just don't want to wait for a
 long job to finish, you can use edmPickEvents.py to setup a CRAB job for you.
Changed:
<
<
To run a CRAB job, do:
>
>
To run a CRAB job:
 
  • First setup the Crab environment following the instructions at SWGuideCrab
Line: 91 to 91
 edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab
Changed:
<
<
(Note that for a large number of events , you do not need to specify the --crab option.) When you run this it give the following screen output
>
>
(Note that for a large number of events , you do not need to specify the --crab option)

When you run this it gives the following screen output

 
Please visit https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideCrab

Revision 42010-09-09 - CharlesPlager

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"
Deleted:
<
<

This page is under constrcution

 Complete: 4
Detailed Review status
Line: 9 to 8
 

Goals of this page:

Changed:
<
<
The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. This utility is a python script called edmPickEvents.py and it runs interactively as well as by submitting a CRAB job.
>
>
The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. This utility is a python script called edmPickEvents.py and and can run an interactive job for a small number of events, or will can set up a CRAB job.

As a recent improvement, this script will allow you access events from data that is not local by using CRAB.

If you plan on making very large "skims' (e.g., all W candidates), please consider collaborating with others as to minimize the number of identical collections we have.

 

Contents

Line: 21 to 31
  cvs co -r V08-01-00 PhysicsTools/Utilities
Changed:
<
<
REMEMBER to do scram b after checking out the package.
>
>
REMEMBER to do scram b and rehash after checking out the package.
 

How to Run edmPickEvents.py

Changed:
<
<
The script edmPickEvents.py can be run interactively or via a CRAB job. If you run interactively, before you run it, make sure you have the dataset that you want to pick events from, available locally. If you have few events to pick, run it interactively. In case you do not have the dataset available locally or you have lot of events to pick, submit a CRAB job.
>
>
The script edmPickEvents.py can be run interactively or via a CRAB job. If you run interactively, before you run it, make sure you have the dataset that you want to pick events from, available locally. If you have few events to pick, run it interactively. In case you do not have the dataset available locally or you have lot of events to pick, submit a CRAB job.

Run edmPickEvents.py Interactively

  To run the script interactively do the following, for example


Changed:
<
<
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --runInteractive
>
>
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704
 

where

Changed:
<
<
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from
  • 140160:98:82878561 140160:101:85437704 are the two events you want to pick. The syntax is Run:Lumi:Event
>
>
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from
  • 140160:98:82878561 and 140160:101:85437704 are the two events you want to pick. The syntax is _Run:Lumi:Event*
  and the screen output would look like this:
Deleted:
<
<
<!--/twistyPlugin twikiMakeVisibleInline-->
 
edmCopyPickMerge outputFile=pickevents.root \
  eventsToProcess=140160:82878561,140160:85437704 \
  inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
Deleted:
<
<
09-Sep-2010 14:12:19 CDT Initiating request to open file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root 09-Sep-2010 14:12:26 CDT Successfully opened file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root Begin processing the 1st record. Run 140160, Event 82878561, LumiSection 98 at 09-Sep-2010 14:12:36 CDT Begin processing the 2nd record. Run 140160, Event 85437704, LumiSection 101 at 09-Sep-2010 14:12:39 CDT 09-Sep-2010 14:12:56 CDT Closed file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

=========================================

MessageLogger Summary

type category sev module subroutine count total ---- -------------------- -- ---------------- ---------------- ----- ----- 1 fileAction -s AfterFile 1 1 2 fileAction -s PoolSource:sourc 1 1 3 fileAction -s file_close 1 1

type category Examples: run/evt run/evt run/evt ---- -------------------- ---------------- ---------------- ---------------- 1 fileAction pre-events 2 fileAction pre-events 3 fileAction PostEndRun

Severity # Occurrences Total Occurrences


------------- ----------------- System 3 3
 
Changed:
<
<
<!--/twistyPlugin-->
>
>
In this case, the user can either paste the above edmCopyPickMerge output or run edmPickEvents.py with the --runInteractive flag which will run it for you (warning, this can take a long time).
  This will created a ROOT file called pickevents.root in the same directory you executed the command from. Also note that edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun.
Deleted:
<
<
You can also run it like this:

edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704

In this case the screen output would look like this:

<!--/twistyPlugin twikiMakeVisibleInline-->

  edmCopyPickMerge outputFile=pickevents.root \
  eventsToProcess=140160:82878561,140160:85437704 \
  inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

<!--/twistyPlugin-->
 
Added:
>
>

Run edmPickEvents.py with CRAB

 
Changed:
<
<
Now you can cut and paste this whole output and run it and the output would look like this:

<!--/twistyPlugin twikiMakeVisibleInline-->


[malik@cmslpc05 src]$ edmCopyPickMerge outputFile=pickevents.root \
?   eventsToProcess=140160:82878561,140160:85437704 \
?   inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
09-Sep-2010 14:08:40 CDT  Initiating request to open file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
09-Sep-2010 14:08:47 CDT  Successfully opened file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
Begin processing the 1st record. Run 140160, Event 82878561, LumiSection 98 at 09-Sep-2010 14:08:57 CDT
Begin processing the 2nd record. Run 140160, Event 85437704, LumiSection 101 at 09-Sep-2010 14:09:00 CDT
09-Sep-2010 14:09:17 CDT  Closed file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

=============================================

MessageLogger Summary

 type     category        sev    module        subroutine        count    total
 ---- -------------------- -- ---------------- ----------------  -----    -----
    1 fileAction           -s AfterFile                              1        1
    2 fileAction           -s PoolSource:sourc                       1        1
    3 fileAction           -s file_close                             1        1

 type    category    Examples: run/evt        run/evt          run/evt
 ---- -------------------- ---------------- ---------------- ----------------
    1 fileAction           pre-events                        
    2 fileAction           pre-events                        
    3 fileAction           PostEndRun                        

Severity    # Occurrences   Total Occurrences
--------    -------------   -----------------
System                  3                   3
[malik@cmslpc05 src]$ 

<!--/twistyPlugin-->
>
>
If you are running over a large number of event, if you know the data set is not available locally, ir you just don't want to wait for a long job to finish, you can use edmPickEvents.py to setup a CRAB job for you.
  To run a CRAB job, do:

Line: 155 to 91
 edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab
Changed:
<
<
When you run this it give the following screen output

<!--/twistyPlugin twikiMakeVisibleInline-->
>
>
(Note that for a large number of events , you do not need to specify the --crab option.) When you run this it give the following screen output
 
Please visit https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideCrab
Line: 171 to 105
 
Deleted:
<
<
<!--/twistyPlugin-->
 This will create the configuration file called pickevents_crab.config for you
Added:
>
>
  • If desired, you can modify the contents of pickEvents_crab.config. Most people should find the defaults sufficient.
 
  • Then create a Crab job
crab -create -cfg pickevents_crab.config

Line: 197 to 133
 crab -getoutput
Added:
>
>
To merge the output of CRAB, you can follow the easy instructions here:
 
Added:
>
>
ls -1 pick*root > myFiles.txt
edmCopyPickMerge inputFiles_load=myFiles.txt output=pickevents_merged.root
  -- SudhirMalik - 09-Sep-2010

Revision 32010-09-09 - SudhirMalik

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

This page is under constrcution

Changed:
<
<
Complete: 0
>
>
Complete: 4
 
Detailed Review status
Added:
>
>
 

Goals of this page:

Changed:
<
<
The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. The utility is called edmPickEvents.py and is located in CVS at CMSSW/PhysicsTools/Utilities/scripts/. When you have few events to copy
>
>
The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. This utility is a python script called edmPickEvents.py and it runs interactively as well as by submitting a CRAB job.
 

Contents

Changed:
<
<
>
>

How to setup the environment to run edmPickEvents.py

The script edmPickEvents.py is part of the !PhysicsTools/Utilities package with tag V08-01-00 and works for CMSSW 36X or higher.

To setup a CMSSW release look HERE and then checkout the tag as follows:

cvs co -r V08-01-00 PhysicsTools/Utilities

REMEMBER to do scram b after checking out the package.

How to Run edmPickEvents.py

 
Changed:
<
<

How to setup the environment to use the script

>
>
The script edmPickEvents.py can be run interactively or via a CRAB job. If you run interactively, before you run it, make sure you have the dataset that you want to pick events from, available locally. If you have few events to pick, run it interactively. In case you do not have the dataset available locally or you have lot of events to pick, submit a CRAB job.
 
Changed:
<
<
The script would be part of the CMSSW release but for now you would have to check out the tag like this ( We assume that you have setup a CMSSW release. If not looke HERE. The utility will work for CMSSW 36X or higher.
>
>
To run the script interactively do the following, for example
 
Changed:
<
<
cvs co -r V08-01-00 PhysicsTools/Utilities
>
>
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --runInteractive
 
Added:
>
>
where
 
Added:
>
>
  • /EG/Run2010A-PromptReco-v4/RECO is the dataset you want to pick events from
  • 140160:98:82878561 140160:101:85437704 are the two events you want to pick. The syntax is Run:Lumi:Event
 
Changed:
<
<
edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun
>
>
and the screen output would look like this:

<!--/twistyPlugin twikiMakeVisibleInline-->
 
Changed:
<
<
edmPickEvents.py "*/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 echo "source /uscmst1/prod/sw/cms/shrc prod" > ~/.profile edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab --crabCondor
>
>
edmCopyPickMerge outputFile=pickevents.root eventsToProcess=140160:82878561,140160:85437704 inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

09-Sep-2010 14:12:19 CDT Initiating request to open file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root 09-Sep-2010 14:12:26 CDT Successfully opened file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root Begin processing the 1st record. Run 140160, Event 82878561, LumiSection 98 at 09-Sep-2010 14:12:36 CDT Begin processing the 2nd record. Run 140160, Event 85437704, LumiSection 101 at 09-Sep-2010 14:12:39 CDT 09-Sep-2010 14:12:56 CDT Closed file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

=========================================

MessageLogger Summary

type category sev module subroutine count total ---- -------------------- -- ---------------- ---------------- ----- ----- 1 fileAction -s AfterFile 1 1 2 fileAction -s PoolSource:sourc 1 1 3 fileAction -s file_close 1 1

type category Examples: run/evt run/evt run/evt ---- -------------------- ---------------- ---------------- ---------------- 1 fileAction pre-events 2 fileAction pre-events 3 fileAction PostEndRun

Severity # Occurrences Total Occurrences


------------- ----------------- System 3 3

<!--/twistyPlugin-->

This will created a ROOT file called pickevents.root in the same directory you executed the command from. Also note that edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun.

You can also run it like this:

edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704

In this case the screen output would look like this:

<!--/twistyPlugin twikiMakeVisibleInline-->

  edmCopyPickMerge outputFile=pickevents.root \
  eventsToProcess=140160:82878561,140160:85437704 \
  inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

<!--/twistyPlugin-->

Now you can cut and paste this whole output and run it and the output would look like this:

<!--/twistyPlugin twikiMakeVisibleInline-->


[malik@cmslpc05 src]$ edmCopyPickMerge outputFile=pickevents.root \
?   eventsToProcess=140160:82878561,140160:85437704 \
?   inputFiles=/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
09-Sep-2010 14:08:40 CDT  Initiating request to open file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
09-Sep-2010 14:08:47 CDT  Successfully opened file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root
Begin processing the 1st record. Run 140160, Event 82878561, LumiSection 98 at 09-Sep-2010 14:08:57 CDT
Begin processing the 2nd record. Run 140160, Event 85437704, LumiSection 101 at 09-Sep-2010 14:09:00 CDT
09-Sep-2010 14:09:17 CDT  Closed file dcap://cmsdca1.fnal.gov:24140/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Run2010A/EG/RECO/v4/000/140/160/20230629-2190-DF11-AA6E-0030487CD178.root

=============================================

MessageLogger Summary

 type     category        sev    module        subroutine        count    total
 ---- -------------------- -- ---------------- ----------------  -----    -----
    1 fileAction           -s AfterFile                              1        1
    2 fileAction           -s PoolSource:sourc                       1        1
    3 fileAction           -s file_close                             1        1

 type    category    Examples: run/evt        run/evt          run/evt
 ---- -------------------- ---------------- ---------------- ----------------
    1 fileAction           pre-events                        
    2 fileAction           pre-events                        
    3 fileAction           PostEndRun                        

Severity    # Occurrences   Total Occurrences
--------    -------------   -----------------
System                  3                   3
[malik@cmslpc05 src]$ 

<!--/twistyPlugin-->

To run a CRAB job, do:

  • First setup the Crab environment following the instructions at SWGuideCrab
  • Then run the script as follows ( In this case we are running the two events via a CRAB job) :

edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab 

When you run this it give the following screen output

<!--/twistyPlugin twikiMakeVisibleInline-->

Please visit https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideCrab

Setup your environment for CRAB.  Then edit pickevents_crab.config to make any desired changed.  The run:
crab -create -cfg pickevents_crab.config
crab -submit

<!--/twistyPlugin-->

This will create the configuration file called pickevents_crab.config for you

  • Then create a Crab job

 crab -create -cfg pickevents_crab.config
Added:
>
>

  • Submit a Crab job

 crab -submit
Added:
>
>

  • You can check the status of your job via


 crab -status
Changed:
<
<
cd crab_0_100909_112617/res/
>
>

Once the CRAB job finishes successfully, you can get the output ROOT file in the craboutput /res directory:


 crab -getoutput
Changed:
<
<
edmFileUtil -e -f file:pickevents_1_1_Bwz.root
>
>
  -- SudhirMalik - 09-Sep-2010

Revision 22010-09-09 - SudhirMalik

Line: 1 to 1
 
META TOPICPARENT name="WorkBook"

This page is under constrcution

Added:
>
>
Complete: 0
Detailed Review status
 
Added:
>
>

Goals of this page:

The goal of this page is to help the users copy, pick and merge events into an output root file from their choice of a data set and a list of events that they are interested in. The utility is called edmPickEvents.py and is located in CVS at CMSSW/PhysicsTools/Utilities/scripts/. When you have few events to copy

Contents

How to setup the environment to use the script

The script would be part of the CMSSW release but for now you would have to check out the tag like this ( We assume that you have setup a CMSSW release. If not looke HERE. The utility will work for CMSSW 36X or higher.

cvs co -r V08-01-00 PhysicsTools/Utilities

edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun

edmPickEvents.py "*/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704
echo "source /uscmst1/prod/sw/cms/shrc prod" > ~/.profile
edmPickEvents.py "/EG/Run2010A-PromptReco-v4/RECO" 140160:98:82878561 140160,101,85437704 --crab --crabCondor
crab -create -cfg pickevents_crab.config
crab -submit
crab -status
cd crab_0_100909_112617/res/
crab -getoutput
edmFileUtil -e -f file:pickevents_1_1_Bwz.root
  -- SudhirMalik - 09-Sep-2010

Revision 12010-09-09 - SudhirMalik

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WorkBook"

This page is under constrcution

-- SudhirMalik - 09-Sep-2010

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback