Run-II FCalMonD3PD Using Run-I D3PDMaker

The Run-II FCalMonD3PD package

As of August 2015, there is no updated version of the Run-I FCalMonD3PD code that works on Run-II ESD files. Until then, we can make some modifications to the Run-I code and preserve most of the information. The Run-I code is available for browsing here and here.

Finding and Downloading Data

To find data you would like to run on, go to https://atlas.web.cern.ch/Atlas/GROUPS/DATAPREPARATION/DataSummary/2015/, or the appropriate year, and locate a run number with luminosity plots that look like what is desired. Or, search for good data only using other methods like GoodRunLists, etc.

Once you have the desired run number you can go to the ATLAS Metadata Interface (AMI) and do a Dataset Search after logging in. If you have an idea of what you want in mind, it's easiest to go to the Simple Search option and search for data sets relating to your run number. A blank space in the AMI Simple Search is the same as the keyword AND by default, so if you want MinBias ESDs for RunNumber 214777, just search for "214777 minbias ESD". The search process may take a while (even a few minutes.) After identifying the data set that is most relevant to your work in the returned list, you can click Provenance to see a graph of other derived data objects, or GANGA export or RucioInfo based on which service you will be using. If after clicking on RucioInfo AMI says "Error :This dataset seems to be unknown to rucio," then you may still be able to download and use the file, it's just that AMI can't currently help you. If this error doesn't come up, AMI will have copy and pastable commands to paste into your terminal for easy downloading of individual files or datasets.

If you decide to use Rucio, the documentation can be found here but the only three commands that are really needed are the 'download', 'list-dids', and 'list-files' options. Oftentimes these files and data sets can range from MB-GB for the files, and GB-TB for the data sets, so it's important if you are on a service where storage space is limited to not download more than one file, or any at all. It is not necessary that you download any files if you don't want to test on a local sample file first. Additionally, if you perform a "rucio list-dids <yourdatahere>" on your data set and Rucio returns errors that the data cannot be found, it's possible that the specific ESD has been deleted or migrated somewhere else already. If Rucio cannot find your data using 'list-dids' then it doesn't matter if you aren't going to download anything locally since the PanDA client won't find the data either. This is unfortunate because if your data has not been compiled into an xAOD elsewhere then it's either unattainable or you will have to make the ESD yourself from RAW/ByteSteam data using Reco_tf.py or something similar, and this WILL have to be done on the GRID since the RAW data will be so large.

Making the Run-II FCalMonD3PD

Before you do anything, check that the athena release is at least as new as the ESD/DESD that you use. To set it up, follow the instructions below:

It is important that the Athena release used is at least as new as the ESD/DESD that you wish to run on, usually using the newest version of Athena is recommended. If your version of Athena isn't new enough the Athena log file will usually say, around the end, that there are segmentation faults, bus errors, etc. Just because your major version of Athena is correct (ex. 20.x.y.z), doesn't necessarily mean the code will work. As of the time of writing this, the following code works, but it is important to ensure that your version of Athena is up-to-date.

Presumably at this point, you either have an ESD downloaded to your local machine, or have a data set and name ready to go, e.x. "data15_13TeV.00267162.physics_MinBias.recon.ESD.f594" (Note: a / may or may not be necessary after the name of your data set for Rucio to find it. This usually results in an error on your PanDA user page like "doBasicRefine : unknown input dataset." Your personal page will be at a location like "http://bigpanda.cern.ch/tasks/?username=Your Name" and will provide details about your jobs without the need for 'pbook' (although you still have to use pbook to kill a job.)

# 1. Log into lxplus or wherever your workspace is

# 2. Set up Athena
mkdir 20.6.0.1
cd 20.6.0.1
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
# This will setup Athena in your current directory
asetup 20.6.0.1,here
# This will download the code for the Run-I D3PDMaker in your current location
svn co svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMonD3PD/trunk FCal/FCalMonD3PD

# 3. Compile the code
cd FCal/FCalMonD3PD/cmt
cmt config
source setup.sh
cmt bro gmake | tee gmake.log
source setup.sh

# 4. Edit your DPD_topOptions.py file
cd ../share/
# Open the file titled DPD_topOptions.py in your editor of choice, vim, emacs, nano, etc.
#
# NECESSARY FOR LOCAL JOBS:
# Near the top of the file there is an uncommented line 'DATAPATH1 = '/afs/... '
# Change this location to the location of a (D)ESD file you have access to, or a directory of (D)ESD files.
#
# NECESSARY FOR LOCAL AND GRID JOBS:
# Supposedly the EvtMax command is being phased out, as of writing this, this is not the case, if this is
# true at the time of using this, then this is the time to adjust this line as well.
# Lastly, we must comment out some lines of the file that seem to give Athena issues. The three lines are:
#      alg += ClusterD3PDObject (10)
#      alg += TowerD3PDObject (10) 
#      alg += PrimaryVertexD3PDObject (10)
#
# You can also find the line that says 'caloD3PD_from_esd.root'' and change it to a new output name.
#
# Now, write the file and continue on

# 5. Run FCalMonD3PD on a local (D)ESD file (takes about 5 minutes to run)
# Change the EvtMax=5 to EvtMax=-1 to run over all events locally.
# Errors will be written into the athena.log
time athena -c EvtMax=5 DPD_topOptions.py >& athena.log
# If the words 'failed' or 'aborted' are output, check the .log file.
# Otherwise, there is now a .root file 'caloD3PD_from_esd.root' in your run location or something else if you chose to rename it.
# This is a root-readable file.

# 6. Run FCalMonD3PD on the GRID
# Submit to the grid to make FCalMonD3PDs (look in SubmitGrid.sh for more examples)
# Setup the PanDA client
setupATLAS
localSetupPandaClients
# For a known data set
# Note: if you are using MC, then after the .testing. in the output you should write that the data is MonteCarlo. Ex. testing.mc15.10Aug2015_...
pathena --inDS data15_13TeV.00267162.physics_MinBias.recon.ESD.f594/ --nGBPerJob=10 --outDS user.${USER}.testing.10Aug2015_15h54 DPD_topOptions.py

# 7. Download results
# If you just want the results of running your job and do not want to perform any slimming, you can download the results of your job by using Rucio
# Note that it may be useful to use list-dids or list-files to investigate which files you may want and their size, as these full datasets can still be quite large
rucio download user.${USER}.testing.10Aug2015_15h54 DPD_topOptions.py

# 8. Slimming
# LOCAL:
cd /FCal/FCalMonD3PD/run
root -b -q -l SlimFCalMonD3PD.C
# GRID:
# (The PanDA client must be set up at this point)
# Call prun (look in SubmitGrid.sh for more examples)
# Note: You can try changing the athenaTag to older versions of Athena at this point if the version of Athena isn't found
prun --exec "rm input.txt; echo %IN | grep D3PD | sed 's/,/ /g' > input.txt; root -b -q -l SlimFCalMonD3PD.C" --outputs caloD3PD_slim.root --athenaTag 20.6.0.1 --inDS user.${USER}.testing.00267360.10Aug2015_15h54_D3PD/ --nGBPerJob 10 --outDS user.${USER}.testing_slim.10Aug2015_15h54 --useShortLivedReplicas
# Now you can download the file from the outDS location exactly as you would have in part 7.

<!--***********************************************************--> <!--Do NOT remove the remaining lines, but add requested info as appropriate--> <!--***********************************************************-->

<!--For significant updates to the topic, consider adding your 'signature' (beneath this editing box)--> Major updates:
-- DagGillberg - 2010-08-18
-- JustinKeung - 2010-09-15

<!--Please add the name of someone who is responsible for this page so that he/she can be contacted if changes are needed.
The creator's name will be added by default, but this can be replaced if appropriate.
Put the name first, without dashes.-->
%RESPONSIBLE% JustinGordonKulp
<!--Once this page has been reviewed, please add the name and the date e.g. StephenHaywood - 31 Oct 2006 -->
%REVIEW% Never reviewed