Difference: DqmWrappers (1 vs. 10)

Revision 102019-08-21 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
DQM Wrappers
>
>
DQM Wrappers
  This page provides a guide to use the DQM bash wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accessed from any lxplus machine.
Line: 34 to 34
 This wrapper provides, as its name states, a replica of the reco configuration used for a specific run and one of its datasets. You can check the reco configuration of any run on https://cmsweb.cern.ch/t0wmadatasvc/prod/reco_config?run=runnumber&primary_dataset=nameofdataset . If there are none available then the wrapper won't work. You will also need to provide a valid LFN to be able to run the final step of the processing.

As an example the wrapper was tested with the following configuration: Run number 323940, Dataset: ZeroBias, LFN: /store/data/Run2018D/ZeroBias/RAW/v1/000/323/940/00000/8F6027B6-0B11-5C41-B790-7523DFB612CE.root as shown in the following image:

Changed:
<
<
Screen_Shot_2019-08-07_at_3.25.28_PM.png
>
>
Screen_Shot_2019-08-21_at_9.18.32_AM.png
  After this the code will run for some time by creating the requested area. Then the wrapper will ask if your CMSSW version is superior to 11_0_0_pre1, because that will decide if the code will take a nThreads value as input for the configuration or not. For this example the answer is N, as shown in the following image: Screen_Shot_2019-08-07_at_12.25.56_PM.png
Line: 53 to 54
  An example was run using the following configuration: run number: 323940, full dataset name: /ZeroBias/Run2018D-PromptReco-v2/DQMIO, LFN: file:/afs/cern.ch/user/f/fiori/public/Andres/9F411A2F-9C94-D54F-894B-83D60BF55C41.root . You can see its configuration on the following image:
Changed:
<
<
Screen_Shot_2019-08-07_at_4.55.40_PM.png
>
>
Screen_Shot_2019-08-21_at_9.28.53_AM.png
  After the wrapper finishes its process you will obtain a message similar to this one depending on your location:
Changed:
<
<
Screen_Shot_2019-08-07_at_4.55.40_PM.png
>
>
Screen_Shot_2019-08-21_at_9.34.26_AM.png
  So now the configuration files RunDQMHarvestingCfg.pkl and RunDQMHarvestingCfg.py are available at the folder shown on screen. If you want to run the configuration then you will have to go to that folder and then run cmsenv on the prompt command line.
Line: 73 to 75
 
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_4.35.46_PM.png" attr="" comment="" date="1565213795" name="Screen_Shot_2019-08-07_at_4.35.46_PM.png" path="Screen_Shot_2019-08-07_at_4.35.46_PM.png" size="104881" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_4.55.40_PM.png" attr="" comment="" date="1565215289" name="Screen_Shot_2019-08-07_at_4.55.40_PM.png" path="Screen_Shot_2019-08-07_at_4.55.40_PM.png" size="76277" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_5.41.58_PM.png" attr="" comment="" date="1565217798" name="Screen_Shot_2019-08-07_at_5.41.58_PM.png" path="Screen_Shot_2019-08-07_at_5.41.58_PM.png" size="121593" user="anquinte" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Screen_Shot_2019-08-21_at_9.18.32_AM.png" attr="" comment="recowrapper config" date="1566397386" name="Screen_Shot_2019-08-21_at_9.18.32_AM.png" path="Screen_Shot_2019-08-21_at_9.18.32_AM.png" size="33210" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-21_at_9.28.53_AM.png" attr="" comment="harvestwrapperconfig" date="1566397785" name="Screen_Shot_2019-08-21_at_9.28.53_AM.png" path="Screen_Shot_2019-08-21_at_9.28.53_AM.png" size="53043" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-21_at_9.34.26_AM.png" attr="" comment="harvestoutput1" date="1566398208" name="Screen_Shot_2019-08-21_at_9.34.26_AM.png" path="Screen_Shot_2019-08-21_at_9.34.26_AM.png" size="31217" user="anquinte" version="1"

Revision 92019-08-20 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
DQM Wrappers
Line: 6 to 6
  These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting and should be run by using bash command on an lxplus machine. (i. e. bash expressWrapper.sh)
Deleted:
<
<
For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have a github account on your own, because some of the code needs to be copied. For it to work properly you should do a fork of the cmss-sw repository https://github.com/cms-sw/cmssw . Also your name, email and github user should be configured on the machine. If you do not have them set, then use the following commands to do it.

git config --global user.name First name Last name

git config --global user.email <Your-Email-Address>

git config --global user.github <Your-GitHub-Account-Username>

This section will be asked by every wrapper at the start of its run as shown in the image.

Screen_Shot_2019-08-07_at_11.25.25_AM.png

 Here is a list of the different wrappers:

Revision 82019-08-08 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<

DQM Wrappers

>
>
DQM Wrappers
  This page provides a guide to use the DQM bash wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accessed from any lxplus machine.
Line: 16 to 19
  Screen_Shot_2019-08-07_at_11.25.25_AM.png
Added:
>
>
Here is a list of the different wrappers:
 
Changed:
<
<
Now every wrapper will be explained with their small differences
>
>
 

Express processing wrapper

This wrapper provides, as its name states, a replica of the express configuration used for a specific run and one of its streams. You can check the express configuration of any run on https://cmsweb.cern.ch/t0wmadatasvc/prod/express_config?run=runnumber . If you only have the run number, the wrapper will provide the streamers available for that specific run. If there are none available then the wrapper won't work. You will also need to provide a valid LFN to be able to run the final step of the processing.

Line: 42 to 44
 

Reco Processing wrapper

Added:
>
>
This wrapper provides, as its name states, a replica of the reco configuration used for a specific run and one of its datasets. You can check the reco configuration of any run on https://cmsweb.cern.ch/t0wmadatasvc/prod/reco_config?run=runnumber&primary_dataset=nameofdataset . If there are none available then the wrapper won't work. You will also need to provide a valid LFN to be able to run the final step of the processing.

As an example the wrapper was tested with the following configuration: Run number 323940, Dataset: ZeroBias, LFN: /store/data/Run2018D/ZeroBias/RAW/v1/000/323/940/00000/8F6027B6-0B11-5C41-B790-7523DFB612CE.root as shown in the following image: Screen_Shot_2019-08-07_at_3.25.28_PM.png

After this the code will run for some time by creating the requested area. Then the wrapper will ask if your CMSSW version is superior to 11_0_0_pre1, because that will decide if the code will take a nThreads value as input for the configuration or not. For this example the answer is N, as shown in the following image: Screen_Shot_2019-08-07_at_12.25.56_PM.png

Finally you will obtain a message similar to this one depending on where you run the wrapper: Screen_Shot_2019-08-07_at_4.04.37_PM.png

So now the configuration files RunPromptRecoCfg.pkl and RunPromptRecoCfg.py are available at the folder shown on screen. If you want to run the configuration then you will have to go to that folder and then run cmsenv on the prompt command line.

Afterwards you can run cmsRun -e RunPromptRecoCfg.py and obtain the result as shown in the image for our example. Screen_Shot_2019-08-07_at_4.35.46_PM.png

Harvesting wrapper

This wrapper provides, as its name states, a replica of the harvesting configuration used for a specific run and a dataset. It takes three inputs: Run number, a full dataset name (i.e. /ZeroBias/Run2018D-PromptReco-v2/DQMIO) and a location for the root file that is going to be used as an input. Unlike the other wrappers, this one does not accept a LFN, so a full path needs to be provided like file:PFN . ROOT file location can be obtained by getting the file using xrootd and copying it to the local machine.

An example was run using the following configuration: run number: 323940, full dataset name: /ZeroBias/Run2018D-PromptReco-v2/DQMIO, LFN: file:/afs/cern.ch/user/f/fiori/public/Andres/9F411A2F-9C94-D54F-894B-83D60BF55C41.root . You can see its configuration on the following image:

Screen_Shot_2019-08-07_at_4.55.40_PM.png

After the wrapper finishes its process you will obtain a message similar to this one depending on your location: Screen_Shot_2019-08-07_at_4.55.40_PM.png

So now the configuration files RunDQMHarvestingCfg.pkl and RunDQMHarvestingCfg.py are available at the folder shown on screen. If you want to run the configuration then you will have to go to that folder and then run cmsenv on the prompt command line.

Afterwards you can use cmsRun -j FrameworkJobReport.xml RunDQMHarvestingCfg.py to do the harvesting and obtain a result similar to the one shown in the following image: Screen_Shot_2019-08-07_at_5.41.58_PM.png

 
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_11.25.25_AM.png" attr="" comment="screenshot of wrapper initiation." date="1565195876" name="Screen_Shot_2019-08-07_at_11.25.25_AM.png" path="Screen_Shot_2019-08-07_at_11.25.25_AM.png" size="36003" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.16.56_PM.png" attr="" comment="" date="1565198372" name="Screen_Shot_2019-08-07_at_12.16.56_PM.png" path="Screen_Shot_2019-08-07_at_12.16.56_PM.png" size="35839" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.25.56_PM.png" attr="" comment="" date="1565198975" name="Screen_Shot_2019-08-07_at_12.25.56_PM.png" path="Screen_Shot_2019-08-07_at_12.25.56_PM.png" size="11540" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.33.17_PM.png" attr="" comment="" date="1565199253" name="Screen_Shot_2019-08-07_at_12.33.17_PM.png" path="Screen_Shot_2019-08-07_at_12.33.17_PM.png" size="23437" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.44.01_PM.png" attr="" comment="" date="1565199900" name="Screen_Shot_2019-08-07_at_12.44.01_PM.png" path="Screen_Shot_2019-08-07_at_12.44.01_PM.png" size="97304" user="anquinte" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_3.25.28_PM.png" attr="" comment="" date="1565209615" name="Screen_Shot_2019-08-07_at_3.25.28_PM.png" path="Screen_Shot_2019-08-07_at_3.25.28_PM.png" size="53357" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_4.04.37_PM.png" attr="" comment="" date="1565213185" name="Screen_Shot_2019-08-07_at_4.04.37_PM.png" path="Screen_Shot_2019-08-07_at_4.04.37_PM.png" size="20401" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_4.35.46_PM.png" attr="" comment="" date="1565213795" name="Screen_Shot_2019-08-07_at_4.35.46_PM.png" path="Screen_Shot_2019-08-07_at_4.35.46_PM.png" size="104881" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_4.55.40_PM.png" attr="" comment="" date="1565215289" name="Screen_Shot_2019-08-07_at_4.55.40_PM.png" path="Screen_Shot_2019-08-07_at_4.55.40_PM.png" size="76277" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_5.41.58_PM.png" attr="" comment="" date="1565217798" name="Screen_Shot_2019-08-07_at_5.41.58_PM.png" path="Screen_Shot_2019-08-07_at_5.41.58_PM.png" size="121593" user="anquinte" version="1"

Revision 72019-08-07 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
+++DQM Wrappers
>
>

DQM Wrappers

  This page provides a guide to use the DQM bash wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accessed from any lxplus machine.

These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting and should be run by using bash command on an lxplus machine. (i. e. bash expressWrapper.sh)

For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have a github account on your own, because some of the code needs to be copied. For it to work properly you should do a fork of the cmss-sw repository https://github.com/cms-sw/cmssw .

Added:
>
>
Also your name, email and github user should be configured on the machine. If you do not have them set, then use the following commands to do it. git config --global user.name First name Last name git config --global user.email <Your-Email-Address> git config --global user.github <Your-GitHub-Account-Username>
 
Added:
>
>
This section will be asked by every wrapper at the start of its run as shown in the image.
 
Added:
>
>
Screen_Shot_2019-08-07_at_11.25.25_AM.png
 
Changed:
<
<
Express processing wrapper
>
>
Now every wrapper will be explained with their small differences
 
Deleted:
<
<
This wrapper provides as its name states
 \ No newline at end of file
Added:
>
>

Express processing wrapper

This wrapper provides, as its name states, a replica of the express configuration used for a specific run and one of its streams. You can check the express configuration of any run on https://cmsweb.cern.ch/t0wmadatasvc/prod/express_config?run=runnumber . If you only have the run number, the wrapper will provide the streamers available for that specific run. If there are none available then the wrapper won't work. You will also need to provide a valid LFN to be able to run the final step of the processing.

As an example, this wrapper was run with the following configuration: Run number 323940, Stream: Express, LFN: /store/data/Run2018D/ZeroBias/RAW/v1/000/323/940/00000/8F6027B6-0B11-5C41-B790-7523DFB612CE.root as shown in the following image:

Screen_Shot_2019-08-07_at_12.16.56_PM.png

After this the code will run for some time by creating the requested area. Then the wrapper will ask if your CMSSW version is superior to 11_0_0_pre1, because that will decide if the code will take a nThreads value as input for the configuration or not. For this example the answer is N, as shown in the following image: Screen_Shot_2019-08-07_at_12.25.56_PM.png

Finally you will obtain a message similar to this one depending on where you run the wrapper: Screen_Shot_2019-08-07_at_12.33.17_PM.png

So now the configuration files RunExpressProcessingCfg.pkl and RunExpressProcessingCfg.py are available at the folder shown on screen. If you want to run the configuration then you will have to go to that folder and then run cmsenv on the prompt command line.

Afterwards you can run cmsRun -e RunExpressProcessingCfg.py and obtain the result as shown in the image for our example. Screen_Shot_2019-08-07_at_12.44.01_PM.png

Reco Processing wrapper

META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_11.25.25_AM.png" attr="" comment="screenshot of wrapper initiation." date="1565195876" name="Screen_Shot_2019-08-07_at_11.25.25_AM.png" path="Screen_Shot_2019-08-07_at_11.25.25_AM.png" size="36003" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.16.56_PM.png" attr="" comment="" date="1565198372" name="Screen_Shot_2019-08-07_at_12.16.56_PM.png" path="Screen_Shot_2019-08-07_at_12.16.56_PM.png" size="35839" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.25.56_PM.png" attr="" comment="" date="1565198975" name="Screen_Shot_2019-08-07_at_12.25.56_PM.png" path="Screen_Shot_2019-08-07_at_12.25.56_PM.png" size="11540" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.33.17_PM.png" attr="" comment="" date="1565199253" name="Screen_Shot_2019-08-07_at_12.33.17_PM.png" path="Screen_Shot_2019-08-07_at_12.33.17_PM.png" size="23437" user="anquinte" version="1"
META FILEATTACHMENT attachment="Screen_Shot_2019-08-07_at_12.44.01_PM.png" attr="" comment="" date="1565199900" name="Screen_Shot_2019-08-07_at_12.44.01_PM.png" path="Screen_Shot_2019-08-07_at_12.44.01_PM.png" size="97304" user="anquinte" version="1"

Revision 62019-08-07 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
+++DQM Wrappers
Changed:
<
<
This page provides a guide to use the DQM wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accesed from any lxplus machine.
>
>
This page provides a guide to use the DQM bash wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accessed from any lxplus machine.

These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting and should be run by using bash command on an lxplus machine. (i. e. bash expressWrapper.sh)

For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have a github account on your own, because some of the code needs to be copied. For it to work properly you should do a fork of the cmss-sw repository https://github.com/cms-sw/cmssw .

 
Deleted:
<
<
These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting.
 
Deleted:
<
<
For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have a github account on your own, because some of the code needs to be copied into your personal fork of the cmss-sw repository https://github.com/cms-sw/cmssw .
 

Express processing wrapper

Revision 52019-08-06 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
+++DQM Wrappers

Revision 42019-08-03 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
+++DQM Wrappers

Revision 32019-08-02 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
+++DQM Wrappers
Line: 6 to 6
  These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting.
Changed:
<
<
For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have your github account configured on
>
>
For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have a github account on your own, because some of the code needs to be copied into your personal fork of the cmss-sw repository https://github.com/cms-sw/cmssw .
 

Express processing wrapper

Revision 22019-08-02 - AndresFelipeQuinteroParra

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
DQM Wrappers
>
>
+++DQM Wrappers
  This page provides a guide to use the DQM wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accesed from any lxplus machine.

These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting. \ No newline at end of file

Added:
>
>
For every wrapper to work, some previous configuration needs to be made on the machine. First of all you need to have your github account configured on

Express processing wrapper

This wrapper provides as its name states

 \ No newline at end of file

Revision 12019-08-02 - AndresFelipeQuinteroParra

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="TWiki.WebPreferences"
DQM Wrappers

This page provides a guide to use the DQM wrappers located on https://github.com/andresfelquintero/wrappers or /afs/cern.ch/user/c/cmst0/public/wrappersTier0Config that can be accesed from any lxplus machine.

These wrappers provides configuration files as outcomes of Express processing, Reco processing or Harvesting.

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback