Difference: AmnonHarelDataMixingRound10 (1 vs. 23)

Revision 232009-08-27 - AmnonHarel

Line: 1 to 1
 
META TOPICPARENT name="AmnonHarelDataMixingRound9"

Introduction

Line: 36 to 36
 
  • The good news is that when adding significant energy (test #1), you get more than when adding insignificant (i.e. below RECO thresholds) energy (test #2)

  • The bad news is that in both cases the energy (digi charge, to be precise) is lower than that in the original MC
Added:
>
>
    • difference is about 4.2GeV or 82/440 photoelectrons (Shuichi: 0.2fc/GeV, Rick: 20 photoelectrons per GeV)
 
Changed:
<
<
  • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)
>
>
  • Rick found that the difference is an unlucky statistical fluctuation from the photostatistics, which use a Poisson distribution for the number of photons collected.
    • To turn it off in further tests (in the digi and mixing stages):
                 process.simHcalUnsuppressedDigis.he.doPhotoStatistics = False
                 process.mixData.he.doPhotoStatistics = False
      
    • After turning them off, Rick gets test #2 (@ HE +23,41,1)
      • Data: 0:3.42646, 1:1.38063, 2:2.33683, 3:1.64851, 4:3.42646, 5:1.38063, 6:1.23356, 7:3.86261, 8:2.31732, 9:1.38063
      • MC: 0:2.67812, 1:2.43128, 2:2.67594, 3:1.64185, 4:90.37, 5:44.0923, 6:16.0231, 7:7.26294, 8:5.27362, 9:3.55629
      • Mixed: 0:3.42646, 1:1.38063, 2:2.33683, 3:2.20218, 4:96.6625, 5:38.6091, 6:13.5729, 7:9.10315, 8:4.75369, 9:2.43804
    • So it looks like I was bit by the photo statistics in the MC digitization stage

Minor Quibbles

  • cmsDriver's help messages don't mention choosing sequence(s) for a step
 

Mike's Instructions

Revision 222009-08-13 - AmnonHarel

Line: 1 to 1
 
META TOPICPARENT name="AmnonHarelDataMixingRound9"

Introduction

Line: 130 to 130
 
Changed:
<
<
Note that the hardest digi, at time slice 4 (i.e. the fifth one), went DOWN by a lot. Considering the non-linearities, this is a lot of energy being subtracted. Is that a reasonable thing to do?
>
>
Note that the hardest digi, at time slice 4 (i.e. the fifth one), went DOWN by a lot. Considering the non-linearities, this is a lot of energy being subtracted. Is that a reasonable thing to do?
 
Added:
>
>
Some answers:
  • Shuichi: the fC are already linearized. This is ~4.2GeV.
  • Rick: This is about 4 sigma, so not just a statistical fluctuation
 

Technical details

Revision 212009-08-12 - AmnonHarel

Line: 1 to 1
 
META TOPICPARENT name="AmnonHarelDataMixingRound9"

Introduction

Added:
>
>
See index in AmnonHarelDataMixing
 Testing Mike's DataMixer, this time with the cmsDriver technology. Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun). See test #1 below.

Revision 202009-08-10 - AmnonHarel

Line: 1 to 1
 
META TOPICPARENT name="AmnonHarelDataMixingRound9"

Introduction

Testing Mike's DataMixer, this time with the cmsDriver technology. Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun).

Added:
>
>
See test #1 below.
 But while getting there, run into the odd looking overlay in "test #2".

Conclusions

Line: 26 to 27
 
  • Can mix existing MC and data as described below
Changed:
<
<
 
mixed (HCAL towers & digis)
>
>
 
  • Interesting overlay test #1: data (HCAL towers & no rechit --> I can't show digis for this one) + MC (HCAL towers & digis) -->
mixed (HCAL towers & digis)

  • The good news is that when adding significant energy (test #1), you get more than when adding insignificant (i.e. below RECO thresholds) energy (test #2)

  • The bad news is that in both cases the energy (digi charge, to be precise) is lower than that in the original MC
 
Deleted:
<
<
  • Computing:
 
    • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)

Mike's Instructions

Line: 84 to 90
 
 
    • dumping the information I need from the MC with dumpGun.py -->
dumpGun.root
  • redid with 500 events MC500.root, dumpMC500.py --> dumpMC500.root
  • redid with 5K events and 50GeV pions MC5K.root ...
  • Changed:
    <
    <
      • redid, dumping all events (bug in my configs that do so fixed Aug 10) with dumpMC500a.py
    >
    >
      • redid, dumping all recHits (bug in my configs that do so fixed Aug 10) with dumpMC500a.py
     
     
    • mixed events: hacked the cmsDriver-produced mixer config file to use the existing MC file. Details below.
      • key point: rerunning the digi step
      • dumping mixed events into my format with dumpnm.py -->
    dumpnm.root
    Line: 103 to 109
     
    • This check agrees with the digis I got from the full data file: HCAL towers & digis
    • Mixed the events with nmt1a.py (fixed the secondary input file, failed to turn on the needed debug statements)
    • dumped the mixed digis with dumpnmt1a.py
    Added:
    >
    >
      • Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a). Fixed a bug in my AHtest_3_cfi.py to do so (Aug 10).
     
     
    • added debug prints and realized the wrong data event is used -->
    I haven't managed to switch the secondary source.
    Added:
    >
    >
    • corrected a typo (fileNames, not filenames) and did nmt1b.py which takes the CRAFT event I wanted.
      • dumped with dumpnmt1b.py
     
    Deleted:
    <
    <
    Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a). Fixed a bug in my AHtest_3_cfi.py to do so (Aug 10).
      An alternative approach to try: take the most energetic MC hit, and aim the pion gun at it at various energies.
    Line: 191 to 199
     
    META FILEATTACHMENT attachment="sad4_210.gif" attr="" comment="CRAFT towers (test #2)" date="1249915221" name="sad4_210.gif" path="sad4_210.gif" size="17734" stream="sad4_210.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sax1_2.gif" attr="" comment="Mixed towers (test #2)" date="1249915342" name="sax1_2.gif" path="sax1_2.gif" size="16126" stream="sax1_2.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sxx1_2h441.png" attr="" comment="Mixed digis (test #2)" date="1249915373" name="sxx1_2h441.png" path="sxx1_2h441.png" size="26836" stream="sxx1_2h441.png" user="Main.AmnonHarel" version="1"
    Added:
    >
    >
    META FILEATTACHMENT attachment="sxxb2h441.png" attr="" comment="Mixed digis (test #1)" date="1249920754" name="sxxb2h441.png" path="sxxb2h441.png" size="27000" stream="sxxb2h441.png" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="saxb2.gif" attr="" comment="Mixed towers (test #1)" date="1249920779" name="saxb2.gif" path="saxb2.gif" size="16187" stream="saxb2.gif" user="Main.AmnonHarel" version="1"

    Revision 192009-08-10 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Testing Mike's DataMixer, this time with the cmsDriver technology.

    Changed:
    <
    <
    Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun), such as this:

    Though overlays look reasonable, I can't find the big energies (~10GeV) from CRAFT in the mixed output. A problem with my test? only one data event (the right one, according to the log file) was used, and there's no trace of it in the overlayed CRAFT energy. The only consistent feature added relative to the original MC is soft and in a different region of the detector.

    >
    >
    Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun). But while getting there, run into the odd looking overlay in "test #2".
     

    Conclusions

    Line: 92 to 89
     
     
      • key point: rerunning the digi step
      • dumping mixed events into my format with dumpnm.py -->
    dumpnm.root
    Added:
    >
    >
    ls StandardSequeces.* | awk -F. '{system("ln -s "$0" "$1".StandardSequences."$3)}'
     

    Specific mixing test(s)

    Added:
    >
    >

    Test #1

     
    Found (*) the location (HCAL tower) with the highest overlap between MC500 and the data file: ('25, 41', 8.9916669999999996, [84], [384]) 
    • Copied that data file with copyD1.py -->
    D1event.root
  • checked it's the right event with dumpD1.py -> dumpD1.root --> this fails because reco wasn't run (see gdd1.log).
  • Line: 103 to 103
     
    • This check agrees with the digis I got from the full data file: HCAL towers & digis
    • Mixed the events with nmt1a.py (fixed the secondary input file, failed to turn on the needed debug statements)
    • dumped the mixed digis with dumpnmt1a.py
    Added:
    >
    >
     
    • added debug prints and realized the wrong data event is used -->
    I haven't managed to switch the secondary source.
      Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a). Fixed a bug in my AHtest_3_cfi.py to do so (Aug 10).

    An alternative approach to try: take the most energetic MC hit, and aim the pion gun at it at various energies.

    Added:
    >
    >

    Test #2

    This is what nmt1a.py actually does, in contrast with what I meant it to do: it mixes data event 68021 event: 7823771 on top of MC event #84.

    • data (HCAL towers & no rechit -->
    I can't show digis for this one) +

    Note that the hardest digi, at time slice 4 (i.e. the fifth one), went DOWN by a lot. Considering the non-linearities, this is a lot of energy being subtracted. Is that a reasonable thing to do?

     

    Technical details

    (*) found by (a) root macro loops over file, updates an std::map, outputs highest energy seen per tower location and the event number where that happened,

    Line: 172 to 188
     
    META FILEATTACHMENT attachment="sam5_84.gif" attr="" comment="MC tower (test #1)" date="1247737407" name="sam5_84.gif" path="sam5_84.gif" size="15653" stream="sam5_84.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sxd4_384h441.png" attr="" comment="CRAFT digis (test #1)" date="1247737438" name="sxd4_384h441.png" path="sxd4_384h441.png" size="29162" stream="sxd4_384h441.png" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sad4_384.gif" attr="" comment="CRAFT towers (test #1)" date="1247737471" name="sad4_384.gif" path="sad4_384.gif" size="17278" stream="sad4_384.gif" user="Main.AmnonHarel" version="1"
    Added:
    >
    >
    META FILEATTACHMENT attachment="sad4_210.gif" attr="" comment="CRAFT towers (test #2)" date="1249915221" name="sad4_210.gif" path="sad4_210.gif" size="17734" stream="sad4_210.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sax1_2.gif" attr="" comment="Mixed towers (test #2)" date="1249915342" name="sax1_2.gif" path="sax1_2.gif" size="16126" stream="sax1_2.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sxx1_2h441.png" attr="" comment="Mixed digis (test #2)" date="1249915373" name="sxx1_2h441.png" path="sxx1_2h441.png" size="26836" stream="sxx1_2h441.png" user="Main.AmnonHarel" version="1"

    Revision 182009-08-10 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 87 to 87
     
     
      • dumping the information I need from the MC with dumpGun.py -->
    dumpGun.root
  • redid with 500 events MC500.root, dumpMC500.py --> dumpMC500.root
  • redid with 5K events and 50GeV pions MC5K.root ...
  • Added:
    >
    >
      • redid, dumping all events (bug in my configs that do so fixed Aug 10) with dumpMC500a.py
     
     
    • mixed events: hacked the cmsDriver-produced mixer config file to use the existing MC file. Details below.
      • key point: rerunning the digi step
      • dumping mixed events into my format with dumpnm.py -->
    dumpnm.root
    Line: 103 to 104
     
    • Mixed the events with nmt1a.py (fixed the secondary input file, failed to turn on the needed debug statements)
    • dumped the mixed digis with dumpnmt1a.py
    Changed:
    <
    <
    Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a)
    >
    >
    Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a). Fixed a bug in my AHtest_3_cfi.py to do so (Aug 10).
      An alternative approach to try: take the most energetic MC hit, and aim the pion gun at it at various energies.

    Revision 172009-08-09 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 100 to 100
     
    • reco & dump with prepD1b.py (earlier attempts prepD1a.py, prepD1.py and prepData1.py aren't quite right)
    • The data event (#348) is from Run: 68021 Event: 14881518
    • This check agrees with the digis I got from the full data file: HCAL towers & digis
    Added:
    >
    >
    • Mixed the events with nmt1a.py (fixed the secondary input file, failed to turn on the needed debug statements)
    • dumped the mixed digis with dumpnmt1a.py
     
    Changed:
    <
    <
    For the next test, will take the most energetic MC hit, and aim the pion gun at it at various energies.
    >
    >
    Results are a bit confusing. Doesn't help that I filtered some recHits. Rerunning with no thresholds (dumpnmt1a, and dumpMC500a)

    An alternative approach to try: take the most energetic MC hit, and aim the pion gun at it at various energies.

     

    Technical details

    Revision 162009-08-09 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 96 to 96
     
    Found (*) the location (HCAL tower) with the highest overlap between MC500 and the data file: ('25, 41', 8.9916669999999996, [84], [384]) 
    • Copied that data file with copyD1.py -->
    D1event.root
    Changed:
    <
    <
     
    • checked it's the right event with dumpD1.py -> dumpD1.root -->
    this fails because reco wasn't run (see gdd1.log). Is this where my mistake was?
  • reco & dump with prepD1a.py (earlier attempts prepD1.py and prepData1.py aren't quite right)
  • >
    >
     
    • checked it's the right event with dumpD1.py -> dumpD1.root -->
    this fails because reco wasn't run (see gdd1.log).
  • reco & dump with prepD1b.py (earlier attempts prepD1a.py, prepD1.py and prepData1.py aren't quite right)
  •  
    • The data event (#348) is from Run: 68021 Event: 14881518
    Added:
    >
    >
    • This check agrees with the digis I got from the full data file: HCAL towers & digis
      For the next test, will take the most energetic MC hit, and aim the pion gun at it at various energies.

    Revision 152009-08-09 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 96 to 96
     
    Found (*) the location (HCAL tower) with the highest overlap between MC500 and the data file: ('25, 41', 8.9916669999999996, [84], [384]) 
    • Copied that data file with copyD1.py -->
    D1event.root
    Changed:
    <
    <
    • checked it's the right event with dumpD1.py -> dumpD1.root
    >
    >
     
    • checked it's the right event with dumpD1.py -> dumpD1.root -->
    this fails because reco wasn't run (see gdd1.log). Is this where my mistake was?
  • reco & dump with prepD1a.py (earlier attempts prepD1.py and prepData1.py aren't quite right)
  •  
    • The data event (#348) is from Run: 68021 Event: 14881518

    For the next test, will take the most energetic MC hit, and aim the pion gun at it at various energies.

    Revision 142009-07-17 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 97 to 97
     
    Found (*) the location (HCAL tower) with the highest overlap between MC500 and the data file: ('25, 41', 8.9916669999999996, [84], [384]) 
    • Copied that data file with copyD1.py -->
    D1event.root
  • checked it's the right event with dumpD1.py -> dumpD1.root
  • Added:
    >
    >
    • The data event (#348) is from Run: 68021 Event: 14881518

    For the next test, will take the most energetic MC hit, and aim the pion gun at it at various energies.

     

    Technical details

    Revision 132009-07-16 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Testing Mike's DataMixer, this time with the cmsDriver technology. Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun), such as this:

    Changed:
    <
    <
    Need to make this plot smile
    >
    >
    Though overlays look reasonable, I can't find the big energies (~10GeV) from CRAFT in the mixed output. A problem with my test? only one data event (the right one, according to the log file) was used, and there's no trace of it in the overlayed CRAFT energy. The only consistent feature added relative to the original MC is soft and in a different region of the detector.
     

    Conclusions

    Line: 101 to 103
     (*) found by (a) root macro loops over file, updates an std::map, outputs highest energy seen per tower location and the event number where that happened, (b) macro run over data and MC files, (c) python script finds locations that appear in both, and prints them sorted by energy.
    Added:
    >
    >
    • Was surprised to realize it's exactly the same data input files used in CMSSW_2. It also works for CMSSW_3.
     
    • setup:

    Revision 122009-07-16 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Testing Mike's DataMixer, this time with the cmsDriver technology.

    Changed:
    <
    <
    Warren Clarida got it working in /uscms/home/wclarida/SIM_PROGS/CMSSW_3_1_0_pre11/src.
    >
    >
    Goal was to see a plot of CRAFT energies (here, probably a muon) overlayed directly on top of simulated signal (100GeV pion gun), such as this:

    Need to make this plot smile

     

    Conclusions

    Line: 23 to 25
     cmsRun SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT_STARTUP31X.py
    Deleted:
    <
    <
     
    • Can mix existing MC and data as described below
    Added:
    >
    >
     
    mixed (HCAL towers & digis)
     
    • Computing:
      • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)
    Line: 35 to 38
      Mike: you need the latest version of EventFilter/RPCRawToDigi. Only use "mixData" as the InputTag.
    Added:
    >
    >
    Warren Clarida got it working in /uscms/home/wclarida/SIM_PROGS/CMSSW_3_1_0_pre11/src (with the GFlash option).
     Warren: The only thing that I changed was the global tag from STARTUP_31X to STARTUP31X_V1

    New package list

    Line: 77 to 82
     
    Changed:
    <
    <
     
      • dumping the information I need from the MC with dumpGun.py -->
    >
    >
     
      • dumping the information I need from the MC with dumpGun.py -->
    dumpGun.root
  • redid with 500 events MC500.root, dumpMC500.py --> dumpMC500.root
  • redid with 5K events and 50GeV pions MC5K.root ...
  •  
     
    • mixed events: hacked the cmsDriver-produced mixer config file to use the existing MC file. Details below.
      • key point: rerunning the digi step
      • dumping mixed events into my format with dumpnm.py -->
    dumpnm.root
    Added:
    >
    >

    Specific mixing test(s)

    Found (*) the location (HCAL tower) with the highest overlap between MC500 and the data file: ('25, 41', 8.9916669999999996, [84], [384])

    • Copied that data file with copyD1.py -->
    D1event.root
  • checked it's the right event with dumpD1.py -> dumpD1.root
  •  

    Technical details

    Added:
    >
    >
    (*) found by (a) root macro loops over file, updates an std::map, outputs highest energy seen per tower location and the event number where that happened, (b) macro run over data and MC files, (c) python script finds locations that appear in both, and prints them sorted by energy.
     
    • setup:
    cmsrel CMSSW_3_1_0_pre11
    Line: 134 to 152
     
      • Got an error ValueError: Step DATAMIX unknown

    -- AmnonHarel - 02 Jul 2009 \ No newline at end of file

    Added:
    >
    >
    • MC digis (test #1):

    META FILEATTACHMENT attachment="sxm5_84h441.png" attr="" comment="MC digis (test #1)" date="1247737194" name="sxm5_84h441.png" path="sxm5_84h441.png" size="32801" stream="sxm5_84h441.png" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sam5_84.gif" attr="" comment="MC tower (test #1)" date="1247737407" name="sam5_84.gif" path="sam5_84.gif" size="15653" stream="sam5_84.gif" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sxd4_384h441.png" attr="" comment="CRAFT digis (test #1)" date="1247737438" name="sxd4_384h441.png" path="sxd4_384h441.png" size="29162" stream="sxd4_384h441.png" user="Main.AmnonHarel" version="1"
    META FILEATTACHMENT attachment="sad4_384.gif" attr="" comment="CRAFT towers (test #1)" date="1247737471" name="sad4_384.gif" path="sad4_384.gif" size="17278" stream="sad4_384.gif" user="Main.AmnonHarel" version="1"

    Revision 112009-07-14 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 73 to 73
     
    • data area: /uscms/d2/aharel/
    • work area: /uscms/home/aharel/CMSSW_3_1_0_pre11
    Added:
    >
    >
     
    • setup: see below under technical details
    • non-collision data: same as in https://twiki.cern.ch/twiki/bin/view/Main/AmnonHarelDataMixingRound4a
    • simulated collisions: FullMC.root which I created from the 100GeV pion gun config as detailed below
      • key point: need to keep the digis. I just used "keep *"
      • dumping the information I need from the MC with dumpGun.py -->
  • mixed events: hacked the cmsDriver-produced mixer config file to use the existing MC file. Details below.
    • key point: rerunning the digi step
    • dumping mixed events into my format with dumpnm.py --> dumpnm.root
  • Technical details

     
    • setup:
    cmsrel CMSSW_3_1_0_pre11

    Revision 102009-07-14 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 24 to 24
     
    Changed:
    <
    <
    >
    >
    • Can mix existing MC and data as described below
     
    • Computing:
      • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)

    Revision 92009-07-12 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 108 to 108
     
     
      • take3: get cmsDriver to produce the new MC for me? cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT.root & AutoMakeMC.log
        • dumped with gdMC.py to see what's there -->
    gdMC.log --> jets are there. Digis are there (e.g. HBHEDataFramesSorted + "simHcalDigis" + "" "HLT"), but where are the recHits?
  • Variants of above, starting with the FullMC files from take2
  • Changed:
    <
    <
        • running without the generation, sim, and digi steps gave the error (nm_no_sim_no_digi_fails.log):
    ---- ProductNotFound BEGIN
    getByLabel: Found zero products matching all criteria
    Looking for type: CrossingFrame<PCaloHit>
    Looking for module label: mix
    Looking for productInstanceName: g4SimHitsHcalHits
    cms::Exception going through module DataMixingModule/mixData run: 1 event: 1
    
    >
    >
        • running without the generation, sim, and digi steps gave the error in nm_no_sim_no_digi_fails.log
     
     
        • putting the digi step back in (nm.py) - it runs! (nm.log) -->
    nm.root
    Added:
    >
    >
     
          • dumped content (gdnm.py) -->
    all the fields names now contain "MIXTEST" (chosen in nm.py), while the original are still available with "HTL" (chosen in makeFullMC.py). E.g. mixed digis are in HBHEDataFramesSorted + "hltHcalDigis" + "" "MIXTEST"
  • dumping mixed events into my format with dumpnm.py
  •  

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10

    Revision 82009-07-12 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 107 to 107
     
     
        • HCAL Digis are now in: HBHEDataFramesSorted + "hltHcalDigis" + "" "HLT"
      • take3: get cmsDriver to produce the new MC for me? cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT.root & AutoMakeMC.log
        • dumped with gdMC.py to see what's there -->
    gdMC.log --> jets are there. Digis are there (e.g. HBHEDataFramesSorted + "simHcalDigis" + "" "HLT"), but where are the recHits?
    Changed:
    <
    <
      • Variants of above, starting with the FullMC files from take2
    >
    >
      • Variants of above, starting with the FullMC files from take2
     
        • running without the generation, sim, and digi steps gave the error (nm_no_sim_no_digi_fails.log):
    ---- ProductNotFound BEGIN
    Line: 117 to 117
     Looking for productInstanceName: g4SimHitsHcalHits cms::Exception going through module DataMixingModule/mixData run: 1 event: 1
    Added:
    >
    >
     
        • putting the digi step back in (nm.py) - it runs! (nm.log) -->
    nm.root
     

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10

    Revision 72009-07-12 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 107 to 107
     
     
        • HCAL Digis are now in: HBHEDataFramesSorted + "hltHcalDigis" + "" "HLT"
      • take3: get cmsDriver to produce the new MC for me? cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT.root & AutoMakeMC.log
        • dumped with gdMC.py to see what's there -->
    gdMC.log --> jets are there. Digis are there (e.g. HBHEDataFramesSorted + "simHcalDigis" + "" "HLT"), but where are the recHits?
    Added:
    >
    >
      • Variants of above, starting with the FullMC files from take2
        • running without the generation, sim, and digi steps gave the error (nm_no_sim_no_digi_fails.log):
    ---- ProductNotFound BEGIN
    getByLabel: Found zero products matching all criteria
    Looking for type: CrossingFrame<PCaloHit>
    Looking for module label: mix
    Looking for productInstanceName: g4SimHitsHcalHits
    cms::Exception going through module DataMixingModule/mixData run: 1 event: 1
    
     

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10

    Revision 62009-07-10 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 100 to 100
     
        • notes symbolic links don't work, must place a copy of the file there AND it must end in _cfi.py, plain .py won't work frown AND probably need to run scram b again.
        • Using it instead of SinglePiPt100.cfi: cmsDriver.py ReadTest_cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
        • Can't hack it. ProductionFilterSequence must not be: missing, empty, pointing to a source, or pointing to a different EDAnlyzer (tried EventContentAnalyzer)
    Changed:
    <
    <
     
      • take2: hack the working config instead (as suggested by Mike). --> nm.py. Problems with MC file? Tried hacking it to just make the MC file -->
    makeMC.py
    >
    >
     
      • take2: hack the working config instead (as suggested by Mike). --> nm.py. Converged to config from take3 below. Problems with MC file? Tried hacking it to just make the MC file -->
    makeMC.py
    • now keeping all fields with makeFullMC.py --> FullMC.root. Dumped to gdFullMC.log: found the following
    • HCAL rechits are now in: HBHERecHitsSorted + "hltHbhereco" + "" "HLT"
    • ECAL rechits are now in: EcalRecHitsSorted + "hltEcalRegionalJetsRecHit" + "EcalRecHitsEB" "HLT"
    • HCAL Digis are now in: HBHEDataFramesSorted + "hltHcalDigis" + "" "HLT"
     
     
      • take3: get cmsDriver to produce the new MC for me? cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT.root & AutoMakeMC.log
        • dumped with gdMC.py to see what's there -->
    gdMC.log --> jets are there. Digis are there (e.g. HBHEDataFramesSorted + "simHcalDigis" + "" "HLT"), but where are the recHits?

    Revision 52009-07-09 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 100 to 100
     
        • notes symbolic links don't work, must place a copy of the file there AND it must end in _cfi.py, plain .py won't work frown AND probably need to run scram b again.
        • Using it instead of SinglePiPt100.cfi: cmsDriver.py ReadTest_cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
        • Can't hack it. ProductionFilterSequence must not be: missing, empty, pointing to a source, or pointing to a different EDAnlyzer (tried EventContentAnalyzer)
    Changed:
    <
    <
     
      • take2: hack the working config instead (as suggested by Mike). -->
    nm.py
    >
    >
     
      • take2: hack the working config instead (as suggested by Mike). --> nm.py. Problems with MC file? Tried hacking it to just make the MC file --> makeMC.py
      • take3: get cmsDriver to produce the new MC for me? cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it --> SinglePiPt100_cfi_GEN_SIM_DIGI_L1_DIGI2RAW_HLT.root & AutoMakeMC.log
        • dumped with gdMC.py to see what's there -->
    gdMC.log --> jets are there. Digis are there (e.g. HBHEDataFramesSorted + "simHcalDigis" + "" "HLT"), but where are the recHits?
     

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10

    Revision 42009-07-09 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 7 to 7
     

    Conclusions

    Added:
    >
    >
    • mixing freshly generated MC with existing data runs almost "out of the box:"
    cmsrel CMSSW_3_1_0_pre11
    cd CMSSW_3_1_0_pre11/src
    cmsenv
    addpkg Configuration/PyReleaseValidation V01-15-10
    addpkg Configuration/StandardSequences V01-17-00
    addpkg Geometry/CMSCommonData V01-05-26
    addpkg IOMC/RandomEngine V00-11-12
    addpkg SimCalorimetry/HcalSimAlgos V02-06-07
    addpkg SimGeneral/DataMixingModule V00-02-02
    scram b
    cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
    cmsRun SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT_STARTUP31X.py
    

     
    • Computing:
      • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)
    Line: 70 to 88
     
    • (Selected) error messages, all due to missing configs
      • compiing src/Configuration/StandardSequences/: GflashDigi_cfi, GflashSim_cfi, mixLowLumPU_mixProdStep1_cfi, mixProdStep2_cfi
    Changed:
    <
    <
      • compiling src/SimGeneral/DataMixingModule/python/supplementary/: !StandardSequeces. DataMixerDataOnSim_cff, StandardSequeces.DataMixerSimOnSim_cff, ...
    >
    >
      • compiling src/SimGeneral/DataMixingModule/python/supplementary/: StandardSequeces. DataMixerDataOnSim_cff, StandardSequeces.DataMixerSimOnSim_cff, ...
     
     
    • Note: Warren also has some head version of SimG4Core/GFlash, but he doesn't have EventFilter/RPCRawToDigi
    • cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP_31X::All --gflash --eventcontent FEVTDEBUG --no_exec --> ImportError: No module named GflashSim_cfi
    • cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP_31X::All --eventcontent FEVTDEBUG --no_exec --->
    produced config file SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT_STARTUP.py --> Failed to run due to unknown global tag.
    Line: 78 to 96
     

    • Controlling the inputs:
    Changed:
    <
    <
      • added package Configuration/Generator, add file ReadTest_cfi.py, that reads QCD3events.root from my data area
    >
    >
      • take1: added package Configuration/Generator, add file ReadTest_cfi.py, that reads QCD3events.root from my data area
     
        • notes symbolic links don't work, must place a copy of the file there AND it must end in _cfi.py, plain .py won't work frown AND probably need to run scram b again.
    Changed:
    <
    <
      • Using it instead of SinglePiPt100.cfi:
    cmsDriver.py ReadTest_cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
    
    >
    >
        • Using it instead of SinglePiPt100.cfi: cmsDriver.py ReadTest_cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
     
      • Can't hack it. ProductionFilterSequence must not be: missing, empty, pointing to a source, or pointing to a different EDAnlyzer (tried EventContentAnalyzer)
    Changed:
    <
    <
    >
    >
     
      • take2: hack the working config instead (as suggested by Mike). -->
    nm.py
     

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10

    Revision 32009-07-05 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Changed:
    <
    <
    Testing Mike's , this time with the new cmsDriver technology.
    >
    >
    Testing Mike's DataMixer, this time with the cmsDriver technology. Warren Clarida got it working in /uscms/home/wclarida/SIM_PROGS/CMSSW_3_1_0_pre11/src.
     

    Conclusions

    Line: 12 to 13
     

    Mike's Instructions

    Added:
    >
    >

    Addenendum

    Mike: you need the latest version of EventFilter/RPCRawToDigi. Only use "mixData" as the InputTag.

    Warren: The only thing that I changed was the global tag from STARTUP_31X to STARTUP31X_V1

    New package list

    SimGeneral/DataMixingModule V00-02-02
    Configuration/PyReleaseValidation V01-15-10
    Configuration/StandardSequences V01-17-00
    Geometry/CMSCommonData V01-05-26
    IOMC/RandomEngine V00-11-12
    

    First round of instructions

     you need to add these tags on top of a CMSSW_3_1_0_pre10
    Line: 34 to 53
     

    How I got there

    Changed:
    <
    <
    • work area: /uscms/home/aharel/CMSSW_3_1_0_pre10
    >
    >
     
    • data area: /uscms/d2/aharel/
    • work area: /uscms/home/aharel/CMSSW_3_1_0_pre11
    • setup:
    cmsrel CMSSW_3_1_0_pre11
    cd CMSSW_3_1_0_pre11/src
    cmsenv
    addpkg Configuration/PyReleaseValidation V01-15-10
    addpkg Configuration/StandardSequences V01-17-00
    addpkg Geometry/CMSCommonData V01-05-26
    addpkg IOMC/RandomEngine V00-11-12
    addpkg SimCalorimetry/HcalSimAlgos V02-06-07
    addpkg SimGeneral/DataMixingModule V00-02-02
    scram b
    
    • (Selected) error messages, all due to missing configs
      • compiing src/Configuration/StandardSequences/: GflashDigi_cfi, GflashSim_cfi, mixLowLumPU_mixProdStep1_cfi, mixProdStep2_cfi
      • compiling src/SimGeneral/DataMixingModule/python/supplementary/: !StandardSequeces. DataMixerDataOnSim_cff, StandardSequeces.DataMixerSimOnSim_cff, ...
    • Note: Warren also has some head version of SimG4Core/GFlash, but he doesn't have EventFilter/RPCRawToDigi
    • cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP_31X::All --gflash --eventcontent FEVTDEBUG --no_exec --> ImportError: No module named GflashSim_cfi
    • cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP_31X::All --eventcontent FEVTDEBUG --no_exec ---> produced config file SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT_STARTUP.py --> Failed to run due to unknown global tag.
    • cmsDriver.py SinglePiPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec ---> produced config file SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT_STARTUP31X.py --> cmsRun it -->
    SinglePiPt100_cfi_GEN_SIM_DIGI_DATAMIX_L1_DIGI2RAW_HLT.root output file and the log file SinglePiPt100.log ---> works.

    • Controlling the inputs:
      • added package Configuration/Generator, add file ReadTest_cfi.py, that reads QCD3events.root from my data area
        • notes symbolic links don't work, must place a copy of the file there AND it must end in _cfi.py, plain .py won't work frown AND probably need to run scram b again.
      • Using it instead of SinglePiPt100.cfi:
    cmsDriver.py ReadTest_cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP31X_V1::All --eventcontent FEVTDEBUG --no_exec
    
      • Can't hack it. ProductionFilterSequence must not be: missing, empty, pointing to a source, or pointing to a different EDAnlyzer (tried EventContentAnalyzer)

    Old work area

    • /uscms/home/aharel/CMSSW_3_1_0_pre10
     
     
      • setup: Didn't need the HcalSimAlgos so far.
      • running:
        • Got an error cmsDriver.py: error: no such option: --gflash --->
    trying without this option

    Revision 22009-07-02 - AmnonHarel

    Line: 1 to 1
     
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Line: 7 to 7
     

    Conclusions

    Added:
    >
    >
    • Computing:
      • cmsDriver's help messages are incomplete? (no mention of choosing sequence for a step)
     

    Mike's Instructions

    you need to add these tags on top of a CMSSW_3_1_0_pre10

    Line: 31 to 34
     

    How I got there

    Changed:
    <
    <
    • work area:
    >
    >
     
    • work area: /uscms/home/aharel/CMSSW_3_1_0_pre10
      • setup: Didn't need the HcalSimAlgos so far.
      • running:
        • Got an error cmsDriver.py: error: no such option: --gflash --->
    trying without this option
  • Looks like the config name is SinglePiPt100.cfi (Pi not Pion)
  • Got an error ValueError: Step DATAMIX unknown
  •   -- AmnonHarel - 02 Jul 2009

    Revision 12009-07-02 - AmnonHarel

    Line: 1 to 1
    Added:
    >
    >
    META TOPICPARENT name="AmnonHarelDataMixingRound9"

    Introduction

    Testing Mike's , this time with the new cmsDriver technology.

    Conclusions

    Mike's Instructions

    you need to add these tags on top of a CMSSW_3_1_0_pre10

    cvs co SimG4Core/GFlash
    cvs co -r testGflashAndDM Geometry/CMSCommonData
    cvs co -r testGflashAndDM Configuration/PyReleaseValidation
    cvs co -r testGflashAndDM Configuration/StandardSequences
    cvs co -r V02-06-07      SimCalorimetry/HcalSimAlgos
    cvs co -r V00-02-02      SimGeneral/DataMixingModule
    
    and then you can use these commands

    For DataOnSim:

    cmsDriver.py SinglePionPt100.cfi -s GEN:ProductionFilterSequence,SIM,DIGI,DATAMIX,L1,DIGI2RAW,HLT -n 10 --conditions FrontierConditions_GlobalTag,STARTUP_31X::All --gflash --eventcontent FEVTDEBUG --no_exec
    
    (all one line)

    you may still need the HcalSimAlgos.

    How I got there

    • work area:

    -- AmnonHarel - 02 Jul 2009

     
    This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
    Ideas, requests, problems regarding TWiki? Send feedback