KiraGroggLog < Sandbox

%TOC%

---++ *W->enu + Jets work log*

---+++ *2010*

---++++ *January*

---+++++ *13 Jan, 2010*

Improved the fitting code to make it slightly more streamlined.  Now only one code is needed (with a bool parameter) for running many pseudodatasets with the same W proportion, or varying the proportion of W events. %BR%
Also now taking the number of pseudo W, ttbar, QCD (Z eventually) to be generated directly from the histograms stored in the fitFile (instead of calculating separately).   Unfortunately the 091215 fitFiles do not have the correct scaling on the histograms, so that has been fixed in !WenuJetsBackgrounds.C and I am currently rerunning to get ePt>20 and ePt>25 fitFiles.
%BR%
Will want to check the starting values and ranges for the function fits to make sure they are reasonable (see how much altering them affects the final results)
Will want to look at real collision data to get an idea of QCD shapes???
Possible dataset (from https://twiki.cern.ch/twiki/bin/viewauth/CMS/FirstCollisionsAnalysis): %BR%
   * Data: /MinimumBias/BeamCommissioning09-&#8208;Dec19thReReco_341_v1/RECO	 %BR%
   * MC: /MinBias/Summer09-&#8208;STARTUP3X_V8K_900GeV-&#8208;v1/GEN-&#8208;SIM-&#8208;RECO/	%BR%
   * Mike has looked at this, and has list of good runs (according to photons): 123732, 123906,123908, 123909, 124009, 124020, 124022, 124023, 124024, 124025, 124027, 124030
Use CMSSW version ?  rereco done in 3_3_6_patch3 and 3_4_1.

---+++++ *14 Jan, 2010*

Revisiting the ABCD method and addressing some of Pedrame's comments about the matrix method plots I sent him.
Have reworked a macro, named =abcdPlots.C=.  Uses the output of =!WenuJetsBackgrounds= to plot MET vs relCombIso for W and QCD (scatter box plot); shows regions.  Results save in =abcdCanvases.root=.
Also gets the correlation information for each region, saved in =abcdCorrelations.txt=.
So far correlations look ok but not great.  New format should make it easier to check the correlations for different region boundaries.
The ABCD results are worse than ever.  And the numbers are suspect.  Need to double check how the number of W and QCD in each region is calculated.  Using counting the the code vs integrals of the plots have different results!

Answered some of the questions Pedrame had on the QCD background studies, but I think he was looking at an old file, so most of them are presumably not relevant.  Oh well.  Not sure where I'm going with this yet.  Probably need to look at the same sort of thing but as a function of jet multiplicity instead of electron pt.

---+++++ *18 Jan, 2010*

ABCD numbers are consistent, so that is no longer a problem.  The results of the method are still disappointing.  Might try making finer binning on a version of the ABCD plots so that the events can be counted from the plot (quicker to change boundaries).  Will still need to rerun to get errors? or can use newest version of root and the function =IntegralAndError()=.
Numbers for inclusive jets are better.  May need finer binning (>400 bins) if want to use plots to count events.  Have new file =abcdIntegrals.txt= to store the region counts, if I can do them this way.  May want to update the version of root on pcwiscms09.
Found a problem in =WenuJetsBackgrounds.C= where =defineObjectCanvas= method is called twice and so the inclusive plots get added twice.  Fixed in =Backgrounds= and in =Objects=.
Need better results!!!

Also adding some plots (jets and met) to /~DQM/Physics/src/!EwkElecDQM.cc that Jessie has been working on.  Not sure exactly what should be included or cut on.  So far just leading jet et and eta, number of jets, and met.

Need to make some changes to =WenuJetsBackgrounds.C= to match the equations Pedrame is looking for (see today's emails).

---+++++ *19 Jan, 2010*

Added transverse mass plots to the !EwkElecDQM code.  Now very similar to muon code.  All the plots look reasonable run over !RelValWE and !RelValZEE.

ABCD still looking bad, also for W mt.  Running Backgrounds again to get correlations for Iso vs ID variables and added the fix to Pedrame's plots (added !DeltaNotIso and !SigeeNotIso fake rates).
Made macro =abcdPlots.C= better.  Now the iso type, variable type, and ranges are parameters.
Need to try more different regions.  Pray.

Next up:  
   * look at real data
   * profile plots of iso vs id variables to find regions of lowest correlation
   * way to optimize ABCD region selection

---+++++ *20 Jan, 2010*

Profile plots of ID vs Iso -- use =TH2::ProfileX= and =TH2::ProfileY=
   * Have these, doesn't look like there is much hope.
Need to also fix the individual ID fake rate plots for Pedrame (code isn't working right, no FR show up... and also need different plots for barrel/endcap)
   * Have divide histograms problem; can do after the fact for now.


To do:
   * Set up CMSSW_3_3_6 area and run over christos' files
   * Set up place to run on real data (also 3_3_6) -- will need to modify code to not look at MC truth...
   * Want print out of ABCD results table (esp the effect on the signal--optimize on this???)


---+++++ *21 Jan, 2010*

Set up CMSSW_3_3_6_patch3, tried compiling ntuple making code, but got many errors.  A lot of the include files have problems...should not include? or missing the correct version?
Possibly this problem is from a lack of a !BuildFile...yes, yes it is.  No more compile troubles, it's a miracle.  
So, when everything seems to fail compiling...make sure there is a !BuildFile.

Changes to gen jet (can use pat jets to get gen jets or gen jet collection).

To do:
   * Run on real data sets using same .py (may need to set all MC truth to False)


---+++++ *22 Jan, 2010*

All farmed jobs have finished running!  Doing some of the !BCtoE samples (Christos' 30-80 sample hasn't finished yet).
Merging the files that are done.

Running on data is not simple.  Need to remove all MC references.  PF2PAT has data version, but using it with PAT and my code is not trivial.

---+++++ *25 Jan, 2010*

Met with Pedrame.  Have more plots to make. Same as before but as function of # jets rather than electron pt.
Also more 2D plots of all Isolation variables vs all ID variables.
Put MET cut on inverted ID & inverted Isolation plot -- will compare effect to same cut on data as a double check that it has a similar effect.
Go through the whole process to get number of QCD predicted with the method (as a function of pt) and compare to the actual in the MC.  Get final, total number, like ABCD method?  Compare.
--Almost finished

Will need new HLT efficiency numbers for 3_3_6 MC.  Finish merging all files and start using.


---+++++ *26 Jan, 2010*

For ABCD method, do I want to treat all the MC (QCD+EWK) as "data" and then try to calculate the number of QCD in region A?  And compare to just the actual QCD number in region A?  This is obviously going to give worse results because there is some EWK contamination in regions B, C, & D.

Commented out MC truth references in python files in producers:
CMSSW_3_3_6/src/PhysicsTools/PatAlgos/python/producersLayer1/*.cff
Also set various match options to =False= in the corresponding .cfi files.

---+++++ *29 Jan, 2010*

ABCD still not good with 3_3_6_patch3.  

Have system that runs on data, using 3_3_6_patch3.  
Mostly just needed this line: %BR%
=removeMCMatching(process, 'All')=
Still need HLT, trigger bits, run number information to run on data.

Sridhara requests an abstract for W+jets to be give at DIS in April.  Below a generic one for Ws. %BR%
Abstract %BR%
The production of W bosons has been observed in pp collisions at a center-of-mass energy of 7 TeV using data collected in the CMS experiment. Events were selected containing an isolated, energetic electron or muon. The presence of an energetic neutrino is demonstrated using the distribution of missing transverse energy (MET), which is calculated from calorimetric and tracking information in three ways. Data-driven methods are used to estimate reconstruction and triggering efficiencies, and well as the main backgrounds. The yield is NNN in the electron channel, and MMM in the muon channel, for an integrated luminosity of LLLL pb-1. Kinematic distributions for the signal are shown.

My Abstract %BR%
The production of W bosons as well as jets have been observed in pp collisions at a center-of-mass energy of 7 TeV using data collected in the CMS experiment. Events were selected containing an isolated, energetic electron or muon. The presence of an energetic neutrino is demonstrated using the distribution of missing transverse energy (MET), which is calculated from calorimetric and tracking information in three ways. Jets have also been produced independently and studying them will give us insight into processes involving W bosons produced in association with jets.  Data-driven methods are used to estimate reconstruction and triggering efficiencies, and well as the main backgrounds. The yield is NNN in the electron channel, and MMM in the muon channel, for an integrated luminosity of LLLL pb-1. Kinematic distributions for the signal are shown.

Notes
   * It may make sense to have separate talks for the electron and muon channel.
   * One could have a talk dedicated to the MET measurement.
   * Details such as the energy and luminosity are subject to change, of course.
   * It might not be possible to measure efficiencies from the data - depends on the luminosity and the quality of the data. 

---+++++ *03 Feb, 2010*

Couple of the job submissions for running on data and related MC:
<verbatim>
farmoutAnalysisJobs --input-files-per-job=5  --input-dbs-path=/MinimumBias/BeamCommissioning09-Dec19thReReco_336p3_v2/RECO minBiasData336 ~/CMSSW_3_3_6_patch3/ ~/CMSSW_3_3_6_patch3/src/UserCode/grogg/test/AOD2Pat2Ntuple-Data.py
farmoutAnalysisJobs --input-files-per-job=5  --input-dbs-path=/MinBias/Summer09-STARTUP3X_V8P_900GeV-v1/GEN-SIM-RECO minBiasMC336 ~/CMSSW_3_3_6_patch3/ ~/CMSSW_3_3_6_patch3/src/UserCode/grogg/test/AOD2Pat2Ntuple-MC.py
farmoutAnalysisJobs --input-files-per-job=5  --input-dbs-path=/MinimumBias/BeamCommissioning09-Jan23ReReco-v1/RECO minBiasData336-Jan23 ~/CMSSW_3_3_6_patch3/ ~/CMSSW_3_3_6_patch3/src/UserCode/grogg/test/AOD2Pat2Ntuple-Data.py
</verbatim>
<verbatim>
mergeFiles --copy-timeout=160 --use-hadd minBiasMC336.root /pnfs/hep.wisc.edu/store/user/grogg/minBiasMC336-AOD2Pat2Ntuple-MC/ >& log&
mergeFiles --copy-timeout=160  --use-hadd minBiasData336.root /pnfs/hep.wisc.edu/store/user/grogg/minBiasData336-AOD2Pat2Ntuple-Data/ >& log &
mergeFiles --copy-timeout=160  --use-hadd minBiasData336-Jan23.root /pnfs/hep.wisc.edu/store/user/grogg/minBiasData336-Jan23-AOD2Pat2Ntuple-Data/ >& log&
</verbatim>

---+++++ *04 Feb, 2010*

My Abstract %BR%
The cross section and kinematic distributions of W bosons produced in association with jets are an essential early measurement at the LHC.  The production of W bosons as well as jets have been observed in pp collisions at a center-of-mass energy of 7 TeV using data collected in the CMS experiment. The W + jets signature is characterized by an isolated, energetic electron or muon, and an energetic neutrino.  The presence of the neutrino is demonstrated using the distribution of missing transverse energy (MET), which is calculated from calorimetric and tracking information. Jets have also been produced and studied independently.  Data-driven methods are used to estimate reconstruction and triggering efficiencies, and well as the main backgrounds. The yield is NNN in the electron channel, and MMM in the muon channel, for an integrated luminosity of LLLL pb-1. 

---+++++ *05 Feb, 2010*

Current Summer09, CMSSW_3_3_6 files do not have PF MET stored in the patTuples or Ntuples. CaloMET does not work well with ABCD method -- too much spread.
Christos will rerun with pfMET added and some other trigger and jet improvements.  Cannot asses background methods until files are done running.

May want to look at more at minbias data/MC in the mean time.  Will need to update the AOD2Pat2Ntuple-Data.py and MC.py for the new version of the ntuple maker that christos has set up, and include jet/met correction info.  HLT might be a little tricky.

---+++++ *21 Mar, 2010*

For the record, the error below came from trying to run 3_5_4 preproduction file on CMSSW_3_5_2.  Moved to 3_5_4, adjusted the build file for changes in helper files, and it works!
<verbatim>
%MSG-s CMSException:  AfterFile 21-Mar-2010 12:24:44 CET BeforeEvents
cms::Exception caught in cmsRun
---- FatalRootError BEGIN
Fatal Root Error: @SUB=TClass::New with placement
cannot create object of class edm::EDProduct version -1 at address 0xd423378
cms::Exception caught in EventProcessor and rethrown
---- FatalRootError END
</verbatim>

Before changing build file, i got this error:
<verbatim>
tmp/slc5_ia32_gcc434/src/UserCode/DQMStandaloneEwkElectron/src/UserCodeDQMStandaloneEwkElectron/libUserCodeDQMStandaloneEwkElectron.so: undefined reference to `TH1AddDirectorySentry::~TH1AddDirectorySentry()'
</verbatim>
Note - undefined reference --> something missing from build file


---++ *RCT work log*

---+++ *2010*

---++++ *January*

---+++++ *14 Jan, 2010*

DCS: All RMCs offline from 10:33 to 10:37 today.  Cause unknown, possible network glitch.  All of the rack information updated one hour later (likely intervention) %BR%
=OFFLINE because Comtrol OFFLINE, Symptom: totalDelay above limit in sendCommand() for RMCX= %BR%
RMC7 went slightly before the others, sent alert to email -- probably because RMC7 was turned on first, so the RMC7 script is the first to try to communicate. %BR%
Should OFFLINE alerts use !SMS? %BR%
FSM was also in "taken" mode but seems to be available now.  Possibly a hung session of mine on the terminal server? %BR%
Unable see (or acknowledge) alerts in the alarm panel, have resent savannah request.  Even the remote UI isn't working.

---+++++ *15 Jan, 2010*

DCS:  Adding new SMS alert:  When RMC is offline for more than X minutes (X is configurable), then =extendedOffline= datapoint goes true, and alert is sent.  X should be about 10 minutes, and it is configurable as the number of reconnection attempts to make (so 6 is a reasonable number for 90 sec reconnect frequency -- 6*90 = 540). %BR%
Made an entirely new project to test the new dplists and alert postInstall script.  Had a little trouble with the correct framework files (fwConfig file was had an unknown function that didn't allow the alerts to be set up).  Forgot to put in the correct dp for the new offline alert, so this was a good idea.  
Stuff for component 4.3.0:
   * dplist
      * RDBArchive -- alarm handling now uses nack alert (so it doesn't have to be acknowledged)
      * RMCspecific -- added =extendedOffline= datapoint for RMC and =attempts_before_offlineAlert= datapoint for RMClog; reset RMC1 COMport and xsums to correct values
   * msg
      * myError.cat -- cleaned up error messages, added one for installation messages
   * panels
      * !CMSfwRCT_frequencies.pnl -- added option to set =attempts_before_offlineAlert=
      * !CMSfwRCT_trendOfflineReason.pnl -- fixed text display
   * scripts
      * !CMSfwRCT_alertSystem.postInstall -- added longOffline alert
      * !CMSfwRCT_monitoring.ctl -- added dpSet of extendedOffline and corresponding 'time' parameter
      * !CMSfwRCT_RDB.postInstall -- changed error message number
      * !FwFsmTree.postInstall -- changed error message number

---+++++ *24 Jan, 2010*

Ran a pattern test, but Rack 2 is OFF (power supply problems), which means the SC in the rack is also off, which led to several of the crates being unavailable or giving strange results.  The unaffected crates did not show any errors.  Will try again after repairs by Pam and Jessie.  

---+++++ *29 Jan, 2010*

Need to start a conditions database for !RCT DCS.  Emailed Bobby asking for a meeting or documentation.  Haven't head back yet.  Instead made of list of what might need to go into the DB (in notebook) 

---+++++ *04 Feb, 2010*

Added more extensive documentation to RCT DCS Software twiki regarding Access Control and Alert Notifications: %BR%
https://twiki.cern.ch/twiki/bin/view/CMS/RCTDCSSoftware#Setting_up_Access_Control


---+++++ *08 Feb, 2010*

Found a bug in NCrateAnalyzerClass.cc.  When the header is found in event < 9, the jet/sums give a segfault.  This is because the jet/sums have an offset of -9, so it tried to find a negative location in the vector.  This was corrected in another section of the code (if the header isn't found in one ofthe crates) by adding a +64 to all the iterator begins.  It seems we were just lucky this didn't show up before.  It has been corrected.

---+++++ *12 Feb, 2010*

DCS:  A power test this morning turned off many computers, DCS included.  When recovering CMS-TRG-DCS-01 tried to start first, and then prevented CMS-TRG-DCS-02 from starting.  Bobby chose STOP_ALL on 01 which allowed 02 to start up properly.  This should all be automatic.  Investigating...
Some of the log messages:
<verbatim>
PVSS00pmon   (1), 2010.02.12 10:08:49.609, SYS,  INFO,        1, Manager Start, START_MODE

PVSS00pmon   (1), 2010.02.12 10:08:53.843, SYS,  INFO,        1, Manager Start, C:\ETM\PVSS2\3.8\bin\PVSS00data.exe -PROJ ServiceProject -pmonIndex 1 

PVSS00data   (0), 2010.02.12 10:08:54.125, SYS,  INFO,        1, Manager Start, PROJ, ServiceProject, V 3.8 - 3.8  SP1 linked at Apr  2 2009 04:48:28

PVSS00data   (0), 2010.02.12 10:08:54.125, REDU, INFO,        3, Trying to connect to, Trying to connect to (SYS: 0 Data -num 0 CONN: 2) @ cms-trg-dcs-01 (may take some time if host is unreachable)

PVSS00data   (0), 2010.02.12 10:08:54.140, SYS,  INFO,        4, Connected to, (SYS: 0 Data -num 0 CONN: 2) @ cms-trg-dcs-01 (10.176.8.229)

PVSS00data   (0), 2010.02.12 10:08:54.140, SYS,  WARNING,   184, The time on cms-trg-dcs-01 and cms-trg-dcs-02 differs approximately 23 sec. Please correct this.

PVSS00data   (0), 2010.02.12 10:08:54.140, REDU, INFO,        0, , Sending recovery request to other replica

PVSS00data   (0), 2010.02.12 10:08:54.203, REDU, INFO,        0, , Other replica temporarily unable to recover us, trying again later

PVSS00data   (0), 2010.02.12 10:08:54.281, SYS,  INFO,      181, Closing connection to (SYS: 0 Data -num 0 CONN: 2)

PVSS00data   (0), 2010.02.12 10:09:56.281, SYS,  INFO,        2, Manager Stop

PVSS00pmon   (1), 2010.02.12 10:09:56.312, SYS,  SEVERE,     10/pmon, The central server manager PVSS00data(0) stopped! Restarting the project

PVSS00pmon   (1), 2010.02.12 10:09:56.468, SYS,  INFO,       14/pmon, Completely stopped the project - restarting it
</verbatim>
This set of messages repeats several times until Bobby intervened.  



%BR%

-- Main.KiraGrogg - 14-Jan-2010
Topic revision: r26 - 2010-03-21 - unknown
Account
- Log In
- Cern Search
- TWiki Search
- Google Search
Sandbox All webs
Edit
Attach
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback