Giulio's Notes about programming

  • Bash
  • C++
  • Python
  • MadGraph
  • Matematica
  • Helpful links
  • Bash

    Command description
    chmod +x fileName.sh compile the code and make it executable
    ./fileName.sh execute fileName.sh
    source fileName.sh execute as source code (equivalent to chmod +x and ./ but as source)

    Basic Bash Commands

    Command description
    echo something print something. It is equivalent to echo "something"
    read VAR equivalent to c++ std::cint. It read VAR as "VAR"

    Variables

    Command description
    ${VAR:-pippo} use the variable VAR. If VAR has not been declared, use pippo
    ${VAR:=pippo} use the variable VAR. If VAR has not been declared, set VAR = pippo
    shift delete $1 Record $2 as $1, record $3 as $2, etc...
    $@ list all the position parameters
    $# return the number of position parameters
    $IFS" Internal Field Separator. It is the char(letter) that divide different parameters in reading (look the example below).

    Array

    Declarization is initializazione and can be given in three ways:
    arr=([2]="Hello" [3]="World!")
    arr=("I" "am" "a" "Programm")
    
    Command description
    unset arr[1] delete the element arr[1]. The other elements will keep the position
    ${array[@]:1:2} delete the element arr[1]. The other elements will keep the position
    ${#array[@]} delete the element arr[1]. The other elements will keep the position

    Some simple bash programmes

    To find what tree is missing:
    for i in $(seq 0 1 333); do ls skimmed_tree_$i.root; done | grep cann
    

    There is a IFS manipolation example here:

    #!/bin/bash
    old_IFS="$IFS"
    IFS=:
    echo "Inserisci tre dati separati da due punti ..."
    read x y z
    IFS=$old_IFS
    echo "x  $x y  $y z  $z"

    If you want to enlist some files with a space in between:

    `ls ${path}/tree_*.root`
    python macro.py `ls ${path}/tree_*.root`
    

    C++

    TLorentzVector TLV;
    TLorentzVector TLV;
    
    TLV.SetPxPyPzE(px, py, pz, e)
    TLV.SetPtEtaPhiM(pt, eta, phi, m)
    
    theta = TLV.Theta();
    cost = TLV.CosTheta();
    phi = TLV.Phi();
    eta = TLV.PseudoRapidity () 
    m2 = TLV.Mag2()         = TLV.M2();
    m  = TLV.Mag()           = TLV.M();
    s  = TLV1*TLV2;           = v1.Dot(v2);     
    
    TVector3 v = TLV.Vect()
    

    Reading and writing files

    #include <stdio.h>
    int main(int argc, char **argv) {
    FILE *in, *out;
    int c;
    in = fopen("input.txt", "r");
    out = fopen("output.txt", "w");
    while ((c = fgetc(in)) != EOF) {
    fputc(c, out);
    }
    fclose(out);
    fclose(in);
    }

    atoi

    #include <stdlib.h>     /* atoi is in stdlib.h */
    int i;
    char c = "1";
    i = atoi (c);

    ofstream

    The official page is here

    ofstream out_name;
    out_name.open("out_name.txt"); 
    out_name << std::setprecision(8) << histos[0]->Integral(0,histos[0]->GetNbinsX()+1) << endl;
    out_name.close();

    String

    A page is here. You have to include:
    #include <string> 
    
    Command description
    str.find(str2) return the position of the first character of the first match. If no matches were found, the function returns string::npos.
    list.insert(6,"Golf") insert "Golf" as list[6] and shift the following

    TLorentzVector

    For any doubt about Lorentz Vector there is the official page . You have to include
    #include "TLorentzVector.h"
    

    Most used commands of TLorentzVector

    TLorentzVector TLV;
    TLorentzVector TLV;
    
    TLV.SetPxPyPzE(px, py, pz, e)
    TLV.SetPtEtaPhiM(pt, eta, phi, m)
    
    theta = TLV.Theta();
    cost = TLV.CosTheta();
    phi = TLV.Phi();
    eta = TLV.PseudoRapidity () 
    m2 = TLV.Mag2()         = TLV.M2();
    m  = TLV.Mag()           = TLV.M();
    s  = TLV1*TLV2;           = v1.Dot(v2);     
    
    TVector3 v = TLV.Vect()
    

    Boost a TLorentzVector

    TLorentzVector MyParticle1(P1_PX, P1_PY, P1_PZ, P1_E);
    TLorentzVector MyParticle2(P2_PX ,P2_PY, P2_PZ, P2_E);
    
    TLorentzVector Particle_CM = MyParticle1 + MyParticle2;
    
    TVector3 Particle_CM_BoostVector = Particle_CM.BoostVector();
    TLorentzVector MyParticle1_newSys = MyParticle1_CMS;
    MyParticle1_newSys.Boost(Particle_CM_BoostVector);

    Python

    You can find everything about python here

    import

    Command description
    import somefile Everything in somefile.py gets imported. To refer to something in the file, append the text somefile. to the front of its name.
    from somefile import * Everything in somefile.py gets imported. To refer to anything in the module, just use its name. Every-thing in the module is now in the current namespace. NB Using this import command can easily overwrite the definition of an existing function or variable!
    from somefile import className Only the item className in somefile.py gets imported. After importing className, you can just use it without a module prefix. Its brought into the current namespace. NB Overwrites the definition of this name if already defined in the current namespace!

    The help() and dir() builtins allow you to check what you have included so far and how to use it ( notice the __ of modules imported by default)

    dir() [-5:] #enlist the last 5 variable declared
    help(math.sqrt) #describe math.sqrt
    

    Basic commands

    Some common commands are:
    Command description
    print(a, b) print a and b
    a = input("Enter a string: ") print "Enter a string: " and inizialize a
    3//2 equivalent to int(3./2.)
    3%2 remainder of the division

    Every word can be used except for :
    and, as, assert, break, class, continue, def, del, elif, else, except, exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while, with, yield

    Builtins

    Command description
    type(object) return the type of object
    range(start,end,step) return integers from start to end. start=0, step=1 are default
    enumerate(N) range(1,N,1)
    sum([1,2,3,4.]) return 10.0
    abs(x) absolute value
    sorted([5, 2, 3, 1, 4]) return [1, 2, 3, 4, 5], a useful page about sorting is here
    sorted(iterable, key=None, reverse=False) the iterable can be sorted with a key function and in ascending or descending order
    map(function_to_apply, list_of_inputs) it returns a list. the elements of list_of_inputs are passed to function_to_apply one by one
    filter(function_to_apply, list_of_inputs) creates a list of list_of_inputs'elements for which function_to_apply returns true

    from functools import reduce
    product = reduce((lambda x, y: x * y), [1, 2, 3, 4])
    

    Write in a text file

    outputFile = "source.txt"
    outputName = open(outputFile, 'w')                   # 'w' = rewrite  ,       'a' = append
    print >> outputName , " variable \t",  x
    outputName.close()  

    String

    "qwe"+"rty" == "qwerty"
    "qwe"*3 == "qweqweqwe"
    mystring = "qwerty"  ,   mystring[3] == "r"
    
    Let mystring be a string
    Command description
    mystring.capitalize() return mystring with the first letter capitalized
    mystring.replace(" ","++") replace each " " with "++" in mystring
    mystring.lower() return mystring with all the letters tined
    mystring.upper() return mystring with all the letters capitalized
    mystring.isalpha() True if there are only alphabetic char
    mystring.isdigit() True if there are only numerical char
    mystring.islower() True if there are only tiny letters
    mystring.isspace() True if there are only spaces e.g. " ".isspace() = true
    mystring.isupper() True if there are only capital letters
    mystring.count("test") count how many "test" there are in mystring
    mylist = list(mystring) return a list made of the letters of mystring
    mystring.join(sequence)  
    mystring.startswith("x") return True if the string begin with "x"
    mystring.endswith("x") return True if the string end with "x"
    mystring.find()  
    mylist = mystring.split() return a list of strings. Each string in the list is a word of mystring
    mylist = mystring.split("x") return a list of strings. Strings in the list are pieces of mystring separated by "x"

    Lists

    L = []                         #or    L = list()
    list = range(0,4)
    list + ["a", "b", "c"] == [0, 1, 2, 3, 'a', 'b', 'c']
    list[2] == 2 
    
    Command description
    list.append("Golf") add "Golf" as element in the end
    list.insert(6,"Golf") insert "Golf" as list[6] and shift the following
    list.len() number of element in the list
    list[start:stop:step] enlist as range() do. Defaults are: start = 0, stop = -1, step = 1
    list [-1::-1] reverse the list
    list.sort() sort the elements
    list.pop() return the lastelement of list and delete it from the list

    Tuples

    A tuple is a sequence object like a list or a string. Its constructed by grouping a sequence of objects together with commas, either without brackets, or with parentheses. At variance with list, tuples are immutable, they dont have append, insert or pop methods
    T = ()                         #or    T = tuple()
    t  = (0,1,2,3)
    t + ("a", "b", "c") == [0, 1, 2, 3, 'a', 'b', 'c']
    t[2] == "2"  
    
    You cannot set
    t[2] = 2
    

    Dictionaries

    D=dict();     D1= {};     D2={0:"q",1:"w"}           # 3 different declaration and initialization
    D['A']   = 2.1
    D['ab'] ='proline'
    D[1]    = 2.1
    "A" in D == True
    
    Command description
    dict.keys() enlist all the keys
    dict.values() enlist all the values
    del dict["A"] delete element with key "A"
    dict.items()( enlist all objects in couple (key,value)
    dict.values( enlist all the values
    dict.values( enlist all the values
    dict.values( enlist all the values

    Loop over a dictionary

    for k,val in dict.items() :
        if k == WantedKeys :
            V = val

    Set

    Command description
    %run script.py execute script.py
    myset = set(not_a_set) return a set with the element of la list, a tuple or a string
    set1.union(set2) return the set union. It does not modify set1
    set1.intersection(set2) return the set intersection. It does not modify set1
    set1.difference(set2) return the set difference. It does not modify set1
    set1.symmetric_difference(set2) return the set symmetric difference. It does not modify set1

    Regular expressions

    Interactive Python (IPython)

    Command description
    %run script.py execute script.py
    %reset clear all namespace
    %hist print history
    %xdel delete variables
    %who list objects in enviroment (use in combo with ? to know who is who)

    Reading and writing files

    Command Description
    open(filename, mode) r (Read), w (Write), a (Append). If a file opened for w does not exist it will be created
    f.readline() read a single line from a file. Returns a string.
    f.readlines([size]) read all the lines up to size bytes and return them in a iterator of s
    f.read([size]) read up to size bytes; returns a string
    f.write(text) write text to file
    This is an exmple of how write a file into an other
    in = open("input.txt", "r")
    out = open("output.txt", "w")
    out.writelines.(in)
    in = close()
    out = close()
    
    or
    open("output.txt", "w").writelines.(open("input.txt", "r"))
    
    You can also write silngle lines
    out.writelines.("This is line 1\nThis is line 2")
    

    Time

    import time
    
    start_time = time.time()
    do something ....
    timeElapsed = time.time() - start_time
    

    Numpy

    import numpy as np
    
    Command Description
    a=np.array([(1,2,3,4),(5,6,7,8)])  
    a.size  
    a.shape return the shape (the dimansions)
    a=a-10; a=a*2 operations on numpy array
    a.T transpose. Equivalent to a.transpose()
    np.max(a) max. Also a.max()
    np.mean(a) mean. Also a.mean()
    np.mean(a[0]) mean on the first row
    a.std() max
    b=a[1].copy() make a copy. If you reassign b, a will not change
    unionArray = np.concatenate((a1,a2,a3)) merge several arrays

    Creation of numpy array

    np.zeros((3,3),'d') 3x3 numpy array made of 0
    np.ones((3,3)) 3x3 numpy array made of 1
    np.eye(3) 3x3 identity
    np.arange(1,10) array([1, 2, 3, 4, 5, 6, 7, 8, 9])
    np.arange(2.3,3.3,.1) array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2]) 0.1is the space
    np.linspace(2.3,3.2,10)) array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2]) 10 is the number of elements
    a=np.array(list(map(bool,(0,1,0,1,1,0,1)))) a=np.array(list(map(bool,(0,1,0,1,1,0,1))))
    np.arange(2.3,3.3,.1) array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2])
    np.arange(2.3,3.3,.1) array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2])

    If "table.txt" contains

    0 1 2
    3 4 5
    6 7 8
    
    Then you can read it and create a numby array simply by:
    np.loadtxt("table.txt")
    

    How to plot in jupyter notebook

    %matplotlib inline
    import numpy as np
    import matplotlib.pyplot as plt
    
    npoints = int(10e2)
    x = np.random.rand(npoints)
    y = np.random.rand(npoints)
    
    plt.scatter(x,y,marker='o',s=10,color='r')
    
    colorInNumpyArray= np.ones(1000)
    plt.scatter(x,y, c = colorInNumpyArray)   

    Skipy

    import scipy as sp
    import sklearn
    from sklearn import datasets
    import sklearn.preprocessing
    from sklearn.decomposition import PCA
    

    Creation of

    from sklearn import datasets
    import sklearn.preprocessing
    from sklearn.decomposition import PCA
    from sklearn.cluster import KMeans
    

    Deep Learning

    Supervised Learning

    %matplotlib inline
    import sklearn
    from sklearn import datasets
    
    iris = datasets.load_iris()
    X = iris.data       #data
    Y = iris.target     #true
    iris.target_names         #it shows the names the number in target mean
    
    
    from sklearn.svm import LinearSVC
    clf = LinearSVC()
    clf.fit(X,Y)
    
    X_new = [[1,2,3,4]]
    X_new = np.asarray(X_new)
    clf.predit(X_new)
     
    
    iris = datasets.load_iris()
    X = iris.data       #data
    Y = iris.target     #true
    iris.target_names         #it shows the names the number in target mean
    
    
    from sklearn.svm import linearSVC
    clf = linearSVC()
    clf.fit(X,Y)
    
    X_new = [[1,2,3,4]]
    X_new = np.asarray(X_new)
    clf.predict(X_new)
     
    iris = datasets.load_iris()
    X = iris.data       #data
    Y = iris.target     #true
    iris.target_names         #it shows the names the number in target mean
    
    
    from sklearn.svm import linearSVC
    clf = linearSVC()
    clf.fit(X,Y)
    
    X_new = [[1,2,3,4]]
    X_new = np.asarray(X_new)
    clf.predit(X_new)
     

    Unsupervised Learning

    There are many methods, one of them is Principle Component Analysis (PCA).
    Here there is an example of unsupervised learning using KMeans.
    %matplotlib inline
    
    from sklearn.cluster import KMeans
    import matplotlib.pyplot as plt
    
    from sklearn import datasets
    iris = datasets.load_iris()
    X = iris.data       #data
    Y = iris.target
    
    kmea = KMeans(n_clusters = 2)    # 'n_clusters' is the number of cluster you will have
    kmea.fit_predict(X)
    
    plt.scatter(X[:,0], X[:,1], c=kmea.labels_)

    Here there is an example of unsupervised learning using DBSCAN.

    %matplotlib inline
    import matplotlib.pyplot as plt
    from sklearn.cluster import DBSCAN
    
    data = some data in a numpy array
    
    estimator = DBSCAN(eps=0.35, min_samples=12,metric="euclidean")
    clusters = estimator.fit_predict(data)
    print(set(clusters))
    plt.scatter(data[clusters==0,0],data[clusters==0,1],c="r")
    plt.scatter(data[clusters==1,0],data[clusters==1,1],c="b",alpha=0.5)
    plt.axis("equal")
    

    MadGraph

    First you have to say the events
    generate p p > mu+ mu-
    generate p p > z > mu+ mu- QED=2 QCD=0
    generate p p > z h , z > mu+ mu- , (h > z l+ l-, z > l+ l-)
    generate p p > mu+ mu- /a                                                         if you don't want to have photon in the diagrams
    
    Adding other source is possible
    add process p p > b b~
    
    Then you must create the folder
    output folderName
    
    Then you launch madgraph
    launch folderName
    

    Display

    Command description
    display particles return the names al all trhe particel in the model
    display multiparticles return the name of the sets of particel (e.g. l = e,mu)
    display diagrams draw feymnan dyagrams
    display processes show the process from inital to final states

    Set parameters without editing the cards

    set ebeam1 65000
    set ebeam2 65000
    set MT 172.
    done
    

    Event generation in CMSSW

    In this page and this other page there are the instruction to generate events with CMSSW environment.

    First step: GridPack production

    At first, without set cms environment:

    git clone http://github.com/cms-sw/genproductions.git genproductions
    cd genproductions/bin/MadGraph5_aMCatNLO/
    ./gridpack_generation.sh <name of process card without _proc_card.dat> <folder containing cards relative to current location> <queue>
    ./gridpack_generation.sh DYJets_4f_LO cards/examples/DYJets_4f_LO local
    

    gridpackTest (you have to do cmsenv)

    tar -xavf <path of gridpack creation>/DYJets_4f_LO_tarball.tar.xz
    bash
    ./runcmsgrid.sh <NEvents> <RandomSeed> <NumberOfCPUs>
    

    lhe to root without hadronization

    In a new area (but the same CMS release):
    cmsenv
    git cms-addpkg GeneratorInterface/LHEInterface
    curl -s --insecure http://cmsdoc.cern.ch/~vciulli/CMSDASBari/run_generic_tarball_cvmfs.sh --retry 2 --create-dirs -o GeneratorInterface/LHEInterface/data/run_generic_tarball_cvmfs.sh
    chmod u+x  GeneratorInterface/LHEInterface/data/run_generic_tarball_cvmfs.sh
    curl -s --insecure http://cmsdoc.cern.ch/~vciulli/CMSDASBari/wellnu012j_5f_LO_MLM_7TeV-fragment.py --retry 2 --create-dirs -o Configuration/GenProduction/python/wellnu012j_5f_LO_MLM_7TeV-fragment.py 
    scram b
    cd $CMSSW_BASE
    mkdir work
    cd work
    
    I renamed wellnu012j_5f_LO_MLM_7TeV in DYJets_4f_LO_MLM_13TeV. Put the right gridPack in the configuration file Configuration/GenProduction/python/wellnu012j_5f_LO_MLM_7TeV-fragment.py and
    cmsDriver.py Configuration/GenProduction/python/DYJets_4f_LO_MLM_13TeV-fragment.py --fileout file:DYJets_4f_LO_MLM_13TeV.root --mc --eventcontent LHE --datatier GEN --conditions auto:mc --step LHE --python_filename DYJets_4f_LO_MLM_13TeV_1_cfg.py -n 1000
    

    Second step: hadronization

    Now you should generate hadronization. First thing to di is to copy the configuration file you need from here and put it in Configuration/GenProduction/python/. Then, in a new area (but the same CMS release):
    cd $CMSSW_BASE/src
    scram b
    cd $CMSSW_BASE/work
    
    if the config file is
     Configuration/GenProduction/python/Hadronizer_DYJets_4f_LO_MLM_13TeV_qcut20_cff.py 
    :
    cmsDriver.py Configuration/GenProduction/python/Hadronizer_DYJets_4f_LO_MLM_13TeV_qcut20_cff.py --fileout file:DYJets_4f_LO_MLM_13TeV_qcut20_GEN.root --mc --eventcontent RAWSIM --datatier GEN-SIM --conditions auto:mc --beamspot NominalCollision2015 --step GEN --python_filename DYJets_4f_LO_MLM_13TeV_qcut20_GEN_1_cfg.py --filein file:DYJets_4f_LO_MLM_13TeV.root -n 1000
    

    Matematica

    Helpful links

    Links accross the page: Others:

    -- GiulioMandorli1 - 2017-03-23

    Edit | Attach | Watch | Print version | History: r36 < r35 < r34 < r33 < r32 | Backlinks | Raw View | WYSIWYG | More topic actions
    Topic revision: r36 - 2018-09-12 - GiulioMandorli1
     
      • Cern Search Icon Cern Search
      • TWiki Search Icon TWiki Search
      • Google Search Icon Google Search

      Sandbox All webs login

    This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
    Ideas, requests, problems regarding TWiki? Send feedback