Giulio's Notes about programming

#!/bin/bash
old_IFS="$IFS"
IFS=:
echo "Inserisci tre dati separati da due punti ..."
read x y z
IFS=$old_IFS
echo "x è $x y è $y z è $z"

If you want to enlist some files with a space in between:

`ls ${path}/tree_*.root`
python macro.py `ls ${path}/tree_*.root`

C++

TLorentzVector TLV;
TLorentzVector TLV;

TLV.SetPxPyPzE(px, py, pz, e)
TLV.SetPtEtaPhiM(pt, eta, phi, m)

theta = TLV.Theta();
cost = TLV.CosTheta();
phi = TLV.Phi();
eta = TLV.PseudoRapidity () 
m2 = TLV.Mag2()         = TLV.M2();
m  = TLV.Mag()           = TLV.M();
s  = TLV1*TLV2;           = v1.Dot(v2);     

TVector3 v = TLV.Vect()

Reading and writing files

#include <stdio.h>
int main(int argc, char **argv) {
FILE *in, *out;
int c;
in = fopen("input.txt", "r");
out = fopen("output.txt", "w");
while ((c = fgetc(in)) != EOF) {
fputc(c, out);
}
fclose(out);
fclose(in);
}

atoi

#include <stdlib.h>     /* atoi is in stdlib.h */
int i;
char c = "1";
i = atoi (c);

ofstream

The official page is here

ofstream out_name;
out_name.open("out_name.txt"); 
out_name << std::setprecision(8) << histos[0]->Integral(0,histos[0]->GetNbinsX()+1) << endl;
out_name.close();

String

A page is here

. You have to include:

#include <string>

Command	description
str.find(str2)	return the position of the first character of the first match. If no matches were found, the function returns string::npos.
list.insert(6,"Golf")	insert "Golf" as list[6] and shift the following

TLorentzVector

For any doubt about Lorentz Vector there is the official page

. You have to include

#include "TLorentzVector.h"

Most used commands of TLorentzVector

TLorentzVector TLV;
TLorentzVector TLV;

TLV.SetPxPyPzE(px, py, pz, e)
TLV.SetPtEtaPhiM(pt, eta, phi, m)

theta = TLV.Theta();
cost = TLV.CosTheta();
phi = TLV.Phi();
eta = TLV.PseudoRapidity () 
m2 = TLV.Mag2()         = TLV.M2();
m  = TLV.Mag()           = TLV.M();
s  = TLV1*TLV2;           = v1.Dot(v2);     

TVector3 v = TLV.Vect()

Boost a TLorentzVector

TLorentzVector MyParticle1(P1_PX, P1_PY, P1_PZ, P1_E);
TLorentzVector MyParticle2(P2_PX ,P2_PY, P2_PZ, P2_E);

TLorentzVector Particle_CM = MyParticle1 + MyParticle2;

TVector3 Particle_CM_BoostVector = Particle_CM.BoostVector();
TLorentzVector MyParticle1_newSys = MyParticle1_CMS;
MyParticle1_newSys.Boost(Particle_CM_BoostVector);

Python

You can find everything about python here

import

Command	description
import somefile	Everything in somefile.py gets imported. To refer to something in the file, append the text “somefile.” to the front of its name.
from somefile import *	Everything in somefile.py gets imported. To refer to anything in the module, just use its name. Every-thing in the module is now in the current namespace. NB Using this import command can easily overwrite the definition of an existing function or variable!
from somefile import className	Only the item className in somefile.py gets imported. After importing className, you can just use it without a module prefix. It’s brought into the current namespace. NB Overwrites the definition of this name if already defined in the current namespace!

The help() and dir() builtins allow you to check what you have included so far and how to use it ( notice the __ of modules imported by default)

dir() [-5:] #enlist the last 5 variable declared
help(math.sqrt) #describe math.sqrt

Basic commands

Some common commands are:

Command	description
print(a, b)	print a and b
a = input("Enter a string: ")	print "Enter a string: " and inizialize a
3//2	equivalent to int(3./2.)
3%2	remainder of the division

Every word can be used except for :
and, as, assert, break, class, continue, def, del, elif, else, except, exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while, with, yield

Builtins

Command	description
type(object)	return the type of object
range(start,end,step)	return integers from start to end. start=0, step=1 are default
enumerate(N)	range(1,N,1)
sum([1,2,3,4.])	return 10.0
abs(x)	absolute value
sorted([5, 2, 3, 1, 4])	return [1, 2, 3, 4, 5], a useful page about sorting is here
sorted(iterable, key=None, reverse=False)	the iterable can be sorted with a key function and in ascending or descending order
map(function_to_apply, list_of_inputs)	it returns a list. the elements of list_of_inputs are passed to function_to_apply one by one
filter(function_to_apply, list_of_inputs)	creates a list of list_of_inputs'elements for which function_to_apply returns true

from functools import reduce
product = reduce((lambda x, y: x * y), [1, 2, 3, 4])

Write in a text file

show code

hide

outputFile = "source.txt"
outputName = open(outputFile, 'w')                   # 'w' = rewrite  ,       'a' = append
print >> outputName , " variable \t",  x
outputName.close()

String

"qwe"+"rty" == "qwerty"
"qwe"*3 == "qweqweqwe"
mystring = "qwerty"  ,   mystring[3] == "r"

Let mystring be a string

Command	description
mystring.capitalize()	return mystring with the first letter capitalized
mystring.replace(" ","++")	replace each " " with "++" in mystring
mystring.lower()	return mystring with all the letters tined
mystring.upper()	return mystring with all the letters capitalized
mystring.isalpha()	True if there are only alphabetic char
mystring.isdigit()	True if there are only numerical char
mystring.islower()	True if there are only tiny letters
mystring.isspace()	True if there are only spaces e.g. " ".isspace() = true
mystring.isupper()	True if there are only capital letters
mystring.count("test")	count how many "test" there are in mystring
mylist = list(mystring)	return a list made of the letters of mystring
mystring.join(sequence)
mystring.startswith("x")	return True if the string begin with "x"
mystring.endswith("x")	return True if the string end with "x"
mystring.find()
mylist = mystring.split()	return a list of strings. Each string in the list is a word of mystring
mylist = mystring.split("x")	return a list of strings. Strings in the list are pieces of mystring separated by "x"

Lists

L = []                         #or    L = list()
list = range(0,4)
list + ["a", "b", "c"] == [0, 1, 2, 3, 'a', 'b', 'c']
list[2] == 2

Command	description
list.append("Golf")	add "Golf" as element in the end
list.insert(6,"Golf")	insert "Golf" as list[6] and shift the following
list.len()	number of element in the list
list[start:stop:step]	enlist as range() do. Defaults are: start = 0, stop = -1, step = 1
list [-1::-1]	reverse the list
list.sort()	sort the elements
list.pop()	return the lastelement of list and delete it from the list

Tuples

A tuple is a sequence object like a list or a string. It’s constructed by grouping a sequence of objects together with commas, either without brackets, or with parentheses. At variance with list, tuples are immutable, they don’t have append, insert or pop methods

T = ()                         #or    T = tuple()
t  = (0,1,2,3)
t + ("a", "b", "c") == [0, 1, 2, 3, 'a', 'b', 'c']
t[2] == "2"

You cannot set

t[2] = 2

Dictionaries

D=dict();     D1= {};     D2={0:"q",1:"w"}           # 3 different declaration and initialization
D['A']   = 2.1
D['ab'] ='proline'
D[1]    = 2.1
"A" in D == True

Command	description
dict.keys()	enlist all the keys
dict.values()	enlist all the values
del dict["A"]	delete element with key "A"
dict.items()(	enlist all objects in couple (key,value)
dict.values(	enlist all the values
dict.values(	enlist all the values
dict.values(	enlist all the values

Loop over a dictionary

for k,val in dict.items() :
    if k == WantedKeys :
        V = val

Set

Command	description
%run script.py	execute script.py
myset = set(not_a_set)	return a set with the element of la list, a tuple or a string
set1.union(set2)	return the set union. It does not modify set1
set1.intersection(set2)	return the set intersection. It does not modify set1
set1.difference(set2)	return the set difference. It does not modify set1
set1.symmetric_difference(set2)	return the set symmetric difference. It does not modify set1

Regular expressions

Interactive Python (IPython)

Command	description
%run script.py	execute script.py
%reset	clear all namespace
%hist	print history
%xdel	delete variables
%who	list objects in enviroment (use in combo with ? to know who is who)

Reading and writing files

Command	Description
open(filename, mode)	‘r’ (Read), ‘w’ (Write), ‘a’ (Append). If a file opened for ‘w’ does not exist it will be created
f.readline()	read a single line from a file. Returns a string.
f.readlines([size])	read all the lines up to size bytes and return them in a iterator of s
f.read([size])	read up to size bytes; returns a string
f.write(text)	write text to file

This is an exmple of how write a file into an other

in = open("input.txt", "r")
out = open("output.txt", "w")
out.writelines.(in)
in = close()
out = close()

open("output.txt", "w").writelines.(open("input.txt", "r"))

You can also write silngle lines

out.writelines.("This is line 1\nThis is line 2")

Time

import time

start_time = time.time()
do something ....
timeElapsed = time.time() - start_time

Numpy

import numpy as np

Command	Description
a=np.array([(1,2,3,4),(5,6,7,8)])
a.size
a.shape	return the shape (the dimansions)
a=a-10; a=a*2	operations on numpy array
a.T	transpose. Equivalent to a.transpose()
np.max(a)	max. Also a.max()
np.mean(a)	mean. Also a.mean()
np.mean(a[0])	mean on the first row
a.std()	max
b=a[1].copy()	make a copy. If you reassign b, a will not change
unionArray = np.concatenate((a1,a2,a3))	merge several arrays

Creation of numpy array

np.zeros((3,3),'d')	3x3 numpy array made of 0
np.ones((3,3))	3x3 numpy array made of 1
np.eye(3)	3x3 identity
np.arange(1,10)	array([1, 2, 3, 4, 5, 6, 7, 8, 9])
np.arange(2.3,3.3,.1)	array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2]) 0.1is the space
np.linspace(2.3,3.2,10))	array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2]) 10 is the number of elements
a=np.array(list(map(bool,(0,1,0,1,1,0,1))))	a=np.array(list(map(bool,(0,1,0,1,1,0,1))))
np.arange(2.3,3.3,.1)	array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2])
np.arange(2.3,3.3,.1)	array([ 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2])

If "table.txt" contains

0 1 2
3 4 5
6 7 8

Then you can read it and create a numby array simply by:

np.loadtxt("table.txt")

How to plot in jupyter notebook

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

npoints = int(10e2)
x = np.random.rand(npoints)
y = np.random.rand(npoints)

plt.scatter(x,y,marker='o',s=10,color='r')

colorInNumpyArray= np.ones(1000)
plt.scatter(x,y, c = colorInNumpyArray)

Skipy

import scipy as sp
import sklearn
from sklearn import datasets
import sklearn.preprocessing
from sklearn.decomposition import PCA

Creation of

from sklearn import datasets
import sklearn.preprocessing
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

PyPlotLib

A tutorial is here

from matplotlib import pyplot as plt

plt.plot(xList, yList)	plot a graph of (xList, yList)
plt.plot(xList, yList, color='b', linestyle='--', marker='.', linewidth=2, label='thisPlot')	plot a graph of (xList, yList)
plt.xlabel('Ages')	set the x label
plt.ylabel('Efficiency')	set the y label
plt.title('This is ....')	set the title
plt.grid(True)	set the title
plt.xscale('log')	set log scale in x axis
plt.legend()	plot the legend
plt.legend(loc = "upper left")	plot the legend in the upper left
plt.legend(loc =(0.01, 0.9))	plot the legend in the upper left
plt.savefig('plot.png')	save the plot
plt.show()	show the canvas

Plot types

plt.plot(xList, yList)	plot a graph of (xList, yList)
plt.plot(xList, yList, color='b', linestyle='--', marker='.', linewidth=2, label='thisPlot')	plot a graph of (xList, yList)
plt.bar(xList, yList)	plot a histogram of (xList, yList)
plt.bar(xList, yList, width = 0.25, color='b', label='thisPlot')	plot a histogram of (xList, yList)
plt.barh(xList, yList)	plot a histogram of xList as a function of yList
plt.pie(pieList)	plot a pie of pieList
plt.pie(pieList, labels=labelList, colors=colList, wedgeprops={"edgecolor":"black"})	plot a pie of pieList
plt.pie(pieList, labels=labelList, explode=[0,0,01,0,....])	plot a pie of pieList with some enlighted slices
plt.pie(pieList, labels=labelList,shadow=True, startangle=90)	plot a pie of pieList
plt.pie(pieList, labels=labelList,autopc='%1.1f%%')	plot a pie of pieList with percent values
plt.stackplot(xList, y1List, y2List, y3LIst)	plot a stack plot
plt.stackplot(xList, y1List, y2List, labels=labList, colors=colList)	plot a stack plot
plt.hist(List, bins=5)	plot a histogram with 5 bins
plt.hist(ages, bins=bins, edgecolor='black', log=True)	plot a histogram with 5 bins
plt.scatter(xList, yList)	2D plot of x-y lists
plt.scatter(xList, yList, s = 2, c="green", marker="X")	2D plot of x-y lists of green X of size 2
plt.scatter(xList, yList, edgecolor='black', linewidth=1, alpha=0.75)	2D plot of x-y lists
plt.scatter(view_count, likes, c=ratio, cmap='summer')	2D plot of x-y lists, example of colormap
plt.plot_date(dateList, yList)	plot line histogram with dates

Set the axis

plt.xlabel('Ages')	set the x label
plt.ylabel('Efficiency')	set the y label
plt.xticks(ticks = xTick, label = xLab)	Draw the label xLab but use xTick as ticks

Set the Style

print(plt.style.available)	print the list of available styles
plt.style.use("ggplot")	set ggplot style
plt.style.use("seaborn")	set seaborn style
plt.xkcd()	set xkcd comic style
plt.tight_layout()	set tighter pad

SubFigure

fig, ax = plt.subplots()	create a canvas
fig, ax = plt.subplots(nrow=2, ncol=2)	create a canvas with 2x2 pads
fig, ax = plt.subplots(nrow=2, ncol=2, sharex=True)	create a canvas shared x axis

fig1, ax1 = plt.subplots()
fig2, ax2 = plt.subplots()

ax1.plot(xList1, yList1, label='All Devs')

ax2.plot(xList2, yList2, label='Python')

ax1.legend()
ax1.set_title('title')      #setting titles has different function for a
ax1.set_ylabel('yLabel')

ax2.legend()
ax2.set_xlabel('xLabel')

plt.show()
fig1.savefig('fig1.png')
fig2.savefig('fig2.png')

Plotting dates

dates = [
    datetime(2020, 6, 24),
    datetime(2020, 6, 27),
    datetime(2020, 6, 30)
]
y = [0, 1, 3]

plt.plot_date(dateList, yList)

plt.gcf().autofmt_xdate()    # GetCurrentFigure   automatic format for dates

from matplotlib import dates as mpl_dates
date_format = mpl_dates.DateFormatter("%d %b %Y")    # decide the format,  https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
plt.gca().xaxis.set_major_formatter(date_format)        # set the format


dates = pd.to_datetime(datesStringList)              # create a list of string of dates "2020-05-30" in list of datetime(2020, 5, 30)
dates.sort_values('Date')                                     # sort dates by date
#newDates.sort_values('Date', inplace=True)      # equivalent to dates = dates.sort_values('Date')

Live data in real time example:

import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

plt.style.use('fivethirtyeight')

def animate(i):
    data = pd.read_csv('data.csv')
    x = data['x_value']
    y1 = data['total_1']
    y2 = data['total_2']

    plt.cla()

    plt.plot(x, y1, label='Channel 1')
    plt.plot(x, y2, label='Channel 2')

    plt.legend(loc='upper left')
    plt.tight_layout()


ani = FuncAnimation(plt.gcf(), animate, interval=1000)   # execute animate function avery interval: 1000 is one second (I think ani= is useless)

Other

plt.fill_between(xList, y1List, Aconstant)	fill the area between a line and a constant or another line. Default constant is 0
plt.fill_between(ages, y1List, y2List, where=(y1List<= y1List),interpolate=True, color='red', alpha=0.25, label='Below Avg')	fill the area between two lines

Draw a vertical line:

median_age = 29
color = '#fc4f30'
plt.axvline(median_age, color=color, label='Age Median', linewidth=2)

colorbar:

cbar = plt.colorbar()
cbar.set_label('Like/Dislike Ratio')

Pandas temporaney position

data = pd.read_csv('2019-05-31-data.csv')
view_count = data['view_count']
likes = data['likes']
ratio = data['ratio']

Deep Learning

Supervised Learning

%matplotlib inline
import sklearn
from sklearn import datasets

iris = datasets.load_iris()
X = iris.data       #data
Y = iris.target     #true
iris.target_names         #it shows the names the number in target mean


from sklearn.svm import LinearSVC
clf = LinearSVC()
clf.fit(X,Y)

X_new = [[1,2,3,4]]
X_new = np.asarray(X_new)
clf.predit(X_new)
 

iris = datasets.load_iris()
X = iris.data       #data
Y = iris.target     #true
iris.target_names         #it shows the names the number in target mean


from sklearn.svm import linearSVC
clf = linearSVC()
clf.fit(X,Y)

X_new = [[1,2,3,4]]
X_new = np.asarray(X_new)
clf.predict(X_new)
 
iris = datasets.load_iris()
X = iris.data       #data
Y = iris.target     #true
iris.target_names         #it shows the names the number in target mean


from sklearn.svm import linearSVC
clf = linearSVC()
clf.fit(X,Y)

X_new = [[1,2,3,4]]
X_new = np.asarray(X_new)
clf.predit(X_new)

Unsupervised Learning

There are many methods, one of them is Principle Component Analysis (PCA).
Here there is an example of unsupervised learning using KMeans.

%matplotlib inline

from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

from sklearn import datasets
iris = datasets.load_iris()
X = iris.data       #data
Y = iris.target

kmea = KMeans(n_clusters = 2)    # 'n_clusters' is the number of cluster you will have
kmea.fit_predict(X)

plt.scatter(X[:,0], X[:,1], c=kmea.labels_)

Here there is an example of unsupervised learning using DBSCAN.