Linux notes and commands


Linux command library How to Use Lxplus at CERN Linux installation at CERN

Copy/paste with Windows10->XLaunch->Ubuntu
ie from a LibreOffice Impress (ppt) screen) :
Copy/paste within Ubuntu window: Highlight linux text with the left hand mouse button down, then just right hand mouse click on cmd line.

Copy/paste from Ubuntu window to Windows10 screen
Highlight linux text with the left hand mouse button down, paste into Impress using the Paste button in Impress.

Other proposals:
Alternatively you can try pasting by pressing Enter or the middle mouse button.
Also: selecting (left click and drag) to copy, and middle mouse (tap down on scroll wheel) to paste. Worked when adding the shift did not.

How to Copy & Paste in PuTTY:
In Windows, you can copy information from an application to the clipboard by highlighting it and pressing "Ctrl-C." You can paste information from the clipboard to a Windows application by pressing "Ctrl-V." In a PuTTY session, the Windows shortcut keys lose their meaning. Instead of "Ctrl-C," you can copy to the Windows clipboard by highlighting text with your mouse. Instead of pressing "Ctrl-V" to paste text from Windows, press the right mouse button.

Between Windows and PuTTY:
To copy from Windows and paste into PuTTY, highlight the text in Windows, press "Ctrl-C," select the PuTTY window, and press the right mouse button to paste. To copy from PuTTy and paste into Windows, highlight the information in PuTTY and press "Ctrl-V" in the Windows application to paste it.

Within PuTTY:
To copy and paste from within PuTTY, simply highlight the text you want to copy, position your cursor at the place you want to paste the text and press the right mouse button. If you're editing a document using a text editor, such as Nano or Vi, you can also use the editor's cut and paste functionality.

shell script

# My first script

echo "Hello World!"
The first line of the script is important. This is a special clue, called a shebang, given to the shell indicating what program is used to interpret the script. In this case, it is /bin/bash. Other scripting languages such as Perl, awk, tcl, Tk, and python also use this mechanism.

The second line is a comment. Everything that appears after a "#" symbol is ignored by bash. As your scripts become bigger and more complicated, comments become vital. They are used by programmers to explain what is going on so that others can figure it out. The last line is the echo command. This command simply prints its arguments on the display.

date +%H:%M gives: 14:22
date; gives: Mon Oct 22 14:23:38 CEST 2018

cat command
merge hints

Append the contents of file2.txt to the end of file1.txt. 
The content of file3.txt is appended to the end of merged contents of file1.txt and file2.txt and so on.
The entire merged file is saved with the name mergedfile.txt in the current working directory:

bash$ cat file1.txt file2.txt file3.txt file4.txt > ./mergedfile.txt

The cat command accepts regular expressions as input file names, 
which means you can use them to reduce the number of arguments:

bash$ cat file*.txt my*.txt > mergedfile.txt
This will merge all the files in the current directory that start with the name file 
and has a txt extension followed by the files that start with my and has a txt 
extension. You have to be careful about using regular expressions, if you want 
to preserve the order of files. If you get the regular expression wrong, it will affect 
the exact order in which the files are merged.

A quick and easy way to make sure the files get merged in the exact order you want
is to use the output of another file listing program such as ls or find and pipe it to the 
cat command. First execute the find command with the regular expression and verify 
the file order…

bash$ find . -name "file*.txt" -o -name "my*.txt"
This will print the files in order such that you can verify it to be correct or modify it 
to match what you want. You can then pipe that output into the cat command.

bash$ find . -name "file*.txt" -o -name "my*.txt" | xargs cat > ./mergedfile.txt
When you merge multiple files into one file using regular expressions to match them, 
especially when it is piped and where the output file is not very obvious, make sure that 
the regular expression does not match the filename of the merged file. In the case 
that it does match, usually the cat command is pretty good at error-ing out with the 
message “input file is output file”. But it helps to be careful to start with.

RAL linux

  • same usrn as at CERN

ssh -Y
firefox &

Linux beginners guide Linux Support Page Printing from Linux ssh faqs

Availabe printers

  • type> /usr/sbin/lpadmincern --list --building XX
  • alias lprpretty 'lpr -P 40-1B-HPCOR -oprettyprint'
  • lprpretty fn to print

Using the convert command for animated gifs etc

  • link, jpg to video
  • convert -delay 20 -loop 0 evt*.gif animatedevts.gif
  • NOTE: does not work on .png files to one animated png - need to convert to gif - can be done on the same command line:
convert -delay 10 -loop 0 *.png animation.gif

importing animated gifs into pdf

linux 50 most used commands List of Unix programmes

cd tricks

grep More grep xargs Unix and Linux shortcut keys



Linux updates

System->Administration->Software update
  • May get errors - try deselting item, or all items (rt click - drop down)
  • cat /etc/redhat-release for release version
  • /usr/bin/yum update ---skip-broken to try to avoid broken updates

Browsing files on host machine

Access file browser windows in the following ways:
  • Click on the file browser window launcher on the top edge panel
  • Choose Applications -> Browse Filesystem
  • Select a folder, then choose File-> Browse Folder


Example syntax for Secure Copy (scp)

Copy the file "foobar.txt" from a remote host to the local host
$ scp /some/local/directory

Copy the file "foobar.txt" from the local host to a remote host
$ scp foobar.txt


Trouble with WinSCP from home - big block SFTP transfer error.
Remove print statement (echo ROOTSYS = $ROOTSYS) from .tcshrc. All OK now.
Put all print statements into .login file !!!!!!!!!!

Combine files

cat file2 >> file1
The >> operator appends the output to the named file or creates the named file if it does not exist.

cat file1 file2 > file3
This concatenates two or more files to one. You can have as many source files as you need. For example,

cat *.txt >> newfile.txt

Linux commands

Command Action
./ . denotes the current directory. Since you want to run a file in your current directory and that directory is not in your $PATH, you need the ./ bit to tell the shell where the executable is. So, ./foo means run the executable called foo that is in this directory.
source source is a bash shell built-in command that executes the content of the file passed as argument, in the current shell. It has a synonym in '.' (period).
  source filename [arguments], or writing with its synonym, '.' (period) :
  . filename [arguments]
  ie source .tcshrc to make changes come into effect during the current session
sh files do chmod 700 to make executable
  do $bash ~davec/ to execute
g++ to compile with C++, ie g++ map-test.C will generate an executable, by default a.out
  g++ map-test.C -o map.out
  g++ filename -o outputfile
  g++ filename -Wall -ansi -o outputfile
  g++ main.cpp f1.cpp f2.cpp f3.cpp for multiple files, keep header files in same folder or explicitly address
  In order to compile with all warnings enabled and to produce standards-compatible C++ code
  If you want to have the compiler treat warnings as errors--meaning you don't even get an executable, you can use the -Werror flag. This will make sure you don't miss an error.
  Do ./a.out to run
  Also see g++
file manipulate file names and attributes
display ie display fn.png will show the image on any workstation running X
gv ie gv file.pdf to display a pdf or eps file
cp -R copy folders with files/folders
  cp -R /afs/ /afs/
  many associated display functions
command line tcsh examples
  set prompt="$HOST "
  note: leaves space to look good
  set prompt = "-- %T %n %~ -- \n$ " gives:
  -- 23:16 davec ~/my/tmp --
\033 \033 commands etc
echo print to standard output
  CMSSW_BASE is /afs/
du -h get size of directory, -h option to get output in human readable format i.e. show output in kilobytes (K), megabytes (M) and gigabytes (G)
du -sh get size of directory but does not list size of subdirectories
  du -h /dir1/file2
Output stream ofstream myfile; ("example.txt");
  myfile << "Writing this to a file.\n";
Redirecting stdout ie: redirect printf output
  freopen ("myfile.txt","w",stdout);
  printf ("This sentence is redirected to a file.");
  fclose (stdout);
  NOTE: "w" creates an empty file for output operations.
  If a file with the same name already exists, its contents are discarded
  and the file is treated as a new empty file.
  freopen ("myfile.txt","a",stdout);
  "a" appends: it opens file for output at the end of a file.
  Output operations always write data at the end of the file, expanding it.
  The file is created if it does not exist.
setenv All variables set with setenv command are automatically exported to subshell.
set All csh variables set to with set command are NOT automatically exported to subshell.
$ the shell sees a word that begins with a "$", tries to find out what was assigned to the variable and substitutes it.
$(command) The characters "$( )" tell the shell, "substitute the results of the enclosed command."
  Don't put spaces on either side of the equal sign when assigning value to variable. For e.g. In following variable declaration there will be no error
  $ no=10
  But there will be problem for any of the following variable declaration:
  $ no =10
  $ no= 10
  $ no = 10
" Double Quotes "Double Quotes" - Anything enclose in double quotes removed meaning of that characters (except \ and $).
' Single quotes 'Single quotes' - Enclosed in single quotes remains unchanged.
` Back quote `Back quote` - To execute command, ie echo "Today is `date`".
  in bash: right_now = $(date +"%x %r %Z") in tsch: set right_now = `date`
  time_stamp="Updated on $right_now by $USER" in tcsh set time_stamp="Updated on $right_now by $USER"
linux info Linux commands page
shell Learning the shell
  Writing shell scripts
command line convention, UPPERCASE for constants and environment variables
  convention, lowercase names for variables
  you can always type the name of the shell to switch.(bash,csh,sh,tcsh)
  Linux shell scripts
  Working with commands
  How to intrduce arguments on the command line
sed stream editor - to alter file contents etc
  Intro, sed by Bruce Barnett!
  Unix Sed Working methodology
  Called as one execution cycle. Cycle continues till end of file/input is reached:
  Read a entire line from stdin/file. Removes any trailing newline. Places the line, in its pattern buffer.
  Modify the pattern buffer according to the supplied commands. Print the pattern buffer to stdout.
  sed -n '3'p thegeekstuff.txt. Prints the 3rd line in the file thegeekstuff.txt
  sed -n '31445,32889'p thegeekstuff.txt. Prints lines 31445 to 32889 from the file thegeekstuff.txt
  sed '2d' fn > fn2 to delete line 2 in file fn and make the new file fn2
  Other examples
lines in PuTTy change the lines stored in the session
 command_with_lots_of_output | less 
screen screen
cd tricks alias ..="cd .." alias ..2="cd ../.." alias ..3="cd ../../.." alias ..4="cd ../../../.." alias ..5="cd ../../../../.."
cd ../../../.. navigate 4 directories up
cd - toggle between last two directories
ls --color directories are in blue, pictures in purple, and executables in green.
unzip unzip
tar -xvwf fn.tar untar a tar file (there is no 'untar' command)
tar -xvwzf fn.tar.gz unzip and untar a tar.gz file
tar -xvwzf fn.tgz for tgz files
csh will execute the top shell command .tcshrc - but this will be a new shell area
ctrl-shift t for a new terminal in same directory !!!!!
rehash sometimes needed to get back CMSSW commands
<ctrl+r> to repeat a command
&!:str repeat command containing str
Scrolling Go into 'Edit' on Terminal window -> Profile Preferences -> Scrolling and set the number of scroll back lines
< file means open a file for reading and associate with STDIN.
<< token Means use the current input stream as STDIN for the program until token is seen. We will ignore this one until we get to scripting.
> file means open "file" for writing and truncate it and associate it with STDOUT.
> any previous "file" will be overwritten
>> file means open a file for writing and seek to the end and associate it with STDOUT. This is how you append to a file using a redirect.
>> file will append to "file"
n>&m means redirect FD n to the same places as FD m. Eg, 2>&1 means send STDERR to the same place that STDOUT is going to.
diff compare two files, diff file1 file2
  $ some-command >> /tmp/log.log 2>&1
find command ie combine, do echo $PATH to check all the areas
  first directory: /afs/
  find /afs/ -iname combine, to search all subsequent directories
  combine is in /afs/
  listed as an executable as -rwxr-xr-x 1 davec zh 166K Mar 17 19:19 combine
  Likewise for cmsRun
  /afs/, -rwxr-xr-x 1 cmsbuild zh 166K Feb 13 13:20 cmsRun
 find -iname '*.*' -print0 | xargs -0 grep "ret" > ret.txt 
 find mydir/ -name '*.C' -print0 | xargs -0 grep "examplestring" 
  /afs/ -name '*.csv' will search for .csv files in davec and subdirectories
 find /afs/ -exec grep -l "iConfig" '{}' \; 
 find /afs/ -noleaf -name 'reco*.cc' -exec ls -l {} \; 
 find /afs/ -noleaf -name '*.cc' -exec grep 'iConfig' *.cc {} \; 
 find /afs/ -noleaf -exec grep -l 'ESHandle' {} > ESHandle.txt \; 
 find /afs/ -exec grep -l "iConfig\|parameter" '{}' \; 
grep if you want to use shell variables, you need double quotes, ie
 grep "$HOME" file 
searches file for the name of your home directory
 grep -r "sevLev" /afs/ 
  for a recursive search including all subdirectories
 ls |grep blah 
lists all files in the current directory whose names contain the string "blah"
  The canonical wildcard character is the dot "."
  grep b.g file will search going through all SINGLE characters between b and g, ie big, bag but not boogy
 grep '$HOME' 
file searches for the string $HOME
 grep "cat\|dog" file 
matches lines containing the word "cat" or the word "dog"
 grep "I am a \(cat\|dog\)" 
matches lines containing the string "I am a cat" or the string "I am a dog"
  To match a selection of characters, use [].
  [Hh]ello matches lines containing hello or Hello
  [0-3] is the same as [0123], [a-k] is the same as [abcdefghijk], [A-C] is the same as [ABC], [A-Ca-k] is the same as [ABCabcdefghijk]
  an expression consisting of a character followed by an escaped question mark, \?, matches one or zero instances of that character.
  bugg\?y matches all of the following: bugy , buggy but not bugggy
  An expression surrounded by "escaped" parentheses is treated by a single character.
  Fred\(eric\)\? Smith matches Fred Smith or Frederic Smith
Escaped character character preceded by a backslash.
  The preceding backslash does one of the following: (a) removes an implied special meaning from a character (b) adds special meaning to a "non-special" character
  The $ character matches the end of the line.
  The ^ character matches the beginning of the line.
  ie, grep "^From.*mscharmi" /var/spool/mail/davec will match only from the very start
  The [] may be used to search for non-matches. This is done by putting a carat ^ as the first character inside the square brackets.
  ? \ . [ ] ^ $ need to be escaped with \
  grep 'hello.gif' file would find hello-gif , hello1gif , helloagif , etc.
  To stop "." being a wildcard, precede with \ , ie grep 'hello\.gif' file
  $ sign loses its meaning if characters follow it (I think)
  the carat ^ loses its meaning if other characters precede it.
  Square brackets behave a little differently. The rules for square brackets go as follows:
  A closing square bracket loses its special meaning if placed first in a list. for example []12] matches ] , 1, or 2.
  A dash - loses it's usual meaning inside lists if it is placed last.
  A carat ^ loses it's special meaning if it is not placed first
  Most special characters lose their meaning inside square brackets
jobs/queues commands
  The simplest command to submit a batch job to the default (8 minutes) queue is:
  bsub To see the possible parameters, type: man bsub.
bsub –q 1nh ~user1/jobs/job1.csh Submit a job in your home directory to the 1 hour queue
  (Remember: chmod 755 filename to make the file executable)
bsub "/bin/hostname" Submit a command to the default (8 minutes) queue
bsub –M 20000 job2 Submit job2 with a 20 MB memory limit
  (It is recommended to specify a memory limit)
bsub -q 8nh -W 120 job4 Submit job4 to the 8 hour queue with a runtime limit of 120 minutes
  (It is recommended to specify a runtime limit)
bsub –R "type=SLC5_64" job3 Run job3 on a SLC5 host
Also: bjobs, bkill, bqueues, bhosts, bmgroup, bmod, bchkpnt
  brestart, bgadd, bgdel, bjgroup, sh, getrlimit
  sbrk, libckpt.a, lsb.users, lsbqueues, lsb.params
  lsb.hosts, lsb.serviceclasses, mbatchd
top see processes which consume most cpu, leave with Q only
kill n the process with pid n will be signalled
kill -n all processes in group n are signalled
ctrl alt F1 gives a terminal to login again
ctrl alt F7 to go back to X screen
ssh -X lxplus To allow tunnelin for emacs etc
ssh -Y lxplus ditto
source .bashrc Will execute the .bashrc file
rm -rf directory name or filename Will force removal. Dangerous!
fs lq quota check
fs listquota quota check
fs examine detailed quota check
chmod u+x fn to give user executable permission
chmod 755 fn to give all read and exec permission, I also get write permission
chmod 700 fn Only I get read, write and exec permission
.tcshrc location for aliases etc. Executed at logon
<ctrl><shift> t For a new terminal in the same directory
<ctrl>xs save
./hello to execute file 'hello', '.' for current directory, '/' something in this directory
ps -u davec to list my processes that are running
lpr lpr -P 40-1B-HPCOR -oprettyprint VPT-UVA-Brunel-info.C
  CERN linux printing page
  /usr/sbin/lpadmincern --list --building XX to list available printers
  To install all printers available in building XX run as root:
  /usr/sbin/lpadmincern --add --building XX
  /usr/sbin/system-config-printer to fine tune printer after installing
  Copy all files and directories in dev recursively to subdirectory bak:
  $ cp -R dev bak


~/scratch0 > fs listquota
Volume Name                    Quota       Used %Used   Partition
s.cms.davec.0                2000000    1827235   91%<<       29%    <<WARNING

~ > fs lq
Volume Name                    Quota       Used %Used   Partition
user.davec                    888000     762767   86%         46%

~/scratch0 > fs examine
File . (1933946694.1.1) contained in volume 1933946694
Volume status for vid = 1933946694 named s.cms.davec.0
Current disk quota is 2000000
Current blocks used are 1827235
The partition has 348838756 blocks available out of 488139640

Linux compiler info



Was getting locale errors (Jan 2020) in while trying to do a cmsenv. Logged in via davec@dcockerill-pc01.
Seems lxplus6 was picking up 'locale' settings from the Ubuntu window davec@dcockerill-pc01 that could not be set.
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
>         LANGUAGE = (unset),
>         LC_ALL = (unset),
>         LANG = "C.UTF-8"
>     are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").

locale -a   on linux6 gave:

On davec@dcockerill-pc01:~$ locale       gave:

davec@dcockerill-pc01: sudo update-locale LANG=en_US.UTF-8  without success.

Update locale area:
davec@dcockerill-pc01: sudo apt-get install language-pack-en language-pack-en-base >> this failed

davec@dcockerill-pc01:~$ sudo apt-get update, THEN
davec@dcockerill-pc01:sudo apt-get install language-pack-en language-pack-en-base   >> this worked, giving:
Generating locales (this might take a while)...
  en_AG.UTF-8... done
  en_AU.UTF-8... done
  en_BW.UTF-8... done
  en_CA.UTF-8... done
  en_DK.UTF-8... done
  en_GB.UTF-8... done
  en_HK.UTF-8... done
  en_IE.UTF-8... done
  en_IL.UTF-8... done
  en_IN.UTF-8... done
  en_NG.UTF-8... done
  en_NZ.UTF-8... done
  en_PH.UTF-8... done
  en_SG.UTF-8... done
  en_ZA.UTF-8... done
  en_ZM.UTF-8... done
  en_ZW.UTF-8... done
Generation complete.

locale -a            >> get

Now try to set up LANG:
davec@dcockerill-pc01:~$ sudo update-locale LANG=en_GB.UTF-8   >> did not work
davec@dcockerill-pc01:~$ LANG=en_GB.UTF-8   >>      THIS worked
davec@dcockerill-pc01:~$ locale
Now (Jan 2020), no more locale errors in lxplus6 when doing cmsenv.

Superuser, su

To add users etc to the PC
sudo su
then enter cern usrn and pwd
then addcernusr xxxxxx
May need to check bin for exact add command.

Otherwise, as a superuser:

  • su -
  • enter pc passwd, then:
  • /usr/sbin/addusercern sproston
  • check with:
  • cat /etc/passwd | grep '/afs/'

-- DavidCockerill - 07-Apr-2011

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt Emacs-from-putty.txt r1 manage 2.9 K 2014-08-03 - 11:28 DavidCockerill  
PDFpdf Linux-PATH.pdf r1 manage 42.1 K 2012-05-01 - 11:34 DavidCockerill  
Texttxt command-line-format.txt r1 manage 14.4 K 2014-08-28 - 12:02 DavidCockerill  
PDFpdf path-info-CMSSW.pdf r1 manage 17.5 K 2012-05-01 - 11:33 DavidCockerill  
Edit | Attach | Watch | Print version | History: r83 < r82 < r81 < r80 < r79 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r83 - 2020-01-27 - DavidCockerill
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback