FLUKA Simulations First Steps

This is a simple recipe that aims to guide a user to install and run batch jobs in the CERN bath system.

Type of Instruction colors:

Commands to run on your terminal

Messages on your terminal

Description of FLUKA Scripts in this recipe:

sim: Runs a cycle of FLUKA simulation define in your input file (toy.inp). When ./sim script runs, the RANDOMIZ card is changed automatically by a new random number, such that the results of each cycle are different from the others.

RANDOMIZ.sh: Changes the seed number of your input file (toy.inp) with a new random number.

multiple.sh: Sends many FLUKA jobs for running parallel processes of the same simulation but using different seed number.

joinfiles_bin.sh: Joins the output USRBIN binary files into a single file (Similar to push the button "Process" in the FLAIR interface). Then the binary file is converted to ASCII file. Usage: ./joinfiles_bin.sh toydirectory USRBINnumber

toy.inp: FLUKA simulation file following the FLAIR Quick Start Guide (This example only includes one USRBIN DOSE scoring): http://www.fluka.org/flair/flair_basic1.pdf

1. Installing FLUKA at your CERN location

This recipe covers an installation of the FLUKA program 32 bits version in the lxplus batch system at CERN. Your CERN username, first letter of username and password are needed

Open a terminal and log to a CERN machine

ssh username@lxplus.cern.ch

After digit your password, you should see:

* ********************************************************************
* Welcome to lxplus048.cern.ch, SLC, 6.7
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* https://cern.ch/ComputingRules
* Puppet environment: qa, Roger state: production
* Foreman hostgroup: bi/inter/plus/login
* LXPLUS Public Login Service
* * WARNING, lxplus048.cern.ch has lemon exceptions:
*  exception.swap_io

* 2016-04-15 - python-cherrypy installed
* ********************************************************************

Download the FLUKA 32 bits source package from the web page in the "public" directory (your flukauser + flukapass are needed)

wget --user=flukauser --password=flukapass --no-check-certificate https://www.fluka.org/packages/fluka2011.2c-linuxAA.tar.gz

Alternate step: Get the source package and tranfer it to the public directory. Open a new terminal, open the file directory and run this command:

scp fluka20111.2c-linuxAA.tar.gz username@lxplus.cern.ch:/afs/cern.ch/user/u/username/public/

Create a fluka directory

mkdir fluka

Copy the source file to fluka directory, uncompress it and make it. Note: After that, you should not see any error messages.

cp fluka2011.2c-linuxAA.tar.gz ./fluka
cd ./fluka
tar -zxvf fluka2011.2c-linuxAA.tar.gz
make
cd ..

2. Run a set of jobs using CERN lxplus: Basic Example

Create your simulation directory in your "public" directory

mkdir work

Open your simulation directory and create a main directory for your FLUKA simulation:

cd work
mkdir simulation

Open simulation and create a place for your FLUKA input files (.inp):

cd simulation 
mkdir FILES

Open FILES directory and download a FLUKA input file (toy.inp) from this twiki to your CERN computer. Then, return back to simulation

cd FILES
wget --no-check-certificate https://twiki.cern.ch/twiki/pub/Sandbox/FLUKASimulations/toy.inp
cd .. 

Download the scripts needed to run your simulation using the batch system. a description of their function is at the bottom of this twiki.

wget --no-check-certificate https://twiki.cern.ch/twiki/pub/Sandbox/FLUKASimulations/multiple.sh
wget --no-check-certificate https://twiki.cern.ch/twiki/pub/Sandbox/FLUKASimulations/sim
wget --no-check-certificate https://twiki.cern.ch/twiki/pub/Sandbox/FLUKASimulations/RANDOMIZ.sh
wget --no-check-certificate https://twiki.cern.ch/twiki/pub/Sandbox/FLUKASimulations/joinfiles_bin.sh 

Open your sim file using nano:

nano sim

Follow the instructions inside the script

###############################################################################
## This script is for sending a FLUKA job
##
## INSTRUCTIONS
## This script receives 1 parameter which is the your identification number
## of the job. After FLUKA simulation finishes a new folder will be created
## inside ./output$mydir directory with your id number, FLUKA results will be
## stored here.
##
## Modify `export FLUPRO` and `export WORK` lines with your FLUKA and WORK
## directory paths.
###############################################################################

To Save your changes and Close, push on the keyboard: CTRL + O, ENTER and CTRL + X

Open your multiple.sh file using nano:

nano multiple.sh

Follow the instructions inside the script

#!/bin/bash

###############################################################################
##INSTRUCTIONS
## This script is for sending many jobs, each with different RANDOM SEED number
## To change the number of jobs modify $initial and $final variables
## You must also check `bsub` parameters to meet your requeriments
##############################################################################

You should note that, In this example $initial=1 and $final=2, therefore we will have two jobs. For instance, in the case of $initial=11 and $final=20, we would run 10 jobs! In order to Save your changes and Close, push on the keyboard: CTRL + O, ENTER and CTRL + X

Remove user permissions to run your scripts

chmod +x multiple.sh sim RANDOMIZ.sh joinfiles_bin.sh

Run 2 batch jobs for your simulation

./multiple.sh

Note: You should see a message similar to this one:

Job  is submitted to queue .
Job  is submitted to queue . 

3. Process your output files

While you are waiting for your jobs, you can check the status of them by using the bjobs command:

bjobs

Then, a message similar to this one will appear:

 JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME
792380298 daguerr PEND 8nh lxplus011.c sim 1 Apr 25 23:20
792380369 daguerr PEND 8nh lxplus011.c sim 2 Apr 25 23:20 

After the simulation is done, run ls command to see what your "simulation" directory has:

ls

FILES  joinfiles_bin.sh  LSFJOB_792427554  LSFJOB_792427562  multiple.sh  outputFILES  RANDOMIZ.sh  sim

Note: The outputFILES directory has your "fort.*" files for each job. LSFJOB directories have some text files with information about each job

Now, In order to combine all of your USRBIN files to a single one, run the joinfiles_bin.sh:

./joinfiles_bin.sh FILES 21

 Type the input file:  Toy                                                                             
  DATE:  4/26/16,  TIME:  1:38:30
  40.
 40
 0
 1
 Type the input file:  Toy                                                                             
  DATE:  4/26/16,  TIME:  1:38:29
  40.
 40
 0
 1
 Type the input file:  Type the output file name:  Total Batches  2
 Type the input file:  Toy                                                                             
       ***** Sum file *****      
  80.
 80
 0
 2

Run ls command to see what your "simulation" directory has:

ls

FILES  joinfiles_bin.sh  LSFJOB_792427554  LSFJOB_792427562  multiple.sh  outputFILES  RANDOMIZ.sh  sim  toy_21.bnn  toy_21.lis

The joined output files are: toy_21.bnn (binary file) and toy_21.lis (ASCII file)

-- DanielGuerrero - 2016-04-25