Quick migration guide for Geant4 version 10.x series

https://http://www.geant4.org/geant4/support/userdocuments.shtml

A page with some performance results can be found here: MultiThreadingTaskForce

Introduction

This document describes how to make a Geant4 application that works both in sequential and in multi-threaded modes on Geant4 version 10.0. If a user has an application that is running in geant4 v9.6.p02 and would like to stick to the sequential mode for the new version 10.0, (s)he does not need to apply the modifications this document describes. On the other hands, please note that the migrated code works with both in sequential and in multi-threaded modes of Geant4 version 10.0.

This is a quick migration guide. This guide provides the minimal information for converting a user's application running on Geant4 version 9.6p02 to be multi-threaded with Geant4 version 10.0. This guide should be used as a short summary, and it is not meant to provide comprehensive information. The user should consult to the Geant4 user's guides for the details. Additional useful information may be found at Geant4 HyperNews. Also, please note that this guide is not meant for the novice user who is new to Geant4.

To use the multi-threaded version, Geant4 library must be built with the following CMAKE option

-DGEANT4_BUILD_MULTITHREADED=ON

In addition, for Mac OS X 10.7, the user has to use CLANG compiler by specifying the following two CMAKE options for both of building a library and building an application

-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++

• If the user is using the external CLHEP, it also has to be compiled with these CLANG options.

Following limitations of Geant4 version 10.0 are known

1. MT mode on Windows OS is not yet supported.
2. Some objects are not cleanly deleted at the termination of the thread/program.
3. Though the user may issue more than one BeamOn, (s)he cannot change the number of threads after Geant4 is first-time initialized.
4. A few functionalities have not yet migrated to multi-threading. That include material scanner, G4RayTracerX. Also, some examples have not yet migrated.

The basics: per-thread vs shared classes

In the multi-threaded mode, generally saying, objects that are stable during the event loop are shared among threads, while objects that are transient during the event loop are thread-local. In general, geometry and physics tables are shared, while all physics and other kinds of processes are thread-local. Event, track, step, trajectory, hits, etc., as well as several Geant4 manager classes such as EventManager, TrackingManager, SteppingManager, TransportationManager, GeometryManager, FieldManager, Navigator, SensitiveDetectorManager, etc. are thread-local. Among the user classes, user initialization classes (G4VUserDetectorConstruction, G4VUserPhysicsList and newly introduced G4VUserActionInitialization) are shared, while user action classes and sensitive detector classes are thread-local. It is not straightforward (and thus not recommended) to access from a shared class object to a thread-local object, e.g. from detector construction to stepping action. Please note that thread-local objects are instantiated and initialized at the end of the first execution of G4RunManager::Initialize() method, in other words, just before the first Idle> prompt. To avoid potential errors, it is advised to always keep in mind which class is shared and which class is thread-local.

Please note the following:

• If a messenger class is instantiated by a tread-local class, UI commands belonging to that messenger class are properly delivered to the target class objects of all threads. If the messenger is instantiated by a shared class, command cannot be delivered to thread-local objects. Special attention is required for setting the electromagnetic field value (see Detector construction section).
• If the user uses G4Allocator for his/her own class, e.g. hit, trajectory or trajectory point, G4Allocator object must be thread-local and thus must be instantiated within the thread. The object new-ed and allocated by the thread-local G4Allocator must be deleted within the same thread (see Thread safety in user code section).
• SteppingVorbose must be thread-local. If the user uses the default G4SteppingVorbose, it is taken care automatically by the Geant4 kernel and the user does not need to do anything. If the user wants to use his/her own SteppingVerbose, it must be instantiated and registered through G4UserActionInitialization class.

A few static methods are available for user's convenience. To use these methods, include "G4Threading.hh".

• G4int G4Threading::G4GetThreadId() returns a unique thread identification number (0 or positive integer) if this method is invoked from a worker thread. It returns -1 if it is invoked by the master thread, and -2 if it is used in the sequential mode.
• G4bool G4Threading::IsWorkerThread() returns true if it is invoked from a worker thread, and returns false if it is invoked from he master thread or in the sequential mode.
• G4int G4Threading::GetNumberOfCores() returns the number of threads available on the machine. This number includes the number of hyper threads.

Number of threads to be used could be specified by the following ways. Please note that this number specifies the number of worker threads that process events, and does not include an additional thread for the master thread.

• Shell environment variable G4FORCENUMBEROFTHREADS. This will overwrite the following alternative settings. G4FORCENUMBEROFTHREADS can be an integer or a keyword "max". If "max" is specified, Geant4 uses all threads of the machine including all hyper threads.
• UI command /run/numberOfThreads. This UI command has to be issued at PreInit> state.
• G4RunManager::SetNumberOfThreads(G4int). This method must be invoked prior to G4RunManager::Initialize().

G4MTRunManager, G4WorkerRunManager

• It returns the pointer to the G4WorkerRunManager of the local thread when it is invoked from thread-local object.
• It returns the pointer to the G4MTRunManager when it is invoked from shared object.
• It returns the pointer to the base G4RunManager if it is used in the sequential mode.

Do not invoke a set-method of a shared class from a thread-local class to modify a data member of the shared class, unless you correctly protect such an invocation with a Mutex lock.

main()

In the main() function, the user should instantiate the user initialization classes only (G4VUserDetectorConstruction, G4VUserPhysicsList and newly introduced G4VUserActionInitialization) and set them to run manager through SetUserInitialization() method. All the user action classes must be instantiated in G4VUserActionInitialization class.

#ifdef G4MULTITHREADED
#include "G4MTRunManager.hh"
#else
#include "G4RunManager.hh"
#endif

int main()
{
#ifdef G4MULTITHREADED
  G4MTRunManager* runManager = new G4MTRunManager;
  runManager->SetNumberOfThreads(<number_of_threads>);
#else
  G4RunManager* runManager = new G4RunManager;
#endif

  runManager->SetUserInitialization(new MyDetectorConstruction);
  runManager->SetUserInitialization(new FTFP_BERT);
  runManager->SetUserInitialization(new MyActionInitialization);

  . . .

  delete runManager;
}

No more than one physics lists should be instantiated, though modular physics lists are allowed. If the user needs to select a physics list dynamically, instantiate a physics list in an if -statement to ensure no more than one physics lists are instantiated.

main()
{
   . . .
   G4VUserPhysicsList* pl1 = new FTFP_BERT;
   G4VUserPhysicsList* pl2 = new QGSP_BERT;
   if( _some_condition_ )
   {  runManager->SetUserInitialization( pl1 ); }
   else
   {  runManager->SetUserInitialization( pl2 ); }
   . . .

main()
{
   . . .
   G4VUserPhysicsList* pl = 0;
   if( _some_condition_ )
   { pl = new FTFP_BERT; }
   else
   { pl = new QGSP_BERT; }
   runManager->SetUserInitialization( pl );
   . . .

User defined physics list

Every process must be thread-local. If the user has his/her own implementation of PhysicsList or PhysicsBuilder, don't instantiate a process in a constructor, as PhysicsList and PhysicsBuilder are shared. Make sure that all process objects are instantiated in the ConstructProcess() method, which is invoked for every thread. Also, don't use a flag data member to protect the ConstructProcess() method, don't keep a pointer to a process as a class data member. Please note that, if the user uses two worker threads, ConstructProcess() will be invoked three times (one for master thread).

If the user has a user-implemented physics list and thus does not use one of the pre-packaged physics lists, and (s)he needs ions to be simulated, add G4GenericIon::GenericIonDefinition() into ConstructParticle() method. This ensures that the generic_ion particle is created at initialization, and ensures all ions (including light ions such as deuteron, alpha) work properly.

Detector construction

G4VUserDetectorConstruction now has a new virtual method ConstructSDandField().

Sensitive detector

Given sensitive detector class objects should be thread-local, instantiation of such thread-local classes should be implemented in this new ConstructSDandField() method, which is invoked for each thread. Construct() method should contain definition of materials, volumes and visualization attributes. To define a sensitive detector in ConstructSDandField() method, a new utility method SetSensitiveDetector("LVName",pSD) is available to make ease of migration, This SetSensitiveDetector("LVName",pSD) method does two things:

1. Register the sensitive detector pointer pSD to G4SDManager, and
2. Set pSD to the logical volume named "LVName".

The ConstructSDandField() method is invoked from the base-class G4RunManager for sequential mode.

If the user needs to define sensitive detector(s) to the volumes defined in a parallel world, (s)he may do so by implementing G4VUserParallelWorld::ConstructSD() method. Please note that defining field in a parallel world is not supported.

G4VPhysicalVolume* MyDetectorConstruction::Construct()
{
   . . .  // material definition
   . . .  // world volume
   G4VPhysicalVolume* worldPV = new G4PVPlacement(...);

   . . .  // detector
   G4LogicalVolume* mySDLV = new G4LogicalVolume(...,"MySDLV");
   . . . 

   return worldPV;
}

void MyDetectorConstruction::ConstructSDandField()
{
   MySensitiveDetector* mySD = new MySensitiveDetector(...);
   SetSensitiveDetector("MySDLV", pMySD);
}

Field

A field class object (both global and local) must be thread-local. Please note that G4TransportationManager is thread-local. Thus, regardless of whether the field class object is global or local to a certain volume, a field object must be assigned to G4FieldManager by the ConstructSDandField() method, or in some other method invoked at thread-local place. If the user wants to change the field values between runs using his/her UI command, special treatment is required. Please keep in mind that ConstructSDandField() method is invoked for the second or later runs only if the geometry is modified. If the field value is changed without changing geometry, this ConstructSDandField() is not invoked. The following is the simplest recipe.

1. Instantiate field class object in the ConstructSDandField() method so that the field object is thread-local.
2. In the constructor of the field class, instantiate a dedicated messenger with a field UI command, so that the messenger and the command are thread-local.
3. When the field value is set to this field class object (which is thread-local), access to G4TransportationManager and set the field and create ChordFinder.

#include "G4Types.hh"
#include "G4VUserDetectorConstruction.hh"
#include "MyField.hh"
class MyDetectorConstruction : public G4VUserDetectorConstruction
{
   . . .
 private:
   static G4ThreadLocal MyField* myField;
};

 
G4ThreadLocal MyField* MyDetectorConstruction::myField = 0;

void MyDetectorConstruction::ConstructSDandField()
{
  if(!myField) myField = new MyField(); 
}

class MyField : public G4Field (or other G4 field class)
{
 public:
   MyField();
   virtual ~MyField();

   virtual void GetFieldValue(const double*, double* ) const;
   void SetFieldValue(G4ThreeVector);

 private:
   MyFieldMessenger* messenger;
   G4ThreeVector fieldVal;
};

MyField::MyField()
{
  messenger = new MyFieldMessenger(this);
  SetFieldValue(G4ThreeVector(0.,0.,0.));
}

~MyField::MyField()
{ delete messenger; }

void MyField::GetFieldValue(const double*, double* BF)
{ BF[0]=fVal.x(); BF[1]=fVal.y(); BF[2]=fVal.z(); }

void MyField::SetFieldValue(G4ThreeVector fVal)
{
  fieldVal = fVal;
  G4FIeldManager* fMan
    = G4TransportationManager::GetTransportationManager()
        ->GetFieldManager();
  fMan->SetDetectorField(this);
  fMan->CreateChordFinder(this);
}

Changing geometry between runs

There are two ways to change geometry between runs.

1. Invoking Set -method of G4VSolid, G4LogicalVolume, G4VPhysicalVolume, etc. of the volume the user has already created in the previous run.
   • After changing the data member, invoke G4RunManager::GeometryHasBeenModified(). This ensures the re-optimization of the geometry for navigation and reconfiguration of material-cuts-couple. It does not invoke Construct() of user detector construction.

1. Deleting all solids, logical volumes and physical volumes and build everything from scratch.
   • Invoke G4RunManager::ReinitializeGeometry(). This ensures invocation of Construct() of user detector construction in the master thread and invocation of ConstructSDandField() in the worker threads. In addition it also enforces GeometryHasBeenModified().
   • Please note that materials and sensitive detectors cannot be deleted. Thus the user has to set the pointers of already-existing materials / sensitive detectors to the relevant logical volumes.

G4VUserActionInitialization class

void MyActionInitialization::Build() const
{
  SetUserAction(new MyPrimaryGeneratorAction);
  SetUserAction(new MyRunAction);
  SetUserAction(new MySteppingAction);
  . . .
}

void MyActionInitialization:BuildForMaster() const
{
  SetUserAction(new MyRunAction);
}

A worker run manager may have UserRunAction for dealing with thread-local things, while the master G4MTRunManager may also have its own UserRunAction for the global run. UserRunAction to be registered to the master for the global run could be an object of the same class defined for the workers, or an instance of a dedicated class. If UserRunAction class is shared by workers and master, a protected method IsMaster() is available to distinguish whether it is used for a worker or for the master (an example implementation is here).

Please note:

• IsMaster() returns true if it is used in the sequential mode. Also, IsMaster() does not return a correct value until that RunAction object is set to a RunManager. Thus, don't use IsMaster() method in the constructor of RunAction.
• Both Build() and BuildForMaster() are const methods. This means that, even though the user instantiates user action classes and register them, (s)he cannot change any data member of this class. This restriction comes from the fact that this G4VUserActionInitialization class object is shared by all threads.
• BuildForMaster() is not invoked for the sequential mode. Thus, if the user implement a dedicated UserRunAction for the master thread, it won't be used at all for the sequential mode. If the user wants exactly the same output for the sequential mode and the global run action, (s)he should use a single UserRunAction class and use IsMaster() method which returns true for both sequential mode and the master thread in multi-threaded mode.
• If the user need an access from thread-local user action class (e.g. stepping action or primary generator action) to the shared user initialization class (e.g. detector construction), (s)he may do so through G4MTRunManager::GetMasterRunManager() to get the pointer of the G4MTRunManager and use that pointer to get the pointer of the user initialization class. Alternatively, the user's ActionInitialization class constructor may take the pointer of the user initialization class.
  • As mentioned earlier, access from shared user initialization class to thread-local user action class is very difficult and not recommended.
  • G4MTRunManager::GetMasterRunManager() returns a null pointer if it is used in the sequential mode.

• User-defined SteppingVerbose must be instantiated in the dedicated virtual method G4VUserActionInitialization::InitializeSteppingVerbose().
• Please be aware that all virtual methods of G4VUserActionInitialization are const methods. The user may instantiate objects and set them to run managers, but cannot keep pointers of such objects. These methods are invoked for every worker threads, and objects instantiated in these methods are supposed to be thread-local.

Input data file for PrimaryGeneratorAction

PrimaryGeneratorAction is a thread-local class. Thus, special attention is required if the user needs an input file to read for primary particles. If thread-local objects of PrimaryGeneratorAction naively open a file and read it, they all read the file from the beginning and they all read exactly the same input. The appropriate implementation of reading an input file for PrimaryGeneratorAction depends on the use-case, more precisely, on the ratio of the cost of accessing to the data file to the total execution time of one event on one thread. If this ratio is low, i.e. execution time dominates (typically the case of high-energy physics experiment where the energy of primary particles are high), the file access can be shared with Mutex locking mechanism. If this ratio is high, i.e. file access is a significant fraction of the execution time (typically the case of medical and space applications where the energy of primary particles are rather low), the input data should be cached in addition to the Mutex locking. To use Mutex locking mechanism, include G4AutoLock.hh header file.

Here is a high-energy physics sample code with G4HEPEvtInterface that reads a Pythia input file. G4HEPEvtInterface has to be a single object shared by all the PrimaryGeneratorAction objects, and the access to G4HEPEvtInterface::GeneratePrimaryVertex() should be protected by Mutex.

#include "G4VUserPrimaryGeneratorAction.hh"
class G4HEPEvtInterface;
class MyHepPrimaryGenAction
 : public G4VUserPrimaryGeneratorAction
{
 public:
   MyHepPrimaryGenAction(G4String fileName);
   ~MyHepPrimaryGenAction();
   . . .
   virtual void GeneratePrimaries(G4Event* anEvent);
 private:
   static G4HEPEvtInterface* hepEvt;
};

#include "MyHepPrimaryGenAction.hh"
#include "G4HEPEvtInterface.hh"
#include "G4AutoLock.hh"
namespace { G4Mutex myHEPPrimGenMutex = G4MUTEX_INITIALIZER; }
G4HEPEvtInterface* MyHepPrimaryGenAction::hepEvt = 0;

MyHepPrimaryGenAction::MyHepPrimaryGenAction(G4String fileName)
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  if( !hepEvt ) hepEvt = new G4HEPEvtInterface( fileName ); 
}

MyHepPrimaryGenAction::~MyHepPrimaryGenAction()
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  if( hepEvt ) { delete hepEvt; hepEvt = 0; } 
}

void MyHepPrimaryGenAction::GeneratePrimaries(G4Event* anEvent)
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  hepEvt->GeneratePrimaryVertex(anEvent);
}

Here is a lower-energy sample code with G4ParticleGun to shoot 10 MeV electrons. The input file contains list of G4ThreeVector of the momentum direction (ex, ey, ez) and it is read by a dedicated file reader class MyFileReader. MyFileReader is shared by all threads, and reads 100 events at a time and buffers them. Primary vertex position is randomized. Please note that, for the simplicity of this sample code, it does not consider the end-of-file.

#include "G4VUserPrimaryGeneratorAction.hh"
class G4ParticleGun;
class MyFileReader;
class MyLowEPrimaryGenAction
 : public G4VUserPrimaryGeneratorAction
{
 public:
   MyLowEPrimaryGenAction(G4String fileName);
   virtual ~MyLowEPrimaryGenAction();
   . . .
   virtual void GeneratePrimaries(G4Event* anEvent);
 private:
   static MyFileReader* fileReader;
   G4ParticleGun* particleGun;
};

#include "G4ParticleTable.hh"
#include "G4ParticleDefinition.hh"
#include "Randomize.hh"
#include "G4AutoLock.hh"
namespace { G4Mutex myLowEPrimGenMutex = G4MUTEX_INITIALIZER; }
MyFileReader* MyLowEPrimaryGenAction::fileReader = 0;

MyLowEPrimaryGenAction::MyLowEPrimaryGenAction(G4String fileName)
{
  G4AutoLock lock(&myLowEPrimGenMutex);
  if( !fileReader ) fileReader = new MyFileReader( fileName ); 
  particleGun = new G4ParticleGun(1);
  G4ParticleTable* particleTable = G4ParticleTable::GetParticleTable();
  G4ParticleDefinition* particle = particleTable->FindParticle("e-");
  particleGun->SetParticleDefinition(particle);
  particleGun->SetParticleEnergy(10.*MeV);
}

MyLowEPrimaryGenAction::~MyLowEPrimaryGenAction()
{
  G4AutoLock lock(&myLowEPrimGenMutex);
  if( fileReader ) { delete fileReader; fileReader = 0; }
}

void MyLowEPrimaryGenAction::GeneratePrimaries(G4Event* anEvent)
{
  G4ThreeVector momDirction(0.,0.,0.);
  if(fileReader)
  {
    G4AutoLock lock(&myLowEPrimGenMutex);
    momDirection = fileReader->GetAnEvent();
  }
  particleGun->SetParticleMomentumDirection(momDirction);
  G4double x0 = 2.* Xmax * (G4UniformRand()-0.5);
  G4double y0 = 2.* Ymax * (G4UniformRand()-0.5);
  particleGun->SetParticlePosition(G4ThreeVector(x0,y0,0.));
  particleGun->GeneratePrimaryVertex(anEvent);
}

#include <list>
#include <fstream>
class MyFileReader
{
 public:
   MyFileReader(G4String fileName);
   ~MyFileReader();
   . . .
   G4ThreeVector GetAnEvent();
 private
   std::ifstream inputFile;
   std::list<G4ThreeVector> evList;
};

MyFileReader::MyFileReader(G4String fileName)
{ inputFile.open(filename.data()); }

MyFileReader::~MyFileReader()
{ inputFile.close(); }

G4ThreeVector MyFileReader::GetAnEvent()
{
  if( evList.size() == 0 )
  {
    for(int i=0;i<100;i++)
    {
      G4double ex, ey, ez;
      inputFile >> ex >> ey >> ez;
      evList.push_back( G4ThreeVector(ex,ey,ez) );
    }
  }
  G4ThreeVector ev = evList.front();
  evList.pop_front();
  return ev;
}

Please note that Mutex has a performance penalty. Thus, in case the user uses many threads and the execution time of one event is very short, the most efficient way is splitting the input file to the number of threads so that each thread reads its own dedicated input file without Mutex lock. In this case, the buffering shown in above-mentioned MyFileReader class should be still used to reduce the file I/O overhead.

Collecting thread-local data

To collect data over worker threads at the end of an event loop, derived class of G4Run should be used. The base class defines the following virtual methods.

• void RecordEvent(const G4Event*)
  • Method to be overwritten by the user for recording events in this (thread-local) run.
  • Invoke G4Run base-class method for recording data member in the base class.
  • If the user in the past has implemented an access to hits or scores in his/her UserEventAction::EndOfEventAction(), (s)he can simply move that code segment to this RecordEvent() method. See the following sample code.
• void Merge(const G4Run*)
  • Method to be overwritten by the user for merging local Run object to the global Run object.
  • Invoke G4Run base-class method to merge data member in the base class.

Please note. If the user uses the built-in command-based scorer, scores are automatically accumulated to the global run.

#include "G4Run.hh"
class MyRun : public G4Run
{
 public:
  MyRun();
  virtual ~MyRun();
  virtual void RecordEvent(const G4Event*);
  virtual void Merge(const G4Run*);
  inline G4double GetTotalEDep() const
  { return fEDep; }
 private:
  G4int fEDepScoreID;
  G4double fEDep;
};

#include "G4SDManager.hh"
MyRun::MyRun()
:fEDep(0.)
{
 fEDepScoreID = G4SDManager::GetSDManager()->GetCollectionID("myDet/myEDepScorer"); 
}

void MyRun::RecordEvent(const G4Event* evt)
{
  G4HCofThisEvent* HCE = evt->GetHCofThisEvent();
  if(HCE) {
   // This part of the code could be replaced to any kind of access
   // to hits collections and scores
   G4THitsMap<G4double>* evtMap = (G4THitsMap<G4double>*)(HCE->GetHC(fEDepScoreID));
   std::map<G4int,G4double*>::iterator itr = evtMap->GetMap()->begin();
   for(; itr != evtMap->GetMap()->end(); itr++) { fEDep += *(itr->second); }
  }
  G4Run::RecordEvent(evt);
}

void MyRun::Merge(const G4Run* aRun)
{
  const MyRun* localRun = static_cast<const MyRun*>(aRun);
  fEDep += localRun->fEDep;
  G4Run::Merge(aRun);
} 

User-defined Run class must be instantiated by the user's RunAction class.

G4Run* MyRunAction::GenerateRun()
{ return new MyRun; }

void MyRunAction::EndOfRunAction(const G4Run* aRun)
{
  const MyRun* theRun = static_cast<const MyRun*>aRun;
  if(IsMaster())
  {
    G4cout << "Global result with " << theRun->GetNumberOfEvent()
           << " events : " << theRun->GetTotalEDep()/GeV << " [GeV]" << G4endl;
  } else {
    G4cout << "Local thread result with " << theRun->GetNumberOfEvent()
           << " events : " << theRun->GetTotalEDep()/GeV << " [GeV]" << G4endl;
  }
}

Thread safety in user code

Instance of G4Allocator has to be thread-local, and the user has to take care of it for user classes. Typically it is the case for user-defined hit, trajectory and trajectory point classes. Please note that, though G4Allocator was used as a static global object in the past Geant4 versions, it is now an object stored in thread-local storage and thus it must be used through the TLS pointer that is specified by G4ThreadLocal keyword. To use G4ThreadLocal keyword, include "globals.hh" or "G4Types.hh" header file. In the sequential mode, G4ThreadLocal keyword is ignored. Please note that objects instantiated with the thread-local G4Allocator MUST NOT be deleted by other thread (master or other worker).

This code snippet below is an example for a hit class.

#include "G4VHit.hh"
#include "G4THitsCollection.hh"
#include "G4Allocator.hh"
#include "G4Types.hh"

class MyHit : public G4VHit
{
  public:
   . . .
    inline void* operator new(size_t);
    inline void  operator delete(void*);
   . . .
};

typedef G4THitsCollection<MyHit> MyHitsCollection;
extern G4ThreadLocal G4Allocator<MyHit>* MyHitAllocator;

inline void* MyHit::operator new(size_t)
{
  if(!MyHitAllocator) MyHitAllocator = new G4Allocator<MyHit>;
  return (void *) MyHitAllocator->MallocSingle();
}

inline void MyHit::operator delete(void *hit)
{
  MyHitAllocator->FreeSingle((MyHit*) hit);
} 

G4ThreadLocal G4Allocator<MyHit>* MyHitAllocator = 0;

MT specific UI commands for G4cout/G4cerr

• /control/cout/useBuffer flag
  • Store G4cout and/or G4cerr stream to a buffer so that output of each thread is grouped.
  • The buffered text will be printed out on a screen for each thread at a time at the end of the job or at the time the user changes the destination to a file.
  • This command has no effect if output goes to a file.
• /control/cout/ignoreThreadsExcept threadID
  • Omit output from threads except the one from the specified thread.
  • If threadID is greater than the actual number of threads, no output is shown from worker threads.
  • To reset, use -1 as threadID.
• /control/cout/setCoutFile fileName ifAppend
  • Send G4cout stream to a file dedicated to each thread. The file name has "G4W_n_" prefix where n represents the thread ID.
  • File name may be changes for each run. If ifAppend parameter is false, the file is overwritten when exactly the same file has already existed.
  • To change the G4cout destination back to the screen, specify the special keyword "**Screen**" as the file name.
• /control/cout/setCerrFile fileName ifAppend
  • Send G4cerr stream to a file dedicated to each thread. The file name has "G4W_n_" prefix where n represents the thread ID.
  • File name may be changes for each run. If ifAppend parameter is false, the file is overwritten when exactly the same file has already existed.
  • To change the G4cerr destination back to the screen, specify the special keyword "**Screen**" as the file name.
• /control/cout/prefixString prefix
  • In case G4cout and/or G4cerr are not buffered, output of all threads are displayed on the screen simultaneously.
  • With this command, the user may specify a prefix for each output line which is supplemented by the thread ID.

Random numbers

Each thread has its own sequence of random numbers. Before the BeamOn, the user may set the seeds to the "master" random number engine. In the UserEventAction, the user has an access to the status of random number engine through G4Event.

An exception to this rule is the instantiation of a random number engine. This is typically done in the main() function. This should read (note the type of defaultEngine variable):

#include "Randomize.hh" 
int main(int argc,char** argv) { 
  CLHEP::RanluxEngine defaultEngine( 1234567, 4 ); 
  G4Random::setTheEngine( &defaultEngine ); 
  G4int seed = time( NULL ); 
  G4Random::setTheSeed( seed ); 

Additional information on seeding of random numbers can be found here.

G4UserWorkerInitialization class

• virtual void WorkerInitialize() const
  • This method is called after the tread is created but before the G4WorkerRunManager is instantiated.
• virtual void WorkerStart() const
  • This method is called once at the beginning of the worker thread when kernel classes and user action classes have already instantiated but geometry and physics have not yet been initialized. This situation is identical to "PreInit" state in the sequential mode.
• virtual void WorkerRunStart() const
  • This method is called before an event loop. Geometry and physics have already been set up for the thread. All threads are synchronized and ready to start the local event loop. This situation is identical to "Idle" state in the sequential mode.
• virtual void WorkerRunEnd() const
  • This method is called for each thread when the local event loop is done, but before the synchronization over threads.
• virtual void WorkerStop() const
  • This method is called once at the end of the life of a worker thread.

Advanced:

How to customize threading model

This topic is discussed in detail in this page: Geant4MTAdvandedTopicsForApplicationDevelopers

Mixing MPI and multi-threading

This topic is discussed in detail in this page: Geant4MTAdvandedTopicsForApplicationDevelopers

Q&A on thread-safety

Speedup visualization in MT (valid form version 10.2 and later)

As of Geant4 10.2, visualisation is handled in multi-threaded mode in a separate thread. This can lead to slow performance as worker threads wait for the visualisation thread to complete. To control this, multithreaded mode simulations that include visualisation may need to consider the following macro commands:

• /vis/multithreading/actionOnEventQueueFull discard/wait: Discard events, or wait for events to finish when the event queue for drawing gets full.
• /vis/multithreading/maxEventQueueSize : define maximum event queue size to be (0 for infinite).