Currently known issues on 10.0-beta01 - Read this section FIRST

Comments This is a beta-release of Geant4 version 10.0. We have changes some interfaces for the production release. Please refer to the release note of version 10.0 for the revised version of the migration guide.

Comments This document is incomplete, but currently meant to be used for the beta-testers to start migrating their Geant4 applications to multi-thread and provide us their feedbacks .Suggestions are appreciated. We will merge this document to the Application Developers Guide at the time of the final version10.0 release.

Comments It is distributed "as is", and full support cannot be provided. Some code may be new or enhanced, therefore still experimental and not fully tested. We don't have full coverage of all the physics/geometry options/phase-spaces for this beta-release. There may be some options/phase-spaces that do not work properly due to thread unsafe implementation. They may cause strange warning messages, event abortions, rare infinite loop or rare crash at random locations.

  • We request every beta-testers to report any instability or suspicious results. Also, check this TWiki page regularly for the known issues.

Following issues and limitations are known as of June 28th, 2013 UPDATED

  1. Though you may issue more than one BeamOn, you cannot change the number of threads after the first run has started. G4MTRunManager::SetNumberOfThreads() method and corresponding /run/numberOfThreads UI command work only up to the first BeamOn.
  2. Radioactive decay and Neutron-HP are not yet supported in multi-threaded mode. Use sequential or single-thread mode. Also, shooting ions with arbitrary excitation energy as primary particles is not supported in multi-threading mode, and we may drop this feature of shooting a primary ion with arbitrary excitation energy from v10.0.
  3. In the multi-threaded mode, the program does not abort for the case that a UI command in a macro file is not found. Such an illegal command is simply ignored.
  4. Event display (including hits) during the event loop is not yet working. If you need to visualize events, keep events (by invoking fEventManager->KeepTheCurrentEvent() method from EndOfEventAction() of your UserEventAction, and include G4EventManager.hh) and use /vis/reviewKeptEvents command after the event loop.
  5. Some objects are not cleanly deleted at the termination of the thread/program.
  6. MT mode on Windows OS is not yet supported.

Examples in 10.0-beta01 that have migrated to multi-threading.

  • Basic examples
    • All basic examples of B1 through B4
  • Extended examples
    • electromagnetic
      • TestEm4
    • Optical
      • OpNovice, wls
    • runAndEvent
      • RE03, RE05
  • Advanced examples
    • dnaphysics
    • microdosimetry

Introduction

This document describes how to make a Geant4 application that works both in sequential and in multi-threaded modes. If a user has an application that is running in geant4 v9.6 and would like to stick to the sequential mode for the new version 10.0, (s)he does not need to apply the modifications this document describes.

To use the multi-threaded version, Geant4 library must be built with the following CMAKE option
-DGEANT4_BUILD_MULTITHREADED=ON
In addition, for Mac OS X 10.7, the user has to use CLANG compiler by specifying the following two CMAKE options for both of building a library and building an application
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
  • If the user is using the external CLHEP, it also has to be compiled with these CLANG options.

The basics: per-thread vs shared classes

In the multi-threaded mode, generally saying, data that are stable during the event loop are shared among threads while data that are transient during the event loop are thread-local. In general, geometry and physics tables are shared, while event, track, step, trajectory, hits, etc., as well as several Geant4 manager classes such as EevntManager, TrackingManager, SteppingManager, TransportationManager, FieldManager, Navigator, SensitiveDetectorManager, etc. are thread-local. Among the user classes, user initialization classes (G4VUserDetectorConstruction, G4VUserPhysicsList and newly introduced G4VUserActionInitialization) are shared, while user action classes and sensitive detector classes are thread-local. It is not straightforward (and thus not recommended) to access from a shared class object to a thread-local object, e.g. from detector construction to stepping action. Please note that thread-local objects are instantiated and initialized at the first BeamOn. To avoid potential errors, it is advised to always keep in mind which class is shared and which class is thread-local.

Please note the following:

  • Every process must be thread-local. If you have your own implementation of PhysicsList or PhysicsBuilder, don't instantiate a process in a constructor, as PhysicsList and PhysicsBuilder are shared. Make sure that all process objects are instantiated in the ConstructProcess() method, which is invoked for every thread. Also, don't use a flag data member to protect the ConstructProcess() method.
  • If a messenger class is instantiated by a tread-local class, UI commands belonging to that messenger class are properly delivered to the target class objects of all threads. If the messenger is instantiated by a shared class, command cannot be delivered to thread-local objects. Special attention is required for setting the electromagnetic field value (see Detector construction section).
  • If the user uses G4Allocator for his/her own class, e.g. hit, trajectory or trajectory point, G4Allocator object must be thread local and thus must be instantiated within the thread. The object new-ed and allocated by the thread-local G4Allocator must be deleted within the same thread (see Thread safety in user code section).
  • SteppingVorbose must be thread-local. If the user uses the default G4SteppingVorbose, it is taken care automatically and the user does not need to do anything. If the user wants to use his/her own SteppingVerbose, it must be instantiated and registered in G4UserWorkerInitialization class.

G4MTRunManager, G4WorkerRunManager

G4MTRunManager is the replacement of G4RunManager for multi-threading mode. When the first BeamOn() is issued, G4MTRunManager creates and starts threads and the event loop is distributed to the threads. The event ID for each thread is assigned as round robin, so that event numbers each thread has are not sequential. G4WorkerRunManager is the local RunManager instantiated by G4MTRunManager to take care of initialization and event loop of a thread. Both G4MTRunManager and G4WorkerRunManager are derived classes from G4RunManager base class. The static method G4RunManager::GetRunManager() returns the following pointer.
  • It returns the pointer to the G4WorkerRunManager of the local thread when it is invoked from thread-local object.
  • It returns the pointer to the G4MTRunManager when it is invoked from shared object.
  • It returns the pointer to the base G4RunManager if it is used in the sequential mode.
G4RunManager has a method GetRunManagerType() that returns an enum named RMType to indicate what kind of RunManager it is. RMType is defined as { sequentialRM, masterRM, workerRM }. From the thread-local object, a static method G4MTRunManager::GetMasterRunManager() is available to access to G4MTRunManager. From a worker thread, the user may access to, for example, detector construction (it is a shared class) through this GetMasterRunManager() method.

For application that have a user-defined run-manager, refer to the Advanced: How to customize threading model section.

main()

The only difference in user's application for sequential and multi-threaded modes is in the main(). For the sequential mode, G4RunManager should be instantiated, while for the multithreaded mode G4MTRunManager should be instantiated. Please note that the pre-processor flag G4MULTITHREADED is set when the Geant4 library is built with the CMAKE option GEANT4_BUILD_MULTITHREADED=ON. In the multi-threaded mode, the user may also specify the number of threads to be used by SetNumberOfThreads() method. Number of threads may also be set by a UI command "/run/numberOfThreads".

In the main() function, the user should instantiate only the user initialization classes (G4VUserDetectorConstruction, G4VUserPhysicsList and newly introduced G4VUserActionInitialization) and set them to run manager through SetUserInitialization() method. All the user action classes must be instantiated in G4VUserActionInitialization class.

main()
#ifdef G4MULTITHREADED
#include "G4MTRunManager.hh"
#else
#include "G4RunManager.hh"
#endif

int main()
{
#ifdef G4MULTITHREADED
  G4MTRunManager* runManager = new G4MTRunManager;
  runManager->SetNumberOfThreads(<number_of_threads>);
#else
  G4RunManager* runManager = new G4RunManager;
#endif

  runManager->SetUserInitialization(new MyDetectorConstruction);
  runManager->SetUserInitialization(new FTFP_BERT);
  runManager->SetUserInitialization(new MyActionInitialization);

  . . .

  delete runManager;
}

Please note the following.

  • The user may instantiate the base-class G4RunManager even if (s)he uses the Geant4 library that is built as multi-threaded. In this case, (s)he does not need to have G4VUserActionInitialization but may set all the action classes to the base-class G4RunManager. It runs in sequential mode, potentially with minor performance overhead.
  • If the user wants to use his/her own SteppingVerbose, it must be instantiated and registered in G4UserWorkerInitialization class. This is an exception that the user's code has to be different in sequential and multi-threaded modes.

User defined physics list

If the user has a user-implemented physics list and thus does not use one of the pre-packaged physics lists, (s)he needs to add G4GenericIon::GenericIonDefinition() into his/her ConstructParticle() method. This ensures that the generic_ion particle is created at initialization. Note (s)he needs this line even if his/her physics list does not explicitly use ion physics.

Every process must be thread-local. If the user has his/her own implementation of PhysicsList or PhysicsBuilder, don't instantiate a process in a constructor, as PhysicsList and PhysicsBuilder are shared. Make sure that all process objects are instantiated in the ConstructProcess() method, which is invoked for every thread.

Detector construction

G4VUserDetectorConstruction now has a new virtual method ConstructSDandField().

Sensitive detector

Given sensitive detector class objects should be thread-local, instantiation of such thread-local classes should be implemented in this new ConstructSDandField() method, which is invoked for each thread. Construct() method should contain definition of materials, volumes and visualization attributes. To define a sensitive detector in ConstructSDandField() method, a new protected method SetSensitiveDetector("LVName",pSD) is available to make ease of migration, This SetSensitiveDetector("LVName",pSD) method does two things:

  1. Register the sensitive detector pointer pSD to G4SDManager, and
  2. Set pSD to the logical volume named "LVName".
Please note that SetSensitiveDetector("LVName",pSD) assumes that "LVName" is unique. If this is not the case, the user need to specify the pointer of the logical volume which the sensitive detector should be assigned to, and use SetSensitiveDetector(G4logicalVolume*, G4VSensitiveDetector*) method instead. The ConstructSDandField() method is invoked from the base-class G4RunManager for sequential mode.

If the user needs to define sensitive detector(s) to the volumes defined in a parallel world, (s)he may do so by implementing G4VUserParallelWorld::ConstructSD() method. Please note that defining field in a parallel world is not supported.

MyDetectorConstruction.cc
G4VPhysicalVolume* MyDetectorConstruction::Construct()
{
   . . .  // material definition
   . . .  // world volume
   G4VPhysicalVolume* worldPV = new G4PVPlacement(...);

   . . .  // detector
   G4LogicalVolume* mySDLV = new G4LogicalVolume(...,"MySDLV");
   . . . 

   return worldPV;
}

void MyDetectorConstruction::ConstructSDandField()
{
   MySensitiveDetector* mySD = new MySensitiveDetector(...);
   SetSensitiveDetector("MySDLV", pMySD);
}

Alternatively, if the user implements Clone() virtual method for all of his/her individual sensitive detector classes, (s)he may define his/her sensitive detectors and assign them to the logical volumes in the Construct() method, and then invoke G4VUserSensitiveDetector::CloneSD() utility method from ConstructSDandField().

Field

If the user's field class is simple and does not have a cache mechanism, there can be only one field instance shared by all threads. If the field class has a cache mechanism, such field class object must be thread-local. Please note that G4TransportationManager is thread-local. Thus, regardless of whether the field class object is shared or thread-local, a field object must be assigned to G4FieldManager in the ConstructSDandField() method, or in some other method invoked at thread-local place. If the user wants to change the field values between runs using his/her UI command, special treatment is required. Please keep in mind that ConstructSDandField() method is invoked for the second or later runs only if the geometry is modified. If the field value is changed without changing geometry, this ConstructSDandField() is not invoked. The following is the simplest recipe.

  1. Instantiate field class object in the ConstructSDandField() method so that the field object is thread-local.
  2. In the constructor of the field class, instantiate a dedicated messenger with a field UI command, so that the messenger and the command are thread-local.
  3. When the field value is set to this field class object (which is thread-local), access to G4TransportationManager and set the field and create ChordFinder.

MyDetectorConstruction.hh MyDetectorConstruction.cc
#include "G4Types.hh"
#include "G4VUserDetectorConstruction.hh"
#include "MyField.hh"
class MyDetectorConstruction : public G4VUserDetectorConstruction
{
   . . .
 private:
   static G4ThreadLocal MyField* myField;
};
 
G4ThreadLocal MyField* MyDetectorConstruction::myField = 0;

void MyDetectorConstruction::ConstructSDandField()
{
  if(!myField) myField = new MyField(); 
}
MyField.hh MyField.cc
class MyField : public G4Field (or other G4 field class)
{
 public:
   MyField();
   virtual ~MyField();

   virtual void GetFieldValue(const double*, double* ) const;
   void SetFieldValue(G4ThreeVector);

 private:
   MyFieldMessenger* messenger;
   G4ThreeVector fieldVal;
};
MyField::MyField()
{
  messenger = new MyFieldMessenger(this);
  SetFieldValue(G4ThreeVector(0.,0.,0.));
}

~MyField::MyField()
{ delete messenger; }

void MyField::GetFieldValue(const double*, double* BF)
{ BF[0]=fVal.x(); BF[1]=fVal.y(); BF[2]=fVal.z(); }

void MyField::SetFieldValue(G4ThreeVector fVal)
{
  fieldVal = fVal;
  G4FIeldManager* fMan
    = G4TransportationManager::GetTransportationManager()
        ->GetFieldManager();
  fMan->SetDetectorField(this);
  fMan->CreateChordFinder(this);
}

G4VUserActionInitialization class

G4VUserActionInitialization is a newly introduced class for the user to instantiate user action classes (both mandatory and optional). As described in the previous section, all the user action classes are thread-local, with the only exception of UserRunAction, which could be defined for both thread-local and global. G4VUserActionInitialization has two virtual method to be implemented, one is Build() and the other is BuildForMaster(). Build() should be used for defining user action classes for local threads (a.k.a. workers) as well as for the sequential mode. BuildForMaster() should be used for defining only the UserRunAction for the global run (a.k.a. master). All user actions must be registered through SetUserAction() protected method defined in the G4VUserActionInitialization base class.

MyActionInitialization.cc
void MyActionInitialization::Build() const
{
  SetUserAction(new MyPrimaryGeneratorAction);
  SetUserAction(new MyRunAction);
  SetUserAction(new MySteppingAction);
  . . .
}

void MyActionInitialization:BuildForMaster() const
{
  SetUserAction(new MyRunAction);
}

The local run manager may have UserRunAction for dealing with thread-local things, while the global G4MTRunManager may also have its own UserRunAction for the global run. UserRunAction to be registered to the master for the global run could be an object of the same class defined for the workers, or an instance of a dedicated class. If UserRunAction class is shared by workers and master, a protected method IsMaster() is available to distinguish whether it is used for a worker or for the master (an example implementation is here).

Please note:

  • IsMaster() returns true if it is used in the sequential mode. Also, IsMaster() does not return a correct value until that RunAction object is set to a RunManager. Thus, don't use IsMaster() method in the constructor of RunAction.
  • Both Build() and BuildForMaster() are const methods. This means that, even though the user instantiates user action classes and register them, (s)he cannot change any data member of this class. This restriction comes from the fact that this G4VUserActionInitialization class object is shared by all threads.
  • BuildForMaster() is not invoked for the sequential mode. Thus, if the user implement a dedicated UserRunAction for the master thread, it won't be used at all for the sequential mode. If the user wants exactly the same output for the sequential mode and the global run action, (s)he should use a single UserRunAction class and use IsMaster() method which returns true for both sequential mode and the master thread in multi-threaded mode.
  • If the user need an access from thread-local user action class (e.g. stepping action or primary generator action) to the shared user initialization class (e.g. detector construction), (s)he may do so through G4MTRunManager::GetMasterRunManager() to get the pointer of the G4MTRunManager and use that pointer to get the pointer of the user initialization class. Alternatively, the user's ActionInitialization class constructor may take the pointer of the user initialization class.
    • As mentioned earlier, access from shared user initialization class to thread-local user action class is very difficult and not recommended.
    • G4MTRunManager::GetMasterRunManager() returns a null pointer if it is used in the sequential mode.

Input data file for PrimaryGeneratorAction

The implementation depends on the use-case, more precisely, on the ratio of the cost of accessing to the data file to the total execution time of one event on one thread. If this ratio is low, i.e. execution time dominates (typically the case of high-energy physics experiment where the energy of primary particles are high), the file access can be shared with Mutex locking mechanism. If this ratio is high, i.e. file access is a significant fraction of the execution time (typically the case of medical and space applications where the energy of primary particles are rather low), the input data should be cached in addition to the Mutex locking. To use Mutex locking mechanism, include G4AutoLock.hh header file.

Here is a high-energy physics sample code with G4HEPEvtInterface that reads a Pythia input file. G4HEPEvtInterface has to be a single object shared by all the PrimaryGeneratorAction objects, and the access to G4HEPEvtInterface::GeneratePrimaryVertex() should be protected by Mutex.

MyHepPrimaryGenAction.hhSorted ascending MyHepPrimaryGenAction.cc
#include "G4VUserPrimaryGeneratorAction.hh"
class G4HEPEvtInterface;
class MyHepPrimaryGenAction
 : public G4VUserPrimaryGeneratorAction
{
 public:
   MyHepPrimaryGenAction(G4String fileName);
   ~MyHepPrimaryGenAction();
   . . .
   virtual void GeneratePrimaries(G4Event* anEvent);
 private:
   static G4HEPEvtInterface* hepEvt;
};
#include "MyHepPrimaryGenAction.hh"
#include "G4HEPEvtInterface.hh"
#include "G4AutoLock.hh"
namespace { G4Mutex myHEPPrimGenMutex = G4MUTEX_INITIALIZER; }
G4HEPEvtInterface* MyHepPrimaryGenAction::hepEvt = 0;

MyHepPrimaryGenAction::MyHepPrimaryGenAction(G4String fileName)
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  if( !hepEvt ) hepEvt = new G4HEPEvtInterface( fileName ); 
}

MyHepPrimaryGenAction::~MyHepPrimaryGenAction()
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  if( hepEvt ) { delete hepEvt; hepEvt = 0; } 
}

void MyHepPrimaryGenAction::GeneratePrimaries(G4Event* anEvent)
{
  G4AutoLock lock(&myHEPPrimGenMutex);
  hepEvt->GeneratePrimaryVertex(anEvent);
}

Here is a lower-energy sample code with G4ParticleGun to shoot 10 MeV electrons. The input file contains list of G4ThreeVector of the momentum direction (ex, ey, ez) and it is read by a dedicated file reader class MyFileReader. MyFileReader is shared by all threads, and reads 100 events at a time and buffers them. Primary vertex position is randomized. Please note that, for the simplicity of this sample code, it does not consider the end-of-file.

MyLowEPrimaryGenAction.hh MyLowEPrimaryGenAction.cc
#include "G4VUserPrimaryGeneratorAction.hh"
class G4ParticleGun;
class MyFileReader;
class MyLowEPrimaryGenAction
 : public G4VUserPrimaryGeneratorAction
{
 public:
   MyLowEPrimaryGenAction(G4String fileName);
   virtual ~MyLowEPrimaryGenAction();
   . . .
   virtual void GeneratePrimaries(G4Event* anEvent);
 private:
   static MyFileReader* fileReader;
   G4ParticleGun* particleGun;
};
#include "G4ParticleTable.hh"
#include "G4ParticleDefinition.hh"
#include "Randomize.hh"
#include "G4AutoLock.hh"
namespace { G4Mutex myLowEPrimGenMutex = G4MUTEX_INITIALIZER; }
MyFileReader* MyLowEPrimaryGenAction::fileReader = 0;

MyLowEPrimaryGenAction::MyLowEPrimaryGenAction(G4String fileName)
{
  G4AutoLock lock(&myLowEPrimGenMutex);
  if( !fileReader ) fileReader = new MyFileReader( fileName ); 
  particleGun = new G4ParticleGun(1);
  G4ParticleTable* particleTable = G4ParticleTable::GetParticleTable();
  G4ParticleDefinition* particle = particleTable->FindParticle("e-");
  particleGun->SetParticleDefinition(particle);
  particleGun->SetParticleEnergy(10.*MeV);
}

MyLowEPrimaryGenAction::~MyLowEPrimaryGenAction()
{
  G4AutoLock lock(&myLowEPrimGenMutex);
  if( fileReader ) { delete fileReader; fileReader = 0; }
}

void MyLowEPrimaryGenAction::GeneratePrimaries(G4Event* anEvent)
{
  G4ThreeVector momDirction(0.,0.,0.);
  if(fileReader)
  {
    G4AutoLock lock(&myLowEPrimGenMutex);
    momDirection = fileReader->GetAnEvent();
  }
  particleGun->SetParticleMomentumDirection(momDirction);
  G4double x0 = 2.* Xmax * (G4UniformRand()-0.5);
  G4double y0 = 2.* Ymax * (G4UniformRand()-0.5);
  particleGun->SetParticlePosition(G4ThreeVector(x0,y0,0.));
  particleGun->GeneratePrimaryVertex(anEvent);
}
MyFileReader.hh MyFileReader.cc
#include <list>
#include <fstream>
class MyFileReader
{
 public:
   MyFileReader(G4String fileName);
   ~MyFileReader();
   . . .
   G4ThreeVector GetAnEvent();
 private
   std::ifstream inputFile;
   std::list<G4ThreeVector> evList;
};
MyFileReader::MyFileReader(G4String fileName)
{ inputFile.open(filename.data()); }

MyFileReader::~MyFileReader()
{ inputFile.close(); }

G4ThreeVector MyFileReader::GetAnEvent()
{
  if( evList.size() == 0 )
  {
    for(int i=0;i<100;i++)
    {
      G4double ex, ey, ez;
      inputFile >> ex >> ey >> ez;
      evList.push_back( G4ThreeVector(ex,ey,ez) );
    }
  }
  G4ThreeVector ev = evList.pop_front();
  return ev;
}

Please note that Mutex has a performance penalty. Thus, in case the user uses many threads and the execution time of one event is short, the most efficient way is splitting the input file to the number of threads so that each thread reads its own dedicated input file without Mutex lock. In this case, the buffering shown in above-mentioned MyFileReader class should be used to reduce the file I/O overhead.

Collecting thread-local data

To collect data over threads at the end of an event loop, derived class of G4Run should be used. The base class defines the following virtual methods.

  • void RecordEvent(const G4Event*)
    • Method to be overwritten by the user for recording events in this (thread-local) run. Invoke G4Run base-class method for recording data member in the base class.
    • If the user in the past has implemented an access to hits or scores in his/her UserEventAction::EndOfEventAction(), (s)he can easily move that code segment to this RecordEvent() method. See the following sample code.
  • void Merge(const G4Run*)
    • Method to be overwritten by the user for merging local Run object to the global Run object. Invoke G4Run base-class method to merge data member in the base class.

If the user uses the built-in command-based scorer, scores are automatically accumulated to the global run.

MyRun.hh MyRun.cc
#include "G4Run.hh"
class MyRun : public G4Run
{
 public:
  MyRun();
  virtual ~MyRun();
  virtual void RecordEvent(const G4Run*);
  virtual void Merge(const G4Run*);
  inline G4double GetTotalEDep() const
  { return fEDep; }
 private:
  G4int fEDepScoreID;
  G4double fEDep;
};
#include "G4SDManager.hh"
MyRun::MyRun()
:fEDep(0.)
{
 fEDepScoreID = G4SDManager::GetSDManager()->GetCollectionID("myDet/myEDepScorer"); 
}

void MyRun::RecordEvent(const G4Event* evt)
{
  G4HCofThisEvent* HCE = evt->GetHCofThisEvent();
  if(HCE) {
   // This part of the code could be replaced to any kind of access
   // to hits collections and scores
   G4THitsMap<G4double>* evtMap = (G4THitsMap<G4double>*)(HCE->GetHC(fEDepScoreID));
   std::map<G4int,G4double*>::iterator itr = evtMap->GetMap()->begin();
   for(; itr != evtMap->GetMap()->end(); itr++) { fEDep += *(itr->second); }
  }
  G4Run::RecordEvent(evt);
}

void MyRun::Merge(const G4Run* aRun)
{
  const MyRun* localRun = static_cast<const MyRun*>aRun;
  fEDep += localRun->fEDep;
  G4Run::Merge(aRun);
} 

User-defined Run class must be instantiated by the user's RunAction class.

MyRunAction.cc
G4Run* MyRunAction::GenerateRun()
{ return new MyRun; }

void MyRunAction::EndOfRunAction(const G4Run* aRun)
{
  const MyRun* theRun = static_cast<const MyRun*>aRun;
  if(IsMaster())
  {
    G4cout << "Global result with " << theRun->GetNumberOfEvent()
           << " events : " << theRun->GetTotalEDep()/GeV << " [GeV]" << G4endl;
  } else {
    G4cout << "Local thread result with " << theRun->GetNumberOfEvent()
           << " events : " << theRun->GetTotalEDep()/GeV << " [GeV]" << G4endl;
  }
}

Thread safety in user code

Instance of G4Allocator has to be thread-local, and the user has to take care of it for user classes. Typically it is the case for user-defined hit, trajectory and trajectory point classes. Please note that, though G4Allocator was used as a static global object in the past Geant4 versions, it is now an object stored in thread-local storage and thus it must be used through the TLS pointer that is specified by G4ThreadLocal keyword. To use G4ThreadLocal keyword, include "globals.hh" or "G4Types.hh" header file. In the sequential mode, G4ThreadLocal keyword is ignored. Please note that objects instantiated with the thread-local G4Allocator MUST NOT be deleted by other thread (master or other worker).

This code snippet below is an example for a hit class.

MyHit.hh MyHit.cc
#include "G4VHit.hh"
#include "G4THitsCollection.hh"
#include "G4Allocator.hh"
#include "G4Types.hh"

class MyHit : public G4VHit
{
  public:
   . . .
    inline void* operator new(size_t);
    inline void  operator delete(void*);
   . . .
};

typedef G4THitsCollection<MyHit> MyHitsCollection;
extern G4ThreadLocal G4Allocator<MyHit>* MyHitAllocator;

inline void* MyHit::operator new(size_t)
{
  if(!MyHitAllocator) MyHitAllocator = new G4Allocator<MyHit>;
  return (void *) MyHitAllocator->MallocSingle();
}

inline void MyHit::operator delete(void *hit)
{
  MyHitAllocator->FreeSingle((MyHit*) hit);
} 

G4ThreadLocal G4Allocator<MyHit>* MyHitAllocator = 0;

MT specific UI commands

Few commands are specific to MT applications. These commands are available only in PreInit and Idle states. To avoid the necessity of having two separate macro files for sequential and MT modes, these commands are available also in sequential builds, but do not have effect.
  • /run/numberOfThreads nThread
    • Defines the number of worker threads to be used. If nThread is 1, the program runs in single-thread mode (not sequential mode).
    • Currently, this command works only before the first BeamOn.
  • /control/cout/setCoutFile fileName ifAppend
    • Send G4cout stream to a file dedicated to each thread. The file name has "G4W_n_" prefix where n represents the thread ID.
    • File name may be changes for each run. If ifAppend parameter is false, the file is overwritten when exactly the same file has already existed.
    • To change the G4cout destination back to the screen, specify the special keyword "**Screen**" as the file name.
  • /control/cout/setCerrFile fileName ifAppend
    • Send G4cerr stream to a file dedicated to each thread. The file name has "G4W_n_" prefix where n represents the thread ID.
    • File name may be changes for each run. If ifAppend parameter is false, the file is overwritten when exactly the same file has already existed.
    • To change the G4cerr destination back to the screen, specify the special keyword "**Screen**" as the file name.
  • /control/cout/useBuffer flag
    • Store G4cout and/or G4cerr stream to a buffer so that output of each thread is grouped.
    • The buffered text will be printed out on a screen for each thread at a time at the end of the job or at the time the user changes the destination to a file.
    • This command has no effect if output goes to a file.
  • /control/cout/prefixString prefix
    • In case G4cout and/or G4cerr are not buffered, output of all threads are displayed on the screen simultaneously.
    • With this command, the user may specify a prefix for each output line which is supplemented by the thread ID.

Random numbers

Each thread must own its different instance of the chosen random engine. The G4Random class provides this thread-safety, and access to the thread-local random number engine has to be made through calls of the type G4Random::. It must be used in place of CLHEP::Random::, which must not be used anymore. In the sequential mode, G4Random is simply a typedef of CLHEP::Random.

Each thread has its own sequence of random numbers. Before the BeamOn, the user may set the seeds to the "master" random number engine. In the UserEventAction, the user has an access to the status of random number engine through G4Event.

An exception to this rule is the instantiation of a random number engine. This is typically done in the main() function. This should read (note the type of defaultEngine variable):

main()
#include "Randomize.hh" 
int main(int argc,char** argv) { 
  CLHEP::RanluxEngine defaultEngine( 1234567, 4 ); 
  G4Random::setTheEngine( &defaultEngine ); 
  G4int seed = time( NULL ); 
  G4Random::setTheSeed( seed ); 

Additional information on seeding of random numbers can be found here.

G4UserWorkerInitialization class

This class is used only for the multi-threaded mode. The object of this class can be set to G4MTRunManager, but not to G4RunManager. G4UserWorkerInitialization class has five virtual methods as the user hooks which are invoked at several occasions of the life cycle of each thread.
  • virtual void WorkerInitialize() const
    • This method is called after the tread is created but before the G4WorkerRunManager is instantiated.
  • virtual void WorkerStart() const
    • This method is called once at the beginning of simulation job when kernel classes and user action classes have already instantiated but geometry and physics have not been yet initialized. This situation is identical to "PreInit" state in the sequential mode.
  • virtual void WorkerRunStart() const
    • This method is called before an event loop. Geometry and physics have already been set up for the thread. All threads are synchronized and ready to start the local event loop. This situation is identical to "Idle" state in the sequential mode.
  • virtual void WorkerRunEnd() const
    • This method is called for each thread when the local event loop is done, but before the synchronization over threads.
  • virtual void WorkerStop() const
    • This method is called once at the end of simulation job.

User-defined SteppingVerbose must be instantiated before G4WorkerRunManager is instantiated for the thread. Thus it must be instantiated and registered in WorkerInitialize() method. For the sequential mode, user-defined SteppingVorbose still has to be instantiated in main() prior to the instantiation of G4RunManager.

main()
#ifdef G4MULTITHREADED
#include "G4MTRunManager.hh"
#include "MyWorkerInitialization.hh"
#else
#include "G4RunManager.hh"
#include "MySteppingVerbose.hh"
#endif

main()
{
#ifdef G4MULTITHREADED
  G4MTRunManager* runManager = new G4MTRunManager;
  runManager->SetNumberOfThreads(number_of_threads);
  run->SetWorkerInitialization(new MyWorkerInitialization);
#else
  G4VSteppingVerbose* verbosity = new MySteppingVerbose;
  G4VSteppingVerbose::SetInstance(verbosity);
  G4RunManager* runManager = new G4RunManager;
#endif
 
   . . .
}
MyWorkerInitialization.hh MyWorkerInitialization.cc
#include "G4UserWorkerInitialization.hh"
class MyWorkerInitialization : public G4UserWorkerInitialization
{
 public:
   MyWorkerInitialization();
   virtual ~MyWorkerInitialization();

   virtual void WorkerInitialize() const;
};
#include "MyWorkerInitialization.hh"
#include "MySteppingVerbose.hh"

void MyWorkerInitialization::WorkerInitialize() const
{
  G4VSteppingVerbose* verbosity = new MySteppingVerbose;
  G4VSteppingVerbose::SetInstance(verbosity);
}

Advanced: How to customize threading model

This topic is discussed in detail in this page: Geant4MTAdvandedTopicsForApplicationDevelopers

Advanced: Mixing MPI and multi-threading

This topic is discussed in detail in this page: Geant4MTAdvandedTopicsForApplicationDevelopers

Advanced: Q&A on thread-safety

A list of advanced topics is available at this page: Geant4MTTipsAndTricks
Edit | Attach | Watch | Print version | History: r109 < r108 < r107 < r106 < r105 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r108 - 2013-12-05 - MakotoAsai
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Geant4 All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback