1) Static of a class type

The G4ThreadLocal keyword can be used only for simple data types: G4doule, G4int, ... and pointers. It cannot be used for classes. If you have static variable of some class type some additional work is needed. The automatic conversion has probably added quite some code to your files. Removing of this code may be tricky (see the document Mini guide), please coordinate with Andrea

2) Lazy initialization of shared variables

Consider the case in which you have a simple static variable that you assign a value to at first use, for example at first event. If the value is calculated via some complex calculation it makes perfect sense to use static to reduce computation. However I am not completely sure how this will behave in a multi-threaded application. My suggestion is to leave the G4ThreadLocal keyword. Let's discuss about that!

3) Adding a new particle

If you define a new particle (e.g. diproton, dineutron) this has to be a G4ThreadLocal variable, since the dictionaries of G4ParticleTable are G4ThreadLocal themselves. See as an example: processes/hadronic/models/cascade/cascade/include/G4Diproton.hh

4) Why I cannot simply add G4ThreadLocal to a declaration of a variable for a generic class data type?

Only very simple data types can be declared as G4ThreadLocal. See 8) for details. See: http://gcc.gnu.org/onlinedocs/gcc-3.3.1/gcc/Thread-Local.html#Thread-Local. This imply that you cannot transform this is not allowed:

static G4ThreadLocal G4Class datum;

5) Why static G4ThreadLocal G4Class* data = new G4Class() is not valid?

See previous point, operator new is not a const-expression as defined by C++ standard, so simply you cannot do it. To achieve the previous do:

static G4ThreadLocal G4Class* data = 0; //0 is a const-expression
if ( !data) data = new G4Class;

6) Originally I've created a static const variable, but after the code transformation the variable is now static G4ThreadLocal and const disappeared, why?

We have seen this in Bertini code. This is because the original variable is used in the code with some other non const static variable in a non-trivial way. Consider the following case (inspired by Bertini):

static const double anArray[] = { 1, 2 , 3 };
static SomeThing anObject( anArray );

The problem is that anObject is not declared const, this mean that the automatic tool transformed anObject it to be G4ThreadLocal. Due to implementation details (the real case is a bit too long to be discussed here) the first const needed to be removed and anArray now becomes a G4ThreadLocal. In the specific case of Bertini, the intended use of anObject was actually "read-only" thus a const keyword was missing, adding const to the declaration of anObject allows one to leaving the const also to anArray and avoid G4ThreadLocal. Conclusion: always use const if you intend to use the variable as a read-only object. If you notice cases of "disappearing" const keyword in your code, please contact Andrea since some discussion is needed!

7) I need to implement a shared resource and I need to add a mutexing mechanism to it. How can I do it in a portable way?

We have prepared a proposal to develop G4 "style" mutexes and locks. The code for this is contained in tag: global-V09-06-02 (currently in proposed state!). Switch to this tag the source/global category, then let's say that you want to give possibility to access a shared resource in your code, you can do:

#include "G4Threading.hh"
#include "G4AutoLock.hh"

namespace {
  NonThreadSafeDataStructure sharedResource; 
  G4Mutex aMutex=G4MUTEX_INITIALIZER;
}

void myfunc() {
   G4AutoLock l(&aMutex);
   //Do some non-thread safe operations
       sharedReource.doSomething();
       l.unlock();
      //Do some thread safe operations
      //...
      //Again some non-thread safe operations
     l.lock();
     sharedResource.doSomething();
     return;
} //aMutex is unlocked automatically when l goes out of scope!

The use of anonym nameespace for file-scope global variable is a good practice, see: http://stackoverflow.com/questions/154469/unnamed-anonymous-namespaces-vs-static-functions

8) What are exactly the data types allowed to be G4ThreadLocal?

The G4ThreadLocal keyword can be used in simple data types. Please refer to here for a complete description: http://gcc.gnu.org/onlinedocs/gcc-3.3.1/gcc/Thread-Local.html#Thread-Local. Summarizing:

• The variable must be global, file-scoped static, or static data member of a class
• The variable must be of trivial type (double, bool) an array or a struct. If it is a class it must be of trivial type (see definition). The definition of trivial is so stict that no class used in Geant4 can be considered trivial. As an example any class that has a constructor is not trivial.

Examples of allowed/not allowed G4ThreadLocal data types:

//==============
//My data types
//==============
struct mytype {
};

class A {
public:
  A() { }; //The presence of a constructor is the problem here: class A is not anymore trivial
};

class B {
private:
  int b;
};
//==============


//Example of use of G4ThreadLocal keyword
class C {
private:
  static G4ThreadLocal double aData; //thread can be used for static data types
  //G4ThreadLocal double anotherData; //Non-static, this is not valid
};

//These are at global scope, it is ok
G4ThreadLocal double a=0;
G4ThreadLocal double b[] = { 1., 2.}; 
G4ThreadLocal mytype c;
//G4ThreadLocal A d; //This is not allowed
G4ThreadLocal B e; //This is allowed!!!


void function()
{
  static G4ThreadLocal double f = 1.;
  //G4ThreadLocal double g = 1.; //This is not allowed, it is not static
}

9) I want to create a shared instance of a singleton for all threads (e.g. a central analysis manager). What should I do?

Since it is a shared instance multiple access to it is possible from the worker threads, you need to use a mutex to protect its use: see point 7). In case of a singleton there is an extra protection to take into consideration that even the accessing the singleton instance itself has to be protected. In Geant4 we typically have the following code to implement the singleton pattern that is now thread-safe:

class G4Singleton {
private:
   static G4Singleton* instance;
   G4Singleton() { }
public:
   static G4Singleton* GetInstance();
};

G4Singleton* G4Singleton::instance = 0;

#include "G4AutoLock.hh"
namespace {
  G4Mutex singletonMutex = G4MUTEX_INITIALIZER;
}

G4Singleton* G4Singleton::GetInstance()
{
  G4AutoLock l(&singletonMutex);
  if ( !instance ) instance = new G4Singleton();
  return instance;
}

This mean that you may be paying some performance due to the lock every time you need access to the singleton instance. A different approach if you are concerned for perfomance (and that I personally encourage) is to guarantee that the singleton instance is created before threads are started. For example calling G4Singleton::GetInsance() in the main() function of your application. You may be wondering if another way of solving this problem exist, you can take a look here for a discussion on singleton and MT: http://en.wikipedia.org/wiki/Double-checked_locking.

Additionally you need to protect access to all data members that are not invariant. To be on the safe side and if performances are not an issue, just add an additional mutex, and use this mutex in ALL methods of your class that access/manipulate data members that are not invariant. For example:

namespace {
   G4Mutex dataManipMutex = G4MUTEX_INITIALIZER;
}

void G4Singleton::DoSomething(G4Event* evt)
{
   G4AutoLock l(&dataManipMutex);
   //...
}

Not that you should not mutex the methods using the same mutex used to protect the GetInstance() method. Important if you need to access the singleton very often (for example for each G4Step) the use of a shared instance is not a good idea: performance will be strongly reduced. The best way to work in this case is to split the singleton in two: one for steps implemented with G4ThreadLocal technology, the second one (not used often) with mutex. Ask yourself: are the methods of the class accessed more than once per event or are the events of my application very simple (and thus fast to be processed)? If the answer is yes, than a global singleton with mutex is probably not optimal (for a discussion of the topic see the paper from X. Dong et al.: "Euro-Par 2010 - Parallel Processing Lecture Notes in Computer Science Volume 6272, 2010, pp 287-303).

10) Splitting of classes

This has been moved to: Geant4MTForKernelDevelopers

11) How are exactly random-numbers seeded?

The use of random number generators is a bit different in multi-threaded mode with respect to sequential mode. To guarantee reproducibility independently of the number of threads some modifications have been made:
The master (main program) can be seeded as it is in sequential mode via G4Random::setTheSeed( … ) (and UI command). To guarantee that RNG numbers do not depend on the number of threads the "history" of random-seeds is not as in sequential. Each thread has its own RNG engine and the status of these are independent. Before the run loop starts the master pre-generates a random seed for each event to be processed. Then threads are spawned and event loop proceeds, at the beginning of each event the worker thread re-seeds the event with this pre-generated map of seeds from the master thread. While this strategy guarantees the thread sequence is independent of threads unfortunately has the drawback that the sequence is different in a sequential or MT build. Some more info on random UI commands:

/random/setSavingFlag 1

By default this will work as in sequential, the "master" run seed file will be called currentRun.rndm; since the master does not process any event the currentEvent.rndm does not make sense and should be ignored. Each thread writes out the equivalent files with a pre-fix string: G4WorkerNNN_currentRun.rndm and G4WorkerNNN_currentEvent.rndm where NNN is the thread ID.

Since may events are processed in parallel the notion of "currentEvent" does not make sense anymore. To solve this problem we have introduced a new UI command:

/random/saveEachEventFlag 1

If set each event RN is stored in a separate file: G4WorkerNNN_runXXXEventYYYY.rndm (note that YYYY is the "global" event number, not the "per-thread" event number).

Note on reading back engine status via

/random/resetEngineFrom filename

Here what happens: the master thread will re-seed itself from this random seed, and will use this to pre-generete the seeds for each event. A note: currently worker threads will also re-seed their "local" run from this file, but since each event is anyway re-seeded this command has no practical effect for workers. We will review this strategy to allow a specific thread to be re-seeded from a given file.