Difference: ATeamDataContainers (4 vs. 5)

Revision 52007-04-02 - unknown

Line: 1 to 1
 
META TOPICPARENT name="TrackATeam"

Data Containers for Track Discussions

Line: 20 to 20
 

ObjectVector

Changed:
<
<
This is the lowest level container in Gaudi. It is designed to contain a list of pointers to data objects. Once an object has been inserted into this container the container owns the object. For instance when the TES deletes an ObjectVector, the contianer in turn deletes all objects it contains.
>
>
This is the lowest level container in Gaudi. It is designed to contain a list of pointers to data objects. Once an object has been inserted into this container the container owns the object. For instance when the TES deletes an ObjectVector, the container in turn deletes all objects it contains.
  The ObjectVector class can be used to store any data object, the only requirement is that it must inherit from the ContainedObject base class.
Line: 30 to 30
 

Consequences of the TES

Changed:
<
<
The most obvious feature of the TES is its designed is based around the use of pointers to data, rather than for instance references. This is in fact a key design choice in Gaudi itself. One consequence of this is data objects need to be created on the heap, via operator new, and ultimately are removed via operator delete. This feature then leads to a long winded discussion of the speed (or lack of) of code that make use use new and delete a lot. It is true that in early implementations of the L1/HLT tracking this was an important consideration and something needed to be done.
>
>
The most obvious feature of the TES is it's design is based around the use of pointers to data, rather than for instance references to object. This is in fact a key design choice in Gaudi itself. One consequence of this is data objects need to be created on the heap, via operator new, and ultimately are removed via operator delete. This feature then leads to a long winded discussion of the speed (or lack of) of code that make use use new and delete a lot. It is true that in early implementations of the L1/HLT tracking this was an important consideration and something needed to be done.
  This lead to the development of various schemes to either work around or try and fix the speed problems.

Fast Containers

Changed:
<
<
The first solution was to effectively abandone the use of new/delete entirely and to create some fast data containers. The PatDataStore is an example of this approach.
>
>
The first solution was to effectively abandon the use of new/delete entirely and to create some fast data containers. The PatDataStore is an example of this approach.
 
PatDataStore
Line: 47 to 47
  Pros :-
Changed:
<
<
  • The approached used by the PatDataStore is clearly very fast.
>
>
  Cons :-
Changed:
<
<
  • The Pat containers have a rather non standard interface. Firstly the user must explicitly reset the container each event, as this is not done for them by the Gaudi framework (note, this could be done using the IncidentService).
>
>
  • The Pat containers have a rather non standard interface. Firstly the user must explicitly reset the container each event, as this is not done for them by the Gaudi framework (note that the TransientFastContainer in the next section solves this problem).
 
  • The user is forced to explicitly call the setXXX for each data member of the class, when the object is being reused. The class constructors cannot be used.
  • IMHO a bad feature is that (unless I am mistaken, this is based on observation of the code and not a real test, which is to be done) the objects retain knowledge of the previous event. If the user does not call the setXXX method for some data member then the value will NOT be the default value for the class, but the value in the last event... This I believe is a bug waiting to happen.
Line: 59 to 59
  A more recent development inspired by the PatDataStore containers is the Gaudi TransientFastContainer. This is an official Gaudi container with its own Gaudi FastContainersSvc.
Changed:
<
<
Whilst the methods are differently named, this container is effectively just the PatDataStore containers. Thus, it shares most of the the pros and cons with that container. One noticable acception though is the FastContainersSvc clears each vector itself at the start of each event, so it is not neccessary for the user to do this.
>
>
Whilst the methods are differently named, this container is effectively just the PatDataStore containers. Thus, it shares most of the the pros and cons with that container. One noticeable acception though is the FastContainersSvc clears each vector itself at the start of each event, so it is not necessary for the user to do this.
 

Overload new and delete

Line: 75 to 75
 pros :-

* Solution is totally transparent to users. No changes to user code are needed to benefit and no custom data management tools to maintain.

Changed:
<
<
* Very minimal maintaince, since well prooven reliable solutions already exist in STL and boost.
>
>
* Very minimal maintenance, since well prooven reliable solutions already exist in STL and boost.
  cons :-
Line: 88 to 88
  pros :-
Changed:
<
<
  • Aparently (I have not used them myself) very fast.
>
>
  • Apparently (I have not used them myself) very fast.
 
  • Interface is very STL like.
  • Available in standard TES
  • No need to clear by-hand each event
Line: 105 to 105
 

Improvements to PatDataStore concept

Changed:
<
<
The PatDataStore provides a proven fast way of managing data objects. It does though have in my opinion problems with its user interface. The primary one being the inability to use a proper constructor and the potential leakage of data between events. However, it should be possible to improve upon this I think. One possible example is the ROOT TClonesArray class (thanks to Matt for pointing it out) which implements a simliar idea but assigns objects to partiular memory locations using the placement new method.
>
>
The PatDataStore provides a proven fast way of managing data objects. It does though have in my opinion problems with its user interface. The primary one being the inability to use a proper constructor and the potential leakage of data between events. However, it should be possible to improve upon this I think. One possible example is the ROOT TClonesArray class (thanks to Matt for pointing it out) which implements a similar idea but assigns objects to particular memory locations using the placement new method.
 

Custom allocators

Changed:
<
<
One area people tend to forget when using STL containers are the allocators, the object which define how objects are added to the container. Since containers like vector are dynamically sized, they need to be able to request more resources as they grow. The allocator that is used by default is std::alocator e.g. When a user creates a vector of type std::vector< TYPE > they are actually creating a std::vector< TYPE, std::allocator< TYPE > >. This default allocator simple uses the default ::new and ::delete methods to assign memory, so whilst the user might not realise it, they are still using new and delete.
>
>
One area people tend to forget when using STL containers are the allocators, the object which define how objects are added to the container. Since containers like vector are dynamically sized, they need to be able to request more resources as they grow. The allocator that is used by default is std::allocator e.g. When a user creates a vector of type std::vector< TYPE > they are actually creating a std::vector< TYPE, std::allocator< TYPE > >. This default allocator simple uses the default ::new and ::delete methods to assign memory, so whilst the user might not realize it, they are still using new and delete.
  Newer version of GCC contain possible alternatives to this, for instance a pool based allocator that follows a similar principle to that used in the event model overloading of new and delete. See for example here for more details.
Line: 117 to 117
  It is clear that there are many different implementations of containers that could be used, either already existing or improvements on the current containers. It is also clear that we need a fast solution, the benchmark being set by the current Pat packages. It is also my opinion (you may disagree) that the Pat containers have problems that we should avoid.
Changed:
<
<
It is also clear that a proper comparision of the speed of all methods is needed before further conclusions can really be drawn. By proper I mean using realistic data objects in realistic usage patterns. This can be done either in gaudi itself, or perhaps more conveniently in some standalone test programs. I volunteer to continue looking into this...
>
>
It is also clear that a proper comparison of the speed of all methods is needed before further conclusions can really be drawn. By proper I mean using realistic data objects in realistic usage patterns. This can be done either in gaudi itself, or perhaps more conveniently in some standalone test programs. I volunteer to continue looking into this...
  Finally, it is really a little premature to consider in too much detail the containers before we have a proper idea on the data objects themselves.
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback