Purpose

In the CLI, the user cannot be sure if certain values should be integers, floats, strings, etc. E.g. j.application.max_events=100 or '100', inputdata='abc/xyz' or DQ2Dataset().

In the GUI, the algorithm that autogenerates GUI component cannot always determine the type of entry widget to create if the default is None, and there are no schema level and GUIConfig level cues.

Typically, type errors are discovered only when the job runs (and fails). Type errors need to be flagged out at the job definition (i.e. attribute assignment) stage.

Requirements

• Must be able to specify a set of legal types for a single attribute.
• Type testing must be done on assignment. Error/exception should provide information on expected type(s) for the attribute concerned.

Changes to schema

Type information added by means of the typelist schema attribute. This attribute takes a list of strings that represent types. Entries in this list can be string representations of native Python types (e.g. 'int', 'bool', 'str') or GANGA classes (e.g. 'Ganga.GPIDev.Lib.File.File.File', 'Ganga.GPIDev.Lib.Dataset.Dataset').

The current implementation of the type system takes into account the value of sequence. If the attribute is a sequence, individual entries in the list are subjected to type testing against typelist. A non- sequence attribute will be subjected to type testing against typelist directly.

For illustration, here is an example of the Executable plugin schema with type definitions:

'exe' : SimpleItem( defvalue='echo', typelist=['str','Ganga.GPIDev.Lib.File.File.File'] ),
'args' : SimpleItem( defvalue=["Hello World"], typelist=['str'], sequence=1 ),
'env' : SimpleItem( defvalue={}, typelist=['str'] )

NB: The type system works the same way for ComponentItem. The only difference is that typelist entries for ComponentItem should not have string representations of native Python types. This is, however, not enforced in the current implementation.

Type system in action

Without type system: (almost anything goes!)

In [2]:j.application.exe = 1
In [3]:j.application.exe = 'xyz'
In [4]:j.application.exe = File('xyz')
In [5]:j.application.args = 1
---------------------------------------------------------------------------
exceptions.TypeError                                 Traceback (most recent call last)

? 

.../python/Ganga/GPIDev/Base/Proxy.py in _setattr(self, x, v)
    265             raise AttributeError("Schema of '%s' has no property '%s'" % (self._impl._name,x))
    266 
--> 267         object.__setattr__(self,x,v)
    268     helptext(_setattr,"""Set a property of %(classname)s with consistency and safety checks.
    269 Setting a [protected] or a unexisting property raises AttributeError.""")

.../python/Ganga/GPIDev/Base/Proxy.py in __set__(self, obj, val)
    133 
    134         if item['sequence']:
--> 135             val = map(stripper,val)
    136         else:
    137             val = stripper(val)

TypeError: argument 2 to map() must support iteration

In [6]:j.application.args = [ 1, 2 ]
In [7]:j.application.args = [ 1, '2' ]
In [8]:j.application.args = [ '1', '2' ]
In [9]:j.application.env = 1
In [10]:j.application.env = []
In [11]:j.application.env = { 1:2 }
In [12]:j.application.env = { 1:'2' }
In [13]:j.application.env = { '1':'2' }

With type system enabled:

In [2]:j.application.exe = 1
Ganga.GPIDev.Base.Proxy            : ERROR    Invalid Ganga type: Ganga.GPIDev.Lib.File.File.File
TypeMismatchError : Attribute "exe" expects a value of the following types: ['str', 'Ganga.GPIDev.Lib.File.File.File']

In [3]:j.application.exe = 'xyz'

In [4]:j.application.exe = File('xyz')

In [5]:j.application.args = 1
TypeMismatchError : Attribute "args" expects a list.

In [6]:j.application.args = [ 1, 2 ]
TypeMismatchError : List entry "1" for args is invalid. Valid types: ['str']

In [7]:j.application.args = [ 1, '2' ]
TypeMismatchError : List entry 1 for args is invalid. Valid types: ['str']

In [8]:j.application.args = [ '1', '2' ]

In [9]:j.application.env = 1
TypeMismatchError : Attribute "env" expects a dictionary.

In [10]:j.application.env = []
TypeMismatchError : Attribute "env" expects a dictionary.

In [11]:j.application.env = { 1:2 }
TypeMismatchError : Dictionary entry 1:2 for attribute env is invalid. Valid types for key/value pairs: ['str']

In [12]:j.application.env = { 1:'2' }
TypeMismatchError : Dictionary entry 1:2 for attribute env is invalid. Valid types for key/value pairs: ['str']

In [13]:j.application.env = { '1':'2' }

Additional considerations

• Should attributes defined as a sequence be allowed to accept a list as an entry i.e. are list of lists legal? Kuba: yes, a list of lists is used for example in ArgsSplitter.values, how do we deal with this? I guess the only generic way is to attach a special callback hook which would do the check? In this particular case the list should have only 2 levels and the leafs should be strings
• Although possible, types defined in typelist should not be experiment-specific. (E.g. DQ2Dataset, LHCbDataset). Base classes should be used instead.
• The type system can be used to eliminate the situation where output dataset objects are assigned as input datasets (and vice versa). However, for this to happen, there needs to be a new set of subclasses for Dataset (e.g. DatasetIn and DatasetOut). By doing so, DQ2Dataset and LHCbDataset can be subclassed from DatasetIn and not Dataset (for example). The type system will then be able to enforce type-checking correctly.

Kuba: by setting typelist to None one can disable the type checking. This may be used in conjunction with checkset.