Special Variable Documentation

The eoldas_SpecialVariable Module

class eoldaslib.eoldas_SpecialVariable.DemoClass(info=[], thisname=None, readers=[], datadir=['.'], env=None, header=None, logger=None, log_terms={}, simple=False)

Bases: eoldaslib.eoldas_ParamStorage.ParamStorage

A demonstration class using SpecialVariable

The behaviour we desire is that a SpecialVariable acts like a ParamStorage (i.e. we can get or set by attribute or item

e.g.

x.state = 3 and x[‘state’] = 3

give the same result, and

print x[‘state’] and print x.state

give the same result. This is easy enough to achieve for all cases other than getting from x.state. It turns out that __getattr__ does not override the default method for state if state is set in the class instance.

To get around that, we have to use a fake name (fakes here) and instead of storing state, we store _state. This makes daling with with all of the conditions a little more complicated and a little slower, but it allows a much more consistent interface.

At any time, a SpecialVariable can simply be over-written by using assigning to its fake name.

e.g.

instance the class

x = demonstration()

set a non-special value ‘cheese’

x.foo = ‘bar’

we can use this as x.foo or x[‘foo’]

print x.foo,x[‘foo’]

which should give bar bar

now use the SpecialVariable. There are many way to load this up, but an easy one is via a dictionary.

data = {‘state’:np.ones(2)*5. ,’foo’:np.ones(10)} name = {‘state’:’of the nation’,’foo’:’bar’} this = {‘data’:data,’name’:name} x.state = this

print x.state,x[‘state’]

which gives [ 5. 5.] [ 5. 5.], so we get the same from either approach. Note that what is returned from the SpecialVariable is only what is in this[‘data’][‘state’], and that is fully the intention of the SpecialVariable class. It can be loaded with rich information from a range of sources, but if you want a quick interpretation of the data (i.e. x.state) you only get what is in x.state, or more fully,

x._state.data.state

The other data that we passed to the SpecialVariable is as it was when read in, but relative to x._state, i.e. we have:

x._state.name.foo

which is bar.

If you want to directly access the SpecialVariable, you can use:

x.get(x.fakes[‘state’])

which is the same as

x._state or x[x.fakes[‘state’]]

It is not adviseable to directly use the underscore access as the fakes lookup dictionary can be changed. It is best to always use x.fakes[‘state’]. Indeed, if you want to override the ‘special’ nature of a term such as ‘state’, you can simply remove their entry from the table:

old_dict = x.fakes.copy() del x.fakes[‘state’]

Now, if you type:

print x.state

You get a KeyError for state, so it would have been better to:

x.fakes = old_dict.copy() del x.fakes[‘state’] x[‘state’] = x[old_dict[‘state’]] print x.state

which should give [ 5. 5.], but the type of x.state will have changed from SpecialVariable to np.ndarray.

If you want to convert the SpecialVariable back to a dictionary you can do:

print x[x.fakes[‘state’]].to_dict()

or a little less verbosely:

print x._state.to_dict()

Methods

get(this)

An alternative interface to get the value of a class member that by-passes any more complex mechanisms. This returns the ‘true’ value of a class member, as opposed to an interpreted value.

set(this, that)

An alternative interface to set the value of a class member that by-passes any more complex mechanisms. This sets the ‘true’ value of a class member, as opposed to an interpreted value.

var(this)

Return the data associated with SpecialVariable this, rather than an interpretation of it

class eoldaslib.eoldas_SpecialVariable.SpecialVariable(info=[], name=None, thisname=None, readers=[], log_terms={}, datadir=['.'], env=None, header=None, writers={}, simple=False, logger=None)

Bases: eoldaslib.eoldas_ParamStorage.ParamStorage

A class that can deal with the datatypes needed for eoldas

It allows a variable to be set to various data types and interprets these into the data structure.

The data structure imposed here is:

self.data.this self.name.this

to store some item called ‘this’. Other information can be stored as well but part of the idea here is to have some imposed constraints on the data structure so that we can sensibly load up odfferent datasets.

The idea is that data will be stored in self.data and associated metadata in self.name. If items are given the same name in both sub-structures, we can easily keep track of them. There is no actual requirement that this is adhered to, but it is certainy permitted and encouraged for the indended use of this class.

Probably the most important thing then about this class is that is a SpecialVariable is assigned different data types, then it can do sensible things with them in the context of the EOLDAS (and wider applications).

When an assignment takes place (either of ther form

self.state = foo

or

self[‘state’] = foo

then what is actually stored depends on the type and nature of foo. The main features as follows:

If foo is a string:

A guess is made that it is a filename, and an attempt is made to read the file. All directories in the list self.dirnames are searched for the filename, and any readable files found are considered candidates, Each of these is read in turn. A set of potential data formats, specified by the readers in readers (self.reader_functions) is considered, as if a sucessful interpretation takes place the data is returned and stiored in the derired variable.

So, for example, if we have self.state = foo as above and foo is a valid, readable file in the list of directories specified, and it is interprable with one of the formats defined, then the main dataset is loaded into:

self.data.state

(alternatively known as self.data[‘state’]).

If foo is a ParamStorage, it should have the same structure as that here

(i.e. self.data and self.name) and these structures are then loaded.

If foo is a dictionary (type dict)

It is first converted to a ParamStorage and then loaded as above.

If foo is any other datatype, it is left pretty much as it, except that

an attempt to convert to a np.array is made.

Depending on the format, there might be data other than the main dataset (e.g. locational information) and these are loaded by the loaders into relevant oarts of self.data and self.name.

For classes that use this class for EOLDAS, we will typically use:

self.data.state : state variable data self.data.sd : uncertainty information as sd

(or a similar fuller representation)

self.data.control : control information (e.g. view angles) self.data.location : location information

with associated descriptor data in the relevant parts of self.name.

The idea for simple use of the data structure then is for all of these datasets represented as 2D datasets, where the number of rows in each of the self.data.state etc field will be the same, but the number of columns will tend to vary (e.g different numbers of state variables).

The reason for considering such a 2D ‘flat’(ish) representation is that it is easy to tabulate and understand. In fact the data will be of quite high dimension. E.g. is the data vary by time, x and y, then we would have 3 columns for self.data.location, with descriptors for the columns in self.name.location, and corresponding state data in self.data.state (with the number of state variables determining the number of columns in that table).

As mentioned, at the pooint of this class, there is no strict requirement for any such structure ti the data loaded or used, but that is the plan for EOLDAS use, so worth documenting at this point.

Methods

get(this)

A method to get the literal value of this, rather than attempt an interpretation (e.g. used when self.simple is True)

init(info=[], name=None, readers=[], log_terms={}, datadir=None, env=None, header=None, writers={}, simple=False, logger=None)

Initialise information in a SpecialVariable instance.

Options:

info : Information tthat can be passed through to reader

methods (a list).

thisname : a name to use to identify this instance in any

logging. By default this is None. If thisname is set to True, then logging is to stdout.

readers : A list of reader methods that are pre-pended to

those already contained in the class.

log_terms : A dictionary of log options. By default

{‘logfile’:None,’logdir’:’log’,’debug’:True} If thisname is set, and logfile specified, then logs are logged to that file. If thisname is set to True, then logging is to stdout.

datadir : A list of directories to search for data files

to interpret if the SpecialVariable is set to a string.

env : An environment variable that can be used to extend

the datadir variable.

header : A header string to use to identify pickle files.

By default, this is set to “EOLDAS – plewis – UCL – V0.1”

simple : A flag to swicth off the ‘complicated’

interpretation methods, i.e. just set and return variables literally, do not try to interpret them.

process_data_string(name, info=[], fmt=None)

Attempt to load data from a string, assuming the string is a filename.

The array self.datadir is searched for readable files with the string ‘name’ (also self.env), and a list of potential files considered for reading.

Each readable file is passed to self.read, and if it is interpretable, it is loaded according to the read method.

Note tha the format can be specified. If not, then all formats are attempted until a sucessful read is made.

read(filename, fmt)

set(this, value)

A method to set the literal value of this, rather than attempt an interpretation (e.g. used when self.simple is True)

write(filename, fmt)

eoldaslib.eoldas_SpecialVariable.demonstration()