Resource Alignment Service (RAS)
************************************
===============================
Temporal Alignment Service
===============================
**Introduction**
----------------
For time-series variables, they contain both the temporal information as well as numerical values at each time step, \
namely data information. Temporal alignment service (TAS) aims to provide the temporal interpolation on the time-series \
variables with missing data information. The alignment is conducted between the variables in the source \
(with temporal and data information) and variables in the target (with only temporal information).
Time-series variables are saved in the `NetCDF `_ format. \
For each NetCDF file, it consists of three optional parts: dimensions, variables and data. \
In order to take full advantage of TAS, for both source and target files, each of them must only contain a single \
time-series variable. Within the dimensions part, the information of grid (nx and ny) and time steps (timestep) \
should be included as shown below::
dimensions:
nx = 20 ;
ny = 20 ;
time = UNLIMITED ; // (9 currently)
timesteps = 9 ;
Within the variables part, the temporal information should be included, such as, the name (long_name) and unit (unit)
of the time-series variable, the information of the time sequence including the time step (timestep) and
start time (starttime) as shown below::
variables:
double time(time) ;
time:units = "hours" ;
time:_FillValue = 9.9692e+36 ;
float Q(time, ny, nx) ;
Q:long_name = "volumetric_discharge" ;
Q:units = "m^3/h" ;
Q:dx = 24.377 ;
Q:dy = 32.855 ;
Q:y_south_edge = 4459392.558 ;
Q:y_north_edge = 4472501.657 ;
Q:x_west_edge = 377239.937 ;
Q:x_east_edge = 391524.633 ;
Q:_FillValue = 9.969e+36f ;
float time_seq(timesteps) ;
time_seq:timestep = "1h" ;
time_seq:starttime = "2013-01-01 11:00:00" ;
The time-series variable, Q is defined with three named dimensions, time, ny and nx which are specified in the dimensions \
part as shown above. For the time variable, it has one named dimension, time. Different from other named dimensions, \
time is usually defined as “UNLIMITED”. As a result, the size of time variable changes according to the size of the \
temporal data assigned to the time-series variable. The time unit needs to be defined for the time variable. \
There are six options to define a time unit: seconds, minutes, hours, days, months, years. In addition, \
the unit of the time-series variable also needs to be specified as the attribute of the time-series variable, \
e.g., the unit of Q is m3/h. In general, values of time-series variables are saved in 1D, 2D, 3D or 4D array and \
the first dimension of the array denotes the sequence number of the time steps.
**Requirements**
----------------
* Python 2.6 or later (python 3 is not supported)
* `Python NetCDF `_ :read/write NetCDF file using python.
* `OpenCog `_ :develop AI systems, especially appropriate for integrative multi-algorithm systems, and artificial general intelligence systems. The `spatiotemporal `_ relationship is used to evaluate the temporal relationship of two time-series events.
**API**
-------
---------------------------------
.. py:class:: TemporalAlignment()
---------------------------------
.. py:function:: ncdump(*args, **kwargs)
Get the global attributes, list of name dimensions and variables from the specified NetCDF file.
:param nc_fid: file ID of a NetCDF file
:return: global attributes, named dimensions and variable names
:rtype: List
.. py:function:: extractTemporalInfo(*args, **kwargs)
Extract the temporal and data value information from a NetCDF file.
:param nc_fid: file ID of a NetCDF file
:param var: a time-series variable
:return: unit, dimension, time sequence, time step and values of the time-series variables in the source and destination files
:rtype: Dictionary and keys are {‘units’, ‘dimension’, 'timesequence', 'timestep', 'vardata'}
.. py:function:: temporalRelations(*args, **kwargs)
Extract the temporal and data value information from a NetCDF file.
:param source: dictionary-like variable with key = {‘units’, ‘dimension’, 'timesequence', 'timestep', 'vardata'}. The elements of the key are self explanatory. ‘units’ means the unit of the variable defined as attribute in the time variable in the NetCDF file; ‘dimension’ describes the shape of the variable, or the number of named dimensions in the NetCDF file; ‘timesequence’ means the time sequence of the variable defined as the time_seq variable in the NetCDF file; ‘timestep’ means time step defined as an attribute in the time_seq variable NetCDF file; ‘vardata’ stores the values of the time-series variable from the data part in NetCDF file.
:param dest: similar to source, the only difference is the vardata is NA in dest.
:return: temporal relationship of time-series events, one of the elements in the set {‘p’, ‘m’, ‘o’, ‘F’, ‘D’, ‘s’, ‘e’, ‘S’, ‘d’, ‘f’, ‘O’, ‘M’, ‘P’}
:rtype: Char
TEMPORAL_RELATIONS = {
'p': 'precedes',
'm': 'meets',
'o': 'overlaps',
'F': 'finished by',
'D': 'contains',
's': 'starts',
'e': 'equals',
'S': 'started by',
'd': 'during',
'f': 'finishes',
'O': 'overlapped by',
'M': 'met by',
'P': 'preceded by'
}
.. py:function:: ifOverlapped(*args, **kwargs)
Check if two time-series events are overlapped in the way that the temporal interpolation can be carried out. According to Allen’s time algebra, thirteen temporal relationships are defined. Temporal interpolation proceeds only when the temporal relationship meets either of them: ‘F’: finished by; ‘D’: contains; ‘e’: equals; ‘S’: started by and ‘f’: finishes.
:param rel: Allen’s time algebra, one of the elements in the set {‘p’, ‘m’, ‘o’, ‘F’, ‘D’, ‘s’, ‘e’, ‘S’, ‘d’, ‘f’, ‘O’, ‘M’, ‘P’}
:return: True or False
:rtype: Boolean
.. py:function:: interpolationSettings(*args, **kwargs)
Define the interpolation method and data processing procedure applied to the time-series variable in the destination file.
:param dest: as defined in :func:`temporalRelations`
:param vartype: the type of information requested for the time-series variable in the destination file. There are three options: ‘instantaneous’: calculate the instantaneous value at each time step; ‘mean’: calculate the mean value over the time step; ‘cumulative’: calculate the aggregated value over the time step.
:param method: three interpolation methods are provided, ‘linear’, ‘polynomial’ and ‘spline’
:param order: the order of the interpolation function. When the ‘polynomial’ or ‘spline’ interpolation method is used, the order of the interpolation function should be specified. For example, to use cubic spline interpolation, set method = ‘spline’ and order = 3.
:return: A dictionary variable with the information of the dimension of the variable, variable type, interpolation method and the order of the interpolation function stored in a key-value as {'dim': dim, 'vartype': vartype, 'method': method, 'order': order}
:rtype: Dictionary
.. py:function:: selInterpolation(*args, **kwargs)
Dynamically select the method defined in the :class:`TemporalAlignment()` according to the dimension of the time-series variable. Within the method, the temporal interpolation is conducted. Parallel implementation of the chosen method is provided.
:param dimensions: the dimension of the time-series variable in the source or destination file
:param multi: either True (parallel implementation) or False (sequential implementation)
:return: method name
:rtype: String
.. py:function:: temporalInterpolation_xx(*args, **kwargs)
Conduct temporal interpolation on the time-series variable in the destination file. xx denotes the dimension of the time-series variable. For example, if the values of the data is stored in a 2D array, xx will be replaced by 2D, i.e., :func:`temporalInterpolation_2D`
:param source: as defined in :func:`temporalRelations`
:param dest: as defined in :func:`temporalRelations`
:param vartype: as defined in :func:`interpolationSettings`
:param method: as defined in :func:`interpolationSettings`
:param order: as defined in :func:`interpolationSettings`
:return: data values for the time-series variable in the destination file
:rtype: Float array
.. py:function:: multi_temporalInterpolation_xx(*args, **kwargs)
The parallel implementation of the :func:`temporalInterpolation_xx` method.
:param: same as :func:`temporalInterpolation_xx`
:return: same as :func:`temporalInterpolation_xx`
:rtype: Json
===============================
Spatial Alignment Service
===============================
**Introduction**
----------------
For spatial variables, they contain both the spatial information as well as numerical values for the domain, \
namely data information. Spatial alignment service (SPAS) aims to provide the spatial interpolation on the spatial \
variables with missing data information. The alignment is conducted between the variables in the source \
(with spatial and data information) and variables in the target (with only spatial information).
Spatial variables are saved in the `NetCDF `_ format. \
For each NetCDF file, it consists of three optional parts: dimensions, variables and data. \
In order to take full advantage of SPAS, for both source and target files, each of them must only contain a single \
spatial variable. Within the dimensions part, the number of grids (nx and ny) should be specified as shown below::
dimensions:
nx = 40 ;
ny = 59 ;
time = UNLIMITED ; // (1 currently)
Note that time is defined as “UNLIMITED” and usually equals to one for the spatial variables.
Within the variables part, the spatial information should be included, such as the bounding box and grid size as shown below::
variables:
float csn:land_surface__elevation(time, nx, ny) ;
csn:land_surface__elevation:grid_size = "0.00290955516225x0.00290955516225" ;
csn:land_surface__elevation:llcorner = "40.3950764765, -88.4464747571" ;
csn:land_surface__elevation:urcorner = "40.510458683, -88.2748110025" ;
csn:land_surface__elevation:units = "m" ;
csn:land_surface__elevation:type = "spatial" ;
int time_seq(timesteps) ;
time_seq:timestep = 1L ;
time_seq:endtime = "2016-06-12T01:00:00Z" ;
time_seq:starttime = "1950-06-12T01:00:00Z" ;
double time(time) ;
The spatial variable, `csn:land_surface__elevation` is defined with three named dimensions, time, nx and ny which are \
specified in the dimensions part as shown above. The geometry and coordinates of the variable must be represented by the Grid
or Mesh Classes supported by Earth System Modeling Framework (`ESMF `_)
in another NetCDF file. For example, the SCRIP format grid is used to represent the geometry and coordinates of the variable,
`csn:land_surface__elevation`::
netcdf csnland_surface__elevation_SCRIP_source {
dimensions:
grid_size = 2360 ;
grid_corners = 4 ;
grid_rank = 2 ;
variables:
int grid_dims(grid_rank) ;
double grid_center_lat(grid_size) ;
grid_center_lat:units = "degrees" ;
double grid_center_lon(grid_size) ;
grid_center_lon:units = "degrees" ;
int grid_imask(grid_size) ;
grid_imask:units = "unitless" ;
double grid_corner_lat(grid_size, grid_corners) ;
grid_corner_lat:units = "degrees" ;
double grid_corner_lon(grid_size, grid_corners) ;
grid_corner_lon:units = "degrees" ;
// global attributes:
:date_created = "Mon Jun 27 13:22:16 CDT 2016" ;
:Createdby = "ESMF_regridding.ncl" ;
:Conventions = "SCRIP" ;
:title = "0.00290955516225x0.00290955516225 grid" ;
data:
grid_dims = 59, 40 ;
grid_center_lat = 40.395076751709 ...
**Requirements**
----------------
* Python 2.6 or later (python 3 is not supported)
* `Python NetCDF `_ :read/write NetCDF file using python.
* `conda 4.0.5 `_
* `ncl 6.3.0 `_
* `ESMPy 7.0.0 `_: install ESMPy with conda
* `gdal 1.11.2`: install gdal with conda
**API**
-------
---------------------------------
.. py:class:: SpatialAlignment()
---------------------------------
.. py:function:: esmfgrid2grid(*args, **kwargs)
Conduct spatial interpolation on the spatial variables in the destination file.
:param source: source file in NetCDF
:param sgrid: SCRIP format representation of the geometry and coordinates of the spatial variable in the source file.
:param dest: destination file in NetCDF
:param dgird: SCRIP format representation of the geometry and coordinates of the spatial variable in the destination file.
:rtype: Json