Data structures

class datatoolbox.Datatable(*args, **kwargs)

Datatable is derrived from pandas dataframe. Datatables contain the addition meta attribute and have autotmated unit conversions

add(other, **kwargs)

Method to add to datatable making use of pd.Dataframe.add but addtionally consider update of the meta data.

Parameters

otherTYPE

DESCRIPTION.

**kwargsTYPE

DESCRIPTION.

Returns

tableTYPE

DESCRIPTION.

aggregate_region(mapping, skipna=False)

This functions added the aggregates to the table according to the provided mapping.( See datatools.mapp.regions)

Returns the result, but does not inplace add it.

append(other, **kwargs)

Append data to the datatable

Parameters

otherdatatable

New data that will be added to the datatable.

kwargsTYPE

Default pandas append arguments.

Returns

datatable

clean()

Clean up the dataframe to only recogniszed regions, years and numeric values. Removed columns and rows with only nan values.

Returns

datatable

DESCRIPTION.

columns_to_datetime()

Conversion method that does convert the colume values to a pandas DatetimeIndex.

Returns

None.

convert(newUnit, context=None, suffix_dict={}, **new_meta)

Convert datatable to different unit and returns converted datatable.

Parameters

newUnitstr

New unit string in which the datatable should be converted.

contextstr, optional

Optional context (e.g. GWPAR4). The default is None.

Returns

datatable

Datatable converted in the new unit.

copy(deep=True)

Make a copy of this Datatable object Parameters

Returns

copy : Datatable

diff(periods=1, axis=0)

Compute the difference between different years in the datatable Equivalent do pandas diff but return datatable.

Parameters

periodsint, optional

DESCRIPTION. The default is 1.

axisint, optional

DESCRIPTION. The default is 0.

Returns

outTYPE

DESCRIPTION.

classmethod from_excel(filepath, sheetName=None)

Create a dataframe from a suitable excel file that is saved by datatoolbox.

Parameters

cls : class

filepathstr

Path to the file.

sheetNamestr, optional

Sheetn ame that is read in. The default is None.

Returns

datatable

DESCRIPTION.

classmethod from_multi_indexed_dataframe(df)

Class function to create a datatable from a multi-indexed dataframe

Parameters

df : multi index dataframe

Returns

table : Datatable

classmethod from_pyam(idf, **kwargs)

Create a datatable from an iam dataframe.

Parameters

clsdatatable class

DESCRIPTION.

idfpyam dataframe

dataframe that contrains the data that is used to create the datatable.

kwargsTYPE

DESCRIPTION.

Returns

datatatbledatatoolbox datatable

Datatable with original unit and related meta data.

generateTableID()

Generates the table ID based on the meta data of the table.

Returns

datatable

DESCRIPTION.

getTableFileName()

For compatibility to windows based sytems, the pipe symbols is replaces by double underscore for the csv filename.

getTableFilePath()

Returns path to data on hard disk

Returns

TYPE

DESCRIPTION.

info()

Returns information about the dataframe like shape, index and column extend and the number of non-nan entries.

Returns

str

Information about datatable.

interpolate(method='linear', add_missing_years=False)

Interpoltate missing data between year with the option to add missing years in the columns.

Parameters

methodsting, optional

Interpolation method. The default is “linear”. - linear

add_missing_yearsbool, optional

If true, missing years within the time value range are added to the dataframe. The default is False.

Returns

datatable

Interpolated dataframe.

reduce(method='linear_piece_wise', eps=1e-06)

Reduce data that is piecewise linear to the core data points (kinks).

source()

Return the source of the table

squeeze_index_to_attrs()

Does move all unique index levels to attrs.

Returns

Datatable

Datatable with only index levels that are non-unique. All other levels are in attrs. This operation can be reversed with table.to_multi_index_dataframe()

to_IamDataFrame(**kwargs)

Function to sustain backwars compatibility Depreciated.

to_csv(fileName=None)

Save the datatable to an annotated csv file.

Parameters

fileNamestr, optional

Path to file. The default is None.

Returns

None.

to_excel(fileName=None, sheetName='Sheet0', writer=None, append=False)

Save datatable to excel.

Parameters

fileNamestr, optional

Relative file path. If None is provide, a writer is expected. The default is None.

sheetNamestr, optional

Sheet name that is read in. The default is “Sheet0”.

writerpandas excel writer, optional

Pandas writer that is used instead opening a new one. The default is None.

appendbool, optional

If true, data is appended to the writer. The default is False.

Returns

None.

to_multi_index_dataframe(meta_keys=None, exclude_meta=['ID', 'creator', 'source_name', 'source_year', '_timeformat'])

Return a new datatable with a mult-index that has all meta data assigned

Parameters

exclude_meta: optinal list of meta keys that should be ignored

Returns

Datatable

New Datatable with multi_index and not meta data.

to_pyam(**kwargs)

Conversion to pyam dataframe.

Parameters

kwargsTYPE

DESCRIPTION.

Raises

AssertionError

DESCRIPTION.

Returns

idfpyam dataframe

DESCRIPTION.

yearlyChange(forward=True)

Noindex:

This methods returns the yearly change for all years (t1) that reported and and where the previous year (t0) is also reported

Parameters

forwardbool

If true, the yearly change is computed in the forward direction, otherwise backwards. Default is forward.

class datatoolbox.DataSet(data_vars: DataVars | None = None, coords: Mapping[Any, Any] | None = None, attrs: Mapping[Any, Any] | None = None)

Very simple class to allow initialization of xarray datasets from pyam, wide pandas dataframes and datatoolbox queries.