Data Model

Scenario data following the IAMC format

Over the past decade, the Integrated Assessment Modeling Consortium (IAMC) developed a standardised tabular timeseries format to exchange scenario data. Previous high-level use cases include reports by the Intergovernmental Panel on Climate Change (IPCC) and model comparison exercises within the Energy Modeling Forum (EMF) hosted by Stanford University.

The table below shows a typical example of integrated-assessment scenario data following the IAMC format from the CD-LINKS project. The pyam package is geared for analysis and visualization of any scenario data provided in this structure.

_images/iamc_template.png

Illustrative example of IAMC-format timeseries data
via the IAMC 1.5°C Scenario Explorer ([1])

Refer to data.ene.iiasa.ac.at/database for more information on the IAMC format and a full list of previous use cases.

The variable column

The variable column implements a “semi-hierarchical” structure using the | character (pipe, not l or i) to indicate the depth.

Semi-hierarchical means that a hierarchy can be imposed, e.g., one can enforce that the sum of Emissions|CO2|Energy and Emissions|CO2|Other must be equal to Emissions|CO2 (if there are no other Emissions|CO2|… variables). However, this is not mandatory, e.g., the sum of Primary Energy|Coal, Primary Energy|Gas and Primary Energy|Fossil should not be equal to Primary Energy because this would double-count fossil fuels.

Refer to the variable list in the documentation pages of the IAMC 1.5°C Scenario Explorer to see the full list of variables used in the recent IPCC Special Report on Global Warming of 1.5 ºC (SR15).

The year column

In its original design, the IAMC data format (see above) assumed that the temporal dimension of any scenario data was restricted to full years represented as integer values.

Two additional use cases are currently supported by pyam in development mode (beta):

  • using representative sub-annual timesteps

  • using continuous time via pandas.datetime, replacing the name of the year column by time

Please reach out to the developers to get more information on this ongoing work.

The pyam.IamDataFrame class

A pyam.IamDataFrame instance is a wrapper for two pandas.DataFrame instances (read the docs):

  • data: The data table is a dataframe containing the timeseries data in “long format”. It has the columns of the long data format ['model', 'scenario', 'region', 'unit', 'year', 'value'].

  • meta: The meta table is a dataframe containing categorisation and descriptive indicators. It has the index pyam.META_IDX = ['model', 'scenario'].

The standard output format is the IAMC-style “wide format”, see the example above. This format can be accessed using pyam.IamDataFrame.timeseries(), which returns a pandas.DataFrame with the index pyam.IAMC_IDX = ['model', 'scenario', 'region', 'variable', 'unit'] and the years as columns.

Filtering

The pyam package provides two methods for filtering scenario data:

An existing class:IamDataFrame can be filtered using pyam.IamDataFrame.filter(col=...), where col can be any column of the data table (i.e., [‘model’, ‘scenario’, ‘region’, ‘unit’, ‘year’]) or any column of the meta table. The returned object is a new pyam.IamDataFrame instance.

A pandas.DataFrame with columns or index ['model', 'scenario'] can be filtered by any meta columns from a pyam.IamDataFrame using pyam.filter_by_meta(data, df, col=..., join_meta=False). The returned object is a pandas.DataFrame down-selected to those models-and-scenarios where the meta column satisfies the criteria given by col=... . Optionally, the meta columns are joined to the returned dataframe.

References

1

Daniel Huppmann, Elmar Kriegler, Volker Krey, Keywan Riahi, Joeri Rogelj, Katherine Calvin, Florian Humpenoeder, Alexander Popp, Steven K. Rose, John Weyant, and et al. IAMC 1.5 °C Scenario Explorer and Data hosted by IIASA (release 2.0). Integrated Assessment Modeling Consortium & International Institute for Applied Systems Analysis, 2019. URL: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer, doi:10.5281/zenodo.3363345.