Data resources integration

Connecting to an IIASA database instance

IIASA’s ixmp Scenario Explorer infrastructure implements a RestAPI to directly query the database server connected to an explorer instance. See https://docs.ece.iiasa.ac.at/ for more information.

The pyam package uses this interface to read timeseries data as well as categorization and quantitative meta indicators. The data is returned as an IamDataFrame. See this tutorial for more information.

pyam.read_iiasa(name, default_only=True, meta=True, creds=None, **kwargs)[source]

Read data from an ixmp4 platform or an IIASA Scenario Explorer database.

Parameters:
namestr
Name of an ixmp4 platform or an IIASA Scenario Explorer database instance.
Use platforms for a list of ixmp4 platforms hosted by IIASA.
Use valid_connections for a list of available Scenario Explorer database instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional

Path to a file with authentication credentials. This feature is deprecated, please run ixmp4 login <username> in a console instead.

**kwargs

Arguments for pyam.read_ixmp4() or pyam.iiasa.Connection.query().

Notes

Credentials (username & password) are not required to access any public ixmp4 or Scenario Explorer database (i.e., with Guest login).

pyam.lazy_read_iiasa(file, name, default_only=True, meta=True, creds=None, **kwargs)[source]

Try to load data from a local cache, failing that, loads it from an IIASA database.

Check if the file in a given location is an up-to-date version of an IIASA database. If so, load it. If not, load data from the IIASA scenario explorer database API and save to that location. Does not check that the previously read version is a complete instance of the database, so if the initial load applies a filter, you will read only data that passes the same filter as well as any additional filter you apply.

Parameters:
filestr or pathlib.Path

The location to test for valid data and save the data if not up-to-date. Must be either xlsx or csv.

namestr
Name of an IIASA Scenario Explorer database instance.
Use valid_connections for a list of available instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional

Path to a file with authentication credentials. This feature is deprecated, please run ixmp4 login <username> in a console instead.

**kwargs

Arguments for pyam.read_ixmp4() or pyam.iiasa.Connection.query().

Notes

This feature does currently not support reading data from ixmp4 platforms.

Credentials (username & password) are not required to access any public ixmp4 or Scenario Explorer database (i.e., with Guest login).

Reading from an ixmp4 platform

The pyam package provides a simple interface to read timeseries data and meta indicators from local or remote ixmp4 platform instances.

pyam.read_ixmp4(platform: Platform | str, default_only: bool = True, model: str | list[str] | None = None, scenario: str | list[str] | None = None, region: str | list[str] | None = None, variable: str | list[str] | None = None, unit: str | list[str] | None = None, year: int | list[int] | None = None)[source]

Read scenario runs from an ixmp4 platform database instance

Parameters:
platformixmp4.Platform or str

The ixmp4 platform database instance to which the scenario data is saved.

default_onlybool, optional

Read only default runs.

model, scenario, region, variable, unitstr or list of str, optional

Filter by these dimensions.

yearint or list of int, optional

Filter by time domain.

Reading UNFCCC inventory data

The package unfccc-di-api (read the docs) provides an interface to the UNFCCC Data Inventory API (link). The pyam package uses this package to query inventory data and return the timeseries data directly as an IamDataFrame.

pyam.read_unfccc(party_code, gases=None, tier=None, mapping=None, model='UNFCCC', scenario='Data Inventory')[source]

Read data from the UNFCCC Data Inventory

This function is a wrappter for the unfccc_di_api.UNFCCCApiReader.query().

The data returned from the UNFCCC Data Inventory is transformed into a structure similar to the format used in IPCC reports and IAM model comparison projects. For compatibility with the iam-units package and the convert_unit, emissions species are formatted to standard text (‘CO2’) instead of subscripts (‘CO₂’) and the unit ‘CO₂ equivalent’ used by UNFCCC is replaced by ‘CO2e’.

Parameters:
party_codestr

ISO3-style code for UNFCCC party (country)

gasesstr or list of str, optional

Emission species to be queried from the data inventory can be stated as subscript-format (‘CO₂’) or simple text (‘CO2’)

tierint or list of int

Pre-specified groupings of UNFCCC data to a variable naming format used in IPCC reports and IAM model comparison projects

mappingdict, optional

Mapping to cast UNFCCC-data columns into IAMC-style variables, e.g.

{
    "Emissions|{gas}|Energy": ("1.  Energy", "*", "*", "*"),
}

where the tuple corresponds to filters for the columns ["category", "classification", "measure", "gas"] and {<col>} tags in the key are replaced by the column value.

modelstr, optional

Name to be used as model identifier

scenariostr, optional

Name to be used as scenario identifier

Returns:
IamDataFrame

Notes

This method is currently not tested due to a change in the UNFCCC-DI API, which sometimes causes a JsonDecodeError. See https://github.com/pik-primap/unfccc_di_api/issues/74 for more info.

Connecting to World Bank data resources

The package pandas-datareader (read the docs) implements a number of connections to publicly accessible data resources, e.g., the World Bank Open Data Catalog. pyam provides a simple utility function to cast the queried timeseries data directly as an IamDataFrame.

pyam.read_worldbank(model='World Bank', scenario='WDI', **kwargs) → IamDataFrame[source]

Read data from the World Bank Data Catalogue and return as IamDataFrame

This function is a simple wrapper for the function wbdata.get_dataframe(). Import the module to retrieve/search the list of indicators (and their id’s), countries, etc.

import wbdata as wb
Parameters:
modelstr, optional

The model name to be used for the returned timeseries data.

scenariostr, optional

The scenario name to be used for the returned timeseries data.

**kwargs

passed to wbdata.get_dataframe()

Returns:
IamDataFrame

Notes

The function wbdata.get_dataframe() takes an indicators argument, which is a dictionary where the keys are desired indicators and the values are the desired column names. If the indicators passed to read_worldbank() is a single indicator code string, we should instead use wbdata.get_series().

The function wbdata.get_dataframe() does not return a unit, but it can be collected for some indicators using the function wbdata.get_indicators(). In the current implementation, unit is defined as n/a for all data; this can be enhanced later (if there is interest from users).