Data resources integration

Connecting to an IIASA Scenario Explorer instance

IIASA’s ixmp Scenario Explorer infrastructure implements a RestAPI to directly query the database server connected to an explorer instance. See https://software.ece.iiasa.ac.at/ixmp-server for more information.

The pyam package uses this interface to read timeseries data as well as categorization and quantitative indicators. The data is returned as an IamDataFrame. See this tutorial for more information.

pyam.read_iiasa(name, default_only=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]

Query an IIASA Scenario Explorer database API and return as IamDataFrame

Parameters:
namestr
Name of an IIASA Scenario Explorer database instance.
Use valid_connections for a list of available instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional
Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).
See pyam.iiasa.Connection for details.
Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr

Authentication server URL

kwargs

Arguments for pyam.iiasa.Connection.query()

pyam.lazy_read_iiasa(file, name, default_only=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]

Try to load data from a local cache, failing that, loads it from the internet.

Check if the file in a given location is an up-to-date version of an IIASA database. If so, load it. If not, load data from the IIASA scenario explorer database API and save to that location. Does not check that the previously read version is a complete instance of the database, so if the initial load applies a filter, you will read only data that passes the same filter as well as any additional filter you apply.

Parameters:
filestr or pathlib.Path

The location to test for valid data and save the data if not up-to-date. Must be either xlsx or csv.

namestr
Name of an IIASA Scenario Explorer database instance.
Use valid_connections for a list of available instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional
Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).
See pyam.iiasa.Connection for details.
Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr

Authentication server URL

kwargs

Arguments for pyam.iiasa.Connection.query()

Reading UNFCCC inventory data

The package unfccc-di-api (read the docs) provides an interface to the UNFCCC Data Inventory API (link). The pyam package uses this package to query inventory data and return the timeseries data directly as an IamDataFrame.

pyam.read_unfccc(party_code, gases=None, tier=None, mapping=None, model='UNFCCC', scenario='Data Inventory')[source]

Read data from the UNFCCC Data Inventory

This function is a wrappter for the unfccc_di_api.UNFCCCApiReader.query().

The data returned from the UNFCCC Data Inventory is transformed into a structure similar to the format used in IPCC reports and IAM model comparison projects. For compatibility with the iam-units package and the convert_unit, emissions species are formatted to standard text (‘CO2’) instead of subscripts (‘CO₂’) and the unit ‘CO₂ equivalent’ used by UNFCCC is replaced by ‘CO2e’.

Parameters:
party_codestr

ISO3-style code for UNFCCC party (country)

gasesstr or list of str, optional

Emission species to be queried from the data inventory can be stated as subscript-format (‘CO₂’) or simple text (‘CO2’)

tierint or list of int

Pre-specified groupings of UNFCCC data to a variable naming format used in IPCC reports and IAM model comparison projects

mappingdict, optional

Mapping to cast UNFCCC-data columns into IAMC-style variables, e.g.

{
    "Emissions|{gas}|Energy": ("1.  Energy", "*", "*", "*"),
}

where the tuple corresponds to filters for the columns ["category", "classification", "measure", "gas"] and {<col>} tags in the key are replaced by the column value.

modelstr, optional

Name to be used as model identifier

scenariostr, optional

Name to be used as scenario identifier

Returns:
IamDataFrame

Notes

This method is currently not tested due to a change in the UNFCCC-DI API, which sometimes causes a JsonDecodeError. See https://github.com/pik-primap/unfccc_di_api/issues/74 for more info.

Connecting to World Bank data resources

The package pandas-datareader (read the docs) implements a number of connections to publicly accessible data resources, e.g., the World Bank Open Data Catalog. pyam provides a simple utility function to cast the queried timeseries data directly as an IamDataFrame.

pyam.read_worldbank(model='World Bank', scenario='WDI', **kwargs)[source]

Read data from the World Bank Data Catalogue and return as IamDataFrame

This function is a simple wrapper for the class pandas_datareader.wb.WorldBankReader and the function pandas_datareader.wb.download(). Import the module to retrieve/search the list of indicators (and their id’s), countries, etc.

from pandas_datareader import wb
Parameters:
modelstr, optional

The model name to be used for the returned timeseries data.

scenariostr, optional

The scenario name to be used for the returned timeseries data.

kwargs

passed to pandas_datareader.wb.download()

Returns:
IamDataFrame

Notes

The function pandas_datareader.wb.download() takes an indicator argument, which can be a string or list of strings. If the indicator passed to read_worldbank() is a dictionary of a World Bank id mapped to a string, the variables in the returned IamDataFrame will be renamed.

The function pandas_datareader.wb.download() does not return a unit, but it can be collected for some indicators using the function pandas_datareader.wb.get_indicators(). In the current implementation, unit is defined as n/a for all data; this can be enhanced later (if there is interest from users).