Data resources integration¶

Connecting to an IIASA Scenario Explorer instance¶

IIASA’s ixmp Scenario Explorer infrastructure implements a RestAPI to directly query the database server connected to an explorer instance. See https://software.ece.iiasa.ac.at/ixmp-server for more information.

The pyam package uses this interface to read timeseries data as well as categorization and quantitative indicators. The data is returned as an IamDataFrame. See this tutorial for more information.

pyam.read_iiasa(name, default=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]¶

Query an IIASA Scenario Explorer database API and return as IamDataFrame

Parameters:

namestr: Name of an IIASA Scenario Explorer database instance.

See pyam.iiasa.Connection.valid_connections.
defaultbool, optional: Return only the default version of each scenario. Any (model, scenario) without a default version is omitted. If False, return all versions.
metabool or list of strings, optional: If True, include all meta categories & quantitative indicators (or subset if list is given).
credsstr or pathlib.Path, optional: Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).

See pyam.iiasa.Connection for details.

Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr: Authentication server URL
kwargs: Arguments for pyam.iiasa.Connection.query()

pyam.lazy_read_iiasa(file, name, default=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]¶

Try to load data from a local cache, failing that, loads it from the internet.

Check if the file in a given location is an up-to-date version of an IIASA database. If so, load it. If not, load data from the IIASA scenario explorer database API and save to that location. Does not check that the previously read version is a complete instance of the database, so if the initial load applies a filter, you will read only data that passes the same filter as well as any additional filter you apply.

Parameters:

filestr or pathlib.Path: The location to test for valid data and save the data if not up-to-date. Must be either xls, xlsx or csv.
namestr: Name of an IIASA Scenario Explorer database instance.

See pyam.iiasa.Connection.valid_connections.
defaultbool, optional: Return only the default version of each scenario. Any (model, scenario) without a default version is omitted. If False, return all versions.
metabool or list of strings, optional: If True, include all meta categories & quantitative indicators (or subset if list is given).
credsstr or pathlib.Path, optional: Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).

See pyam.iiasa.Connection for details.

Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr: Authentication server URL
kwargs: Arguments for pyam.iiasa.Connection.query()

Reading UNFCCC inventory data¶

The package unfccc-di-api (read the docs) provides an interface to the UNFCCC Data Inventory API (link). The pyam package uses this package to query inventory data and return the timeseries data directly as an IamDataFrame.

pyam.read_unfccc(party_code, gases=None, tier=None, mapping=None, model='UNFCCC', scenario='Data Inventory')[source]¶

Read data from the UNFCCC Data Inventory

This function is a wrappter for the unfccc_di_api.UNFCCCApiReader.query().

The data returned from the UNFCCC Data Inventory is transformed into a structure similar to the format used in IPCC reports and IAM model comparison projects. For compatibility with the iam-units package and the convert_unit, emissions species are formatted to standard text (‘CO2’) instead of subscripts (‘CO₂’) and the unit ‘CO₂ equivalent’ used by UNFCCC is replaced by ‘CO2e’.

Parameters:

party_codestr

ISO3-style code for UNFCCC party (country)

gasesstr or list of str, optional

Emission species to be queried from the data inventory can be stated as subscript-format (‘CO₂’) or simple text (‘CO2’)

tierint or list of int

Pre-specified groupings of UNFCCC data to a variable naming format used in IPCC reports and IAM model comparison projects

mappingdict, optional

Mapping to cast UNFCCC-data columns into IAMC-style variables, e.g.

{
    'Emissions|{gas}|Energy': ('1.  Energy', '*', '*', '*'),
}

where the tuple corresponds to filters for the columns [‘category’, ‘classification’, ‘measure’, ‘gas’] and {<col>} tags in the key are replaced by the column value.

modelstr, optional

Name to be used as model identifier

scenariostr, optional

Name to be used as scenario identifier

Returns:

IamDataFrame

Connecting to World Bank data resources¶

The package pandas-datareader (read the docs) implements a number of connections to publicly accessible data resources, e.g., the World Bank Open Data Catalog. pyam provides a simple utility function to cast the queried timeseries data directly as an IamDataFrame.

pyam.read_worldbank(model='World Bank', scenario='WDI', **kwargs)[source]¶

Read data from the World Bank Data Catalogue and return as IamDataFrame

This function is a simple wrapper for the class pandas_datareader.wb.WorldBankReader and the function pandas_datareader.wb.download(). Import the module to retrieve/search the list of indicators (and their id’s), countries, etc.

from pandas_datareader import wb

Parameters:

modelstr, optional: The model name to be used for the returned timeseries data.
scenariostr, optional: The scenario name to be used for the returned timeseries data.
kwargs: passed to pandas_datareader.wb.download()

Returns:

IamDataFrame

Notes

The function pandas_datareader.wb.download() takes an indicator argument, which can be a string or list of strings. If the indicator passed to read_worldbank() is a dictionary of a World Bank id mapped to a string, the variables in the returned IamDataFrame will be renamed.

The function pandas_datareader.wb.download() does not return a unit, but it can be collected for some indicators using the function pandas_datareader.wb.get_indicators(). In the current implementation, unit is defined as n/a for all data; this can be enhanced later (if there is interest from users).