Databases hosted by IIASA

The pyam package allows to directly query the scenario databases hosted by the IIASA Energy, Climate and Environment program (ECE), commonly known as the Scenario Explorer infrastructure. It is developed and maintained by the ECE Scenario Services and Scientific Software team.

You do not have to provide username/password credentials to connect to any public database instance using pyam. However, to connect to project-internal databases, you have to create an account at the IIASA-ECE Manager Service (https://manager.ece.iiasa.ac.at). Please contact the respective project coordinator for permission to access a project-internal database.

To store the credentials on your machine so that pyam can use it to query a database, we depend on the Python package ixmp4 . You only have to do this once (unless you change your password).

The credentials will be valid for connecting to Scenario Apps based on ixmp4 as well as for (legacy) Scenario Explorer database backends (see below).

In a console, run the following:

ixmp4 login <username>

You will be prompted to enter your password.

Warning

Your username and password will be saved locally in plain-text for future use!

Scenario Apps instances

Coming soon…

Scenario Explorer instances

The Scenario Explorer infrastructure developed by the Scenario Services and Scientific Software team was developed and used for projects from 2018 until 2023.

See this tutorial for more information.

class pyam.iiasa.Connection(name=None, creds=None, auth_url='https://api.manager.ece.iiasa.ac.at')[source]

A class to facilitate querying an IIASA Scenario Explorer database API

Parameters:
namestr, optional

The name of a database API. Use valid_connections for a list of available APIs.

credsstr or pathlib.Path, optional

By default, the class will search for user credentials which were set using pyam.iiasa.set_config(). Alternatively, you can provide a path to a yaml file with entries of ‘username’ and ‘password’.

auth_urlstr, optional

custom authentication server URL

Notes

Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).

Attributes:
current_connection

Currently connected resource (database API connection)

meta_columns

Return the list of meta indicators in the connected resource

valid_connections

Return available resources (database API connections)

Methods

connect(name)

Connect to a specific resource (database API)

index([default_only])

Return the index of models and scenarios

meta([default_only, run_id])

Return categories and indicators (meta) of scenarios

models()

List all models in the connected resource

properties([default_only])

Return the audit properties of scenarios

query([default_only, meta])

Query the connected resource for timeseries data (with filters)

regions([include_synonyms])

List all regions in the connected resource

scenarios()

List all scenarios in the connected resource

variables()

List all variables in the connected resource

convert_regions_payload

connect(name)[source]

Connect to a specific resource (database API)

property current_connection

Currently connected resource (database API connection)

index(default_only=True, **kwargs)[source]

Return the index of models and scenarios

Parameters:
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

kwargs

Arguments to filter by model and scenario, * can be used as wildcard.

meta(default_only=True, run_id=False, **kwargs)[source]

Return categories and indicators (meta) of scenarios

Parameters:
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

run_idbool, optional

Include “run id” column

kwargs

Arguments to filer by model and scenario, * can be used as wildcard

property meta_columns

Return the list of meta indicators in the connected resource

models()[source]

List all models in the connected resource

properties(default_only=True, **kwargs)[source]

Return the audit properties of scenarios

Parameters:
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

kwargs

Arguments to filer by model and scenario, * can be used as wildcard

query(default_only=True, meta=True, **kwargs)[source]

Query the connected resource for timeseries data (with filters)

Parameters:
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list, optional

If True, merge all meta columns indicators (or subset if list is given).

kwargs

Available keyword arguments include

  • model

  • scenario

  • region

  • variable

Returns:
IamDataFrame

Examples

You can read from a pyam.iiasa.Connection instance using keyword arguments similar to filtering an IamDataFrame:

Connection.query(model='MESSAGE*', scenario='SSP2*',
                 variable=['Emissions|CO2', 'Primary Energy'])
regions(include_synonyms=False)[source]

List all regions in the connected resource

Parameters:
include_synonymsbool

whether to include synonyms (possibly leading to duplicate region names for regions with more than one synonym)

scenarios()[source]

List all scenarios in the connected resource

property valid_connections

Return available resources (database API connections)

variables()[source]

List all variables in the connected resource

pyam.iiasa.read_iiasa(name, default_only=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]

Query an IIASA Scenario Explorer database API and return as IamDataFrame

Parameters:
namestr
Name of an IIASA Scenario Explorer database instance.
Use valid_connections for a list of available instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional
Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).
See pyam.iiasa.Connection for details.
Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr

Authentication server URL

kwargs

Arguments for pyam.iiasa.Connection.query()

pyam.iiasa.lazy_read_iiasa(file, name, default_only=True, meta=True, creds=None, base_url='https://api.manager.ece.iiasa.ac.at', **kwargs)[source]

Try to load data from a local cache, failing that, loads it from the internet.

Check if the file in a given location is an up-to-date version of an IIASA database. If so, load it. If not, load data from the IIASA scenario explorer database API and save to that location. Does not check that the previously read version is a complete instance of the database, so if the initial load applies a filter, you will read only data that passes the same filter as well as any additional filter you apply.

Parameters:
filestr or pathlib.Path

The location to test for valid data and save the data if not up-to-date. Must be either xlsx or csv.

namestr
Name of an IIASA Scenario Explorer database instance.
Use valid_connections for a list of available instances.
default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list of strings, optional

If True, include all meta categories & quantitative indicators (or subset if list is given).

credsstr or pathlib.Path, optional
Credentials (username & password) are not required to access any public Scenario Explorer instances (i.e., with Guest login).
See pyam.iiasa.Connection for details.
Use pyam.iiasa.set_config() to set credentials for accessing private/restricted Scenario Explorer instances.
base_urlstr

Authentication server URL

kwargs

Arguments for pyam.iiasa.Connection.query()