Databases hosted by IIASA¶

The pyam package allows to directly query the scenario databases hosted by the IIASA Energy, Climate and Environment program (ECE), commonly known as the Scenario Explorer infrastructure. It is developed and maintained by the ECE Scenario Services and Scientific Software team.

You do not have to provide username/password credentials to connect to any public database instance using pyam. However, to connect to project-internal databases, you have to create an account at the IIASA-ECE Manager Service (https://manager.ece.iiasa.ac.at). Please contact the respective project coordinator for permission to access a project-internal database.

To store the credentials on your machine so that pyam can use it to query a database, we depend on the Python package ixmp4. You only have to do this once (unless you change your password).

The credentials will be valid for connecting to Scenario Apps based on ixmp4 as well as for (legacy) Scenario Explorer database backends (see below).

In a console, run the following:

ixmp4 login <username>

You will be prompted to enter your password.

Warning

Your username and password will be saved locally in plain-text for future use!

Scenario Apps instances¶

The Scenario Apps use the ixmp4 package as a database backend. You can list all available ixmp4 platforms hosted by IIASA using the following function:

pyam.iiasa.platforms() → None[source]¶

Print a list of available ixmp4 platforms hosted by IIASA

Scenario Explorer instances (legacy service)¶

The Scenario Explorer infrastructure developed by the Scenario Services and Scientific Software team was developed and used for projects from 2018 until 2023.

See this tutorial for more information.

class pyam.iiasa.Connection(name=None, creds=None, auth_url='https://api.manager.ece.iiasa.ac.at')[source]¶

A class to facilitate querying an IIASA Scenario Explorer database API

Parameters:

namestr, optional: The name of a database API. Use valid_connections for a list of available APIs.
credspathlib.Path or str, optional: Path to a file with authentication credentials. This feature is deprecated, please run ixmp4 login <username> in a console instead.
auth_urlstr, optional: custom authentication server URL

Notes

Credentials (username & password) are not required to access public ixmp4 or Scenario Explorer databases (i.e., with Guest login).

Attributes:

current_connection: Currently connected resource (database API connection)
meta_columns: Return the list of meta indicators in the connected resource
valid_connections: Return available resources (database API connections)

Methods

`connect`(name)	Connect to a specific resource (database API)
`index`([default_only])	Return the index of models and scenarios
`meta`([default_only, run_id])	Return categories and indicators (meta) of scenarios
`models`()	List all models in the connected resource
`properties`([default_only])	Return the audit properties of scenarios
`query`([default_only, meta])	Query the connected resource for timeseries data (with filters)
`regions`([include_synonyms])	List all regions in the connected resource
`scenarios`()	List all scenarios in the connected resource
`variables`()	List all variables in the connected resource

convert_regions_payload

connect(name)[source]¶: Connect to a specific resource (database API)

property current_connection¶: Currently connected resource (database API connection)

index(default_only=True, **kwargs)[source]¶

Return the index of models and scenarios

Parameters:

default_onlybool, optional: If True, return only the default version of a model/scenario. If False, return all versions.
**kwargs: Arguments to filter by model and scenario, * can be used as wildcard.

meta(default_only=True, run_id=False, **kwargs)[source]¶

Return categories and indicators (meta) of scenarios

Parameters:

default_onlybool, optional: If True, return only the default version of a model/scenario. If False, return all versions.
run_idbool, optional: Include “run id” column
**kwargs: Arguments to filer by model and scenario, * can be used as wildcard

property meta_columns¶: Return the list of meta indicators in the connected resource

models()[source]¶: List all models in the connected resource

properties(default_only=True, **kwargs)[source]¶

Return the audit properties of scenarios

Parameters:

default_onlybool, optional: If True, return only the default version of a model/scenario. If False, return all versions.
**kwargs: Arguments to filer by model and scenario, * can be used as wildcard

query(default_only=True, meta=True, **kwargs)[source]¶

Query the connected resource for timeseries data (with filters)

Parameters:

default_onlybool, optional

If True, return only the default version of a model/scenario. If False, return all versions.

metabool or list, optional

If True, merge all meta columns indicators (or subset if list is given).

**kwargs

Available keyword arguments include

model
scenario
region
variable

Returns:

IamDataFrame

Examples

You can read from a pyam.iiasa.Connection instance using keyword arguments similar to filtering an IamDataFrame:

Connection.query(
    model="MESSAGE*",
    scenario="SSP2*",
    variable=["Emissions|CO2", "Primary Energy"],
)

regions(include_synonyms=False)[source]¶

List all regions in the connected resource

Parameters:

include_synonymsbool: whether to include synonyms (possibly leading to duplicate region names for regions with more than one synonym)

scenarios()[source]¶: List all scenarios in the connected resource

property valid_connections¶: Return available resources (database API connections)

variables()[source]¶: List all variables in the connected resource

pyam.iiasa.read_iiasa(name, default_only=True, meta=True, creds=None, **kwargs)[source]

Read data from an ixmp4 platform or an IIASA Scenario Explorer database.

Parameters:

namestr: Name of an ixmp4 platform or an IIASA Scenario Explorer database instance.

Use platforms for a list of ixmp4 platforms hosted by IIASA.

Use valid_connections for a list of available Scenario Explorer database instances.
default_onlybool, optional: If True, return only the default version of a model/scenario. If False, return all versions.
metabool or list of strings, optional: If True, include all meta categories & quantitative indicators (or subset if list is given).
credsstr or pathlib.Path, optional: Path to a file with authentication credentials. This feature is deprecated, please run ixmp4 login <username> in a console instead.
**kwargs: Arguments for pyam.read_ixmp4() or pyam.iiasa.Connection.query().

Notes

Credentials (username & password) are not required to access any public ixmp4 or Scenario Explorer database (i.e., with Guest login).

pyam.iiasa.lazy_read_iiasa(file, name, default_only=True, meta=True, creds=None, **kwargs)[source]

Try to load data from a local cache, failing that, loads it from an IIASA database.

Check if the file in a given location is an up-to-date version of an IIASA database. If so, load it. If not, load data from the IIASA scenario explorer database API and save to that location. Does not check that the previously read version is a complete instance of the database, so if the initial load applies a filter, you will read only data that passes the same filter as well as any additional filter you apply.

Parameters:

filestr or pathlib.Path: The location to test for valid data and save the data if not up-to-date. Must be either xlsx or csv.
namestr: Name of an IIASA Scenario Explorer database instance.

Use valid_connections for a list of available instances.
default_onlybool, optional: If True, return only the default version of a model/scenario. If False, return all versions.
metabool or list of strings, optional: If True, include all meta categories & quantitative indicators (or subset if list is given).
credsstr or pathlib.Path, optional: Path to a file with authentication credentials. This feature is deprecated, please run ixmp4 login <username> in a console instead.
**kwargs: Arguments for pyam.read_ixmp4() or pyam.iiasa.Connection.query().

Notes

This feature does currently not support reading data from ixmp4 platforms.

Credentials (username & password) are not required to access any public ixmp4 or Scenario Explorer database (i.e., with Guest login).