Read directly from IIASA data resources

The IIASA Energy Program hosts a suite of Scenario Explorer instances and related infrastructure to support analysis of integrated-assessment pathways in IPCC reports and model comparison projects. High-profile use cases include the IAMC 1.5°C Scenario Explorer hosted by IIASA supporting the IPCC Special Report on Global Warming of 1.5°C (SR15) and the Horizon 2020 project CD-LINKS.

IIASA’s modeling platform infrastructure and the Scenario Explorer UI is not only a great resource on its own, but it also allows the underlying datasets to be directly queried. pyam takes advantage of this ability to allow you to easily pull data and work with it in your Python data processing and analysis workflow.

[1]:
import pyam
pyam - INFO: Running in a notebook, setting `pyam` logging level to `logging.INFO` and adding stderr handler

Connecting to a data resource (aka the database API of a Scenario Explorer instance)

Accessing a data resource is done via a Connection object. By default, your can connect to all public scenario explorers instances.

[2]:
conn = pyam.iiasa.Connection()
conn.valid_connections
pyam.iiasa - INFO: You are connected as an anonymous user
[2]:
['cdlinks',
 'cmin',
 'deeds',
 'hotspots',
 'integration-test',
 'nexus-basins',
 'ngfs',
 'paris_lttg',
 'ripples',
 'senses',
 'setnav',
 'iamc15',
 'iamc15-qa',
 'sandbox']

If you have credentials to connect to a non-public or restricted Scenario Explorer instance, you can store this information by running the following command in a separate Python console:

import pyam
pyam.iiasa.set_config(<username>, <password>)

When initializing a new Connection instance, pyam will automatically search for the configuration in a known location.

In this example, we will be retrieving data from the IAMC 1.5°C Scenario Explorer hosted by IIASA (link), which provides the quantiative scenario ensemble underpinning the IPCC Special Report on Global Warming of 1.5C (SR15).

This can be done either via the constructor:

pyam.iiasa.Connection('iamc15')

or, if you want to query multiple databases, via the explicit connect() method:

conn = pyam.iiasa.Connection()
conn.connect('iamc15')

We also provide some convenience functions to shorten the amount of code you have to write. Under the hood, read_iiasa() is just opening a connection to a database API and sends a query to the resource.

In this tutorial, we will query specific subsets of data in a manner similar to pyam.IamDataFrame.filter().

[3]:
df = pyam.read_iiasa(
    'iamc15',
    model='MESSAGEix*',
    variable=['Emissions|CO2', 'Primary Energy|Coal'],
    region='World',
    meta=['category']
)
pyam.iiasa - INFO: You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about
pyam.iiasa - INFO: You are connected as an anonymous user

Here we pulled out all times series data for model(s) that start with ‘MESSAGEix’ that are in the ‘World’ region and associated with the two named variables. We also added the meta column “category”, which tells us the climate impact categorisation of each scenario as assessed in the IPCC SR15.

Let’s plot CO2 emissions.

[4]:
ax = df.filter(variable='Emissions|CO2').line_plot(
    color='category',
    legend=dict(loc='center left', bbox_to_anchor=(1.0, 0.5))
)
../_images/tutorials_iiasa_dbs_9_0.png

And now continue doing all of your analysis!

[5]:
ax = df.scatter(
    x='Primary Energy|Coal',
    y='Emissions|CO2',
    color='category',
    legend=dict(loc='center left', bbox_to_anchor=(1.0, 0.5))
)
../_images/tutorials_iiasa_dbs_11_0.png

Exploring the data resource

If you’re interested in what data is available in the data source, you can use pyam.iiasa.Connection to do so.

[6]:
conn = pyam.iiasa.Connection('iamc15')
pyam.iiasa - INFO: You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about
pyam.iiasa - INFO: You are connected as an anonymous user

The Connection object has a number of useful functions for listing what’s available in the data resource. These functions follow the conventions of the IamDataFrame class (where possible).

A few of them are shown below.

[7]:
conn.models().head()
[7]:
0        AIM/CGE 2.0
1        AIM/CGE 2.1
2      C-ROADS-5.005
3           GCAM 4.2
4    GENeSYS-MOD 1.0
Name: model, dtype: object
[8]:
conn.scenarios().head()
[8]:
0    ADVANCE_2020_1.5C-2100
1        ADVANCE_2020_Med2C
2         ADVANCE_2020_WB2C
3        ADVANCE_2030_Med2C
4    ADVANCE_2030_Price1.5C
Name: scenario, dtype: object
[9]:
conn.variables().head()
[9]:
0    AR5 climate diagnostics|Concentration|CO2|FAIR...
1    AR5 climate diagnostics|Concentration|CO2|MAGI...
2    AR5 climate diagnostics|Forcing|Aerosol|Direct...
3    AR5 climate diagnostics|Forcing|Aerosol|MAGICC...
4    AR5 climate diagnostics|Forcing|Aerosol|Total|...
Name: variable, dtype: object
[10]:
conn.regions().head()
[10]:
0     World
1    R5ROWO
2    R5ASIA
3     R5LAM
4     R5MAF
Name: region, dtype: object

A number of different categorization and quantitative indicators are available for model/scenario combinations. These are usually called meta indicators in pyam.

We queried the meta-indicator “category” in the above example, but there are many more. You can get a list with the following command:

[11]:
conn.meta_columns.head()
[11]:
0       Kyoto-GHG|2010 (SAR)
1                   baseline
2          carbon price|2030
3    carbon price|2030 (NPV)
4          carbon price|2050
Name: name, dtype: object

You can directly query the Connection, which will return a pyam.IamDataFrame

[12]:
df = conn.query(
    model='MESSAGEix*',
    variable=['Emissions|CO2', 'Primary Energy|Coal'],
    region='World'
)

…so that you can directly continue with your analysis and visualization workflow using pyam!

[13]:
ax = df.filter(variable='Primary Energy|Coal').line_plot(
    color='scenario',
    legend=dict(loc='center left', bbox_to_anchor=(1.0, 0.5))
)
../_images/tutorials_iiasa_dbs_24_0.png
[ ]: