{
"cells": [
{
"cell_type": "markdown",
"id": "welcome-sharing",
"metadata": {},
"source": [
"# Integration with R\n",
"\n",
"This tutorial illustrates how to use **pyam** features\n",
"in an **R workflow** using the [reticulate package](https://rstudio.github.io/reticulate/).\n",
"\n",
"\n",
"\n",
"The source code of the tutorial is available in the folder\n",
"[docs/R_tutorials](https://github.com/IAMconsortium/pyam/tree/master/docs/R_tutorials)\n",
"of the **pyam** GitHub repository.\n",
"\n",
"## Installation and requirements\n",
"\n",
"Assuming you have a working and current R installation,\n",
"you can follow the steps below to install the requirements to run this tutorial:\n",
"\n",
"0. Install Python using [Anaconda](https://www.anaconda.com) \n",
" *This is the recommended approach for users that aren't already experienced with Python...* \n",
"\n",
"\n",
"1. Install **pyam** ([read the docs](https://pyam-iamc.readthedocs.io/en/stable/install.html)\n",
" for more information) \n",
" ```\n",
" conda install -c conda-forge pyam\n",
" ```\n",
"\n",
"2. To use R in a Jupyter notebook as in this tutorial,\n",
" install Jupyter and the [IRkernel](https://irkernel.github.io) via conda \n",
" ```\n",
" conda install jupyter ir-kernel\n",
" ```\n",
"\n",
"3. In R, install the **reticulate** package from CRAN \n",
" ``` r\n",
" install.packages(\"reticulate\")\n",
" ```\n",
"\n",
"
\n",
"\n",
"This tutorial is an illustration on how to get started,\n",
"not a full-fledged documentation of a **pyam-R** interface... \n",
"The integration between Python and R using **reticulate** is a bit fickle -\n",
"having multiple Python installations/environments\n",
"or outdated versions of **reticulate**/**pandas** can cause issues. \n",
"Please [read the docs](https://rstudio.github.io/reticulate/) if you experience any problems!\n",
"\n",
"
\n",
"\n",
"This notebook was run with R version 4.0 and Python 3.8 on Mac OS.\n",
"\n",
"****\n",
"\n",
"**Developers note:**\n",
"Running this notebook on CI and RTD is currently not supported. \n",
"For the time being, this notebook is not executed by **nbsphinx**\n",
"and has to be saved with output."
]
},
{
"cell_type": "markdown",
"id": "surprised-charles",
"metadata": {},
"source": [
"## Setting up the session"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "specified-administration",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Loading required package: reticulate\n",
"\n"
]
}
],
"source": [
"require(reticulate)\n"
]
},
{
"cell_type": "markdown",
"id": "durable-longer",
"metadata": {},
"source": [
"The next line instructs **reticulate** to use the conda 'base' environment. \n",
"[Read the docs](https://rstudio.github.io/reticulate/reference/use_python.html)\n",
"in case you want to use an alternative installation or environment!"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "varied-functionality",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [],
"source": [
"use_condaenv()\n"
]
},
{
"cell_type": "markdown",
"id": "developing-collect",
"metadata": {},
"source": [
"Now, we import **pyam** to the R session."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "registered-stack",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [],
"source": [
"pyam <- import(\"pyam\")\n"
]
},
{
"cell_type": "markdown",
"id": "christian-share",
"metadata": {},
"source": [
"## Passing an R dataframe to pyam\n",
"\n",
"The first cell of this section creates a simple timeseries data table\n",
"following the IAMC format as an R dataframe."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "increased-substance",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"A data.frame: 3 × 7\n",
"\n",
"\tmodel | scenario | region | variable | unit | X2005 | X2010 |
\n",
"\t<chr> | <chr> | <chr> | <chr> | <chr> | <dbl> | <dbl> |
\n",
"\n",
"\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 1.0 | 6 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 0.5 | 3 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2.0 | 7 |
\n",
"\n",
"
\n"
],
"text/latex": [
"A data.frame: 3 × 7\n",
"\\begin{tabular}{lllllll}\n",
" model & scenario & region & variable & unit & X2005 & X2010\\\\\n",
" & & & & & & \\\\\n",
"\\hline\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 1.0 & 6\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 0.5 & 3\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2.0 & 7\\\\\n",
"\\end{tabular}\n"
],
"text/markdown": [
"\n",
"A data.frame: 3 × 7\n",
"\n",
"| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | X2005 <dbl> | X2010 <dbl> |\n",
"|---|---|---|---|---|---|---|\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 1.0 | 6 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 0.5 | 3 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2.0 | 7 |\n",
"\n"
],
"text/plain": [
" model scenario region variable unit X2005 X2010\n",
"1 model_a scen_a World Primary Energy EJ/yr 1.0 6 \n",
"2 model_a scen_a World Primary Energy|Coal EJ/yr 0.5 3 \n",
"3 model_a scen_b World Primary Energy EJ/yr 2.0 7 "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"data <- data.frame(\n",
" model = c(\"model_a\", \"model_a\", \"model_a\"),\n",
" scenario = c(\"scen_a\", \"scen_a\", \"scen_b\"),\n",
" region = c(\"World\", \"World\", \"World\"),\n",
" variable = c(\"Primary Energy\", \"Primary Energy|Coal\", \"Primary Energy\"),\n",
" unit = c(\"EJ/yr\", \"EJ/yr\", \"EJ/yr\"),\n",
" \"2005\" = c(1, 0.5, 2),\n",
" \"2010\" = c(6, 3, 7)\n",
")\n",
"data\n"
]
},
{
"cell_type": "markdown",
"id": "chemical-prophet",
"metadata": {},
"source": [
"The following cell casts the data to a **pyam.IamDataFrame**.\n",
"\n",
"\n",
"\n",
"When migrating code from **Python** to **R**, keep in mind that \n",
"you have to use `$` instead of `.` to call an object's methods!\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "spread-spring",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"data": {
"text/plain": [
"\n",
"Index:\n",
" * model : model_a (1)\n",
" * scenario : scen_a, scen_b (2)\n",
"Timeseries data coordinates:\n",
" region : World (1)\n",
" variable : Primary Energy, Primary Energy|Coal (2)\n",
" unit : EJ/yr (1)\n",
" year : 2005, 2010 (2)\n",
"Meta indicators:\n",
" exclude (bool) False (1)"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"df <- pyam$IamDataFrame(data)\n",
"df\n"
]
},
{
"cell_type": "markdown",
"id": "figured-client",
"metadata": {},
"source": [
"*Ending a cell of a Jupyter notebook with the **IamDataFrame** instance\n",
"shows the same overview as if you were in a Python environment!*\n",
"\n",
"The last cell of this section retrieves the timeseries data from the **IamDataFrame** in long form\n",
"and returns it again as an **R data.frame**."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "different-consensus",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"A data.frame: 6 × 7\n",
"\n",
"\tmodel | scenario | region | variable | unit | year | value |
\n",
"\t<chr> | <chr> | <chr> | <chr> | <chr> | <dbl> | <dbl> |
\n",
"\n",
"\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |
\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |
\n",
"\n",
"
\n"
],
"text/latex": [
"A data.frame: 6 × 7\n",
"\\begin{tabular}{lllllll}\n",
" model & scenario & region & variable & unit & year & value\\\\\n",
" & & & & & & \\\\\n",
"\\hline\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2005 & 1.0\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2010 & 6.0\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2005 & 0.5\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2010 & 3.0\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2005 & 2.0\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2010 & 7.0\\\\\n",
"\\end{tabular}\n"
],
"text/markdown": [
"\n",
"A data.frame: 6 × 7\n",
"\n",
"| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | year <dbl> | value <dbl> |\n",
"|---|---|---|---|---|---|---|\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |\n",
"\n"
],
"text/plain": [
" model scenario region variable unit year value\n",
"1 model_a scen_a World Primary Energy EJ/yr 2005 1.0 \n",
"2 model_a scen_a World Primary Energy EJ/yr 2010 6.0 \n",
"3 model_a scen_a World Primary Energy|Coal EJ/yr 2005 0.5 \n",
"4 model_a scen_a World Primary Energy|Coal EJ/yr 2010 3.0 \n",
"5 model_a scen_b World Primary Energy EJ/yr 2005 2.0 \n",
"6 model_a scen_b World Primary Energy EJ/yr 2010 7.0 "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"df$data\n"
]
},
{
"cell_type": "markdown",
"id": "angry-tragedy",
"metadata": {},
"source": [
"## Calling a pyam method\n",
"\n",
"For illustration, we will use the [interpolate()](https://pyam-iamc.readthedocs.io/en/stable/api/iamdataframe.html#pyam.IamDataFrame.interpolate) method\n",
"on the **IamDataFrame** instance.\n",
"\n",
"\n",
"\n",
"One key difference between **R** and **Python** are the default types: \n",
"whereas Python will interpret \"2007\" as an integer, R will treat the same input as a float. \n",
"In the example below, we have to explicitly instruct R to pass an integer to the **pyam** function.\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "urban-eclipse",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [],
"source": [
"x <- df$interpolate(as.integer(2007), inplace = TRUE)\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "confident-lebanon",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"A data.frame: 9 × 7\n",
"\n",
"\tmodel | scenario | region | variable | unit | year | value |
\n",
"\t<chr> | <chr> | <chr> | <chr> | <chr> | <dbl> | <dbl> |
\n",
"\n",
"\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |
\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 2007 | 3.0 |
\n",
"\tmodel_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2007 | 1.5 |
\n",
"\tmodel_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2007 | 4.0 |
\n",
"\tmodel_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |
\n",
"\n",
"
\n"
],
"text/latex": [
"A data.frame: 9 × 7\n",
"\\begin{tabular}{lllllll}\n",
" model & scenario & region & variable & unit & year & value\\\\\n",
" & & & & & & \\\\\n",
"\\hline\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2005 & 1.0\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2007 & 3.0\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2010 & 6.0\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2005 & 0.5\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2007 & 1.5\\\\\n",
"\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2010 & 3.0\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2005 & 2.0\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2007 & 4.0\\\\\n",
"\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2010 & 7.0\\\\\n",
"\\end{tabular}\n"
],
"text/markdown": [
"\n",
"A data.frame: 9 × 7\n",
"\n",
"| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | year <dbl> | value <dbl> |\n",
"|---|---|---|---|---|---|---|\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 2007 | 3.0 |\n",
"| model_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2007 | 1.5 |\n",
"| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2007 | 4.0 |\n",
"| model_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |\n",
"\n"
],
"text/plain": [
" model scenario region variable unit year value\n",
"1 model_a scen_a World Primary Energy EJ/yr 2005 1.0 \n",
"2 model_a scen_a World Primary Energy EJ/yr 2007 3.0 \n",
"3 model_a scen_a World Primary Energy EJ/yr 2010 6.0 \n",
"4 model_a scen_a World Primary Energy|Coal EJ/yr 2005 0.5 \n",
"5 model_a scen_a World Primary Energy|Coal EJ/yr 2007 1.5 \n",
"6 model_a scen_a World Primary Energy|Coal EJ/yr 2010 3.0 \n",
"7 model_a scen_b World Primary Energy EJ/yr 2005 2.0 \n",
"8 model_a scen_b World Primary Energy EJ/yr 2007 4.0 \n",
"9 model_a scen_b World Primary Energy EJ/yr 2010 7.0 "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"df$data\n"
]
},
{
"cell_type": "markdown",
"id": "d4fa2637",
"metadata": {},
"source": [
"## Query data from an IIASA data resource\n",
"\n",
"The IIASA *Energy, Climate, and Environment* Program hosts a suite of **Scenario Explorer** instances and related infrastructure to support analysis of integrated-assessment pathways in IPCC reports and model comparison projects. \n",
"High-profile use cases include the [IAMC 1.5°C Scenario Explorer hosted by IIASA](https://data.ece.iiasa.ac.at/iamc-1.5c-explorer) supporting the *IPCC Special Report on Global Warming of 1.5°C* (SR15) and the Horizon 2020 project [CD-LINKS](https://data.ece.iiasa.ac.at/cd-links).\n",
"\n",
"The **pyam** package can retrieve scenario data directly from any **Scenario Explorer** database instance\n",
"hosted by IIASA for use in your processing and analysis workflows."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "91c7fddf",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [],
"source": [
"df <- pyam$read_iiasa(\n",
" \"iamc15\",\n",
" model = \"MESSAGEix*\",\n",
" variable = c(\"Emissions|CO2\", \"Primary Energy|Coal\"),\n",
" region = \"World\",\n",
" meta = c(\"category\")\n",
")\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "377eac4c",
"metadata": {
"vscode": {
"languageId": "r"
}
},
"outputs": [
{
"data": {
"text/plain": [
"\n",
"Index:\n",
" * model : MESSAGEix-GLOBIOM 1.0 (1)\n",
" * scenario : CD-LINKS_INDCi, CD-LINKS_NPi, CD-LINKS_NPi2020_1000, ... LowEnergyDemand (7)\n",
"Timeseries data coordinates:\n",
" region : World (1)\n",
" variable : Emissions|CO2, Primary Energy|Coal (2)\n",
" unit : EJ/yr, Mt CO2/yr (2)\n",
" year : 2000, 2005, 2010, 2020, 2030, 2040, 2050, 2060, ... 2100 (12)\n",
"Meta indicators:\n",
" version (object) 1 (1)\n",
" carbon price|Avg NPV (2030-2100) (object) 0.9123249940000001, ... 30.1503013256 (7)\n",
" year of netzero CO2 emissions (object) nan, 2065, 2078, 2053, 2060.0 (5)\n",
" carbon price|2100 (NPV) (object) 0.0426419715, 0.012143570900000001, ... 14.2051784281 (7)\n",
" cumulative CO2 emissions (2016 to peak warming, Gt CO2) (object) ... (7)\n",
" ..."
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"df\n"
]
},
{
"cell_type": "markdown",
"id": "a0eb9b43",
"metadata": {},
"source": [
"See the [pyam-IIASA-database tutorial](https://pyam-iamc.readthedocs.io/en/stable/tutorials/iiasa.html)\n",
"or the [API documentation](https://pyam-iamc.readthedocs.io/en/stable/api/iiasa.html)\n",
"for more information and a complete list of features!"
]
},
{
"cell_type": "markdown",
"id": "isolated-whole",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"\n",
" \n",
"**New to Jupyter notebooks?** \n",
"Read [this page](https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/) for helpful tips and tricks when working with Jupyter notebooks.\n",
"\n",
"
\n",
"\n",
"### Questions?\n",
"\n",
"Take a look at the other tutorials to see the scope of **pyam** features - then join our [mailing list](https://groups.io/g/pyam)!\n",
"\n",
"### Problems?\n",
"\n",
"If you encounter any functions or methods that don't work as expected,\n",
"please check whether there are already any [issues](https://github.com/IAMconsortium/pyam/labels/R) in our GitHub repo. \n",
"If not, start a [new one](https://github.com/IAMconsortium/pyam/issues/new)!"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "R",
"language": "R",
"name": "ir"
},
"language_info": {
"codemirror_mode": "r",
"file_extension": ".r",
"mimetype": "text/x-r-source",
"name": "R",
"pygments_lexer": "r",
"version": "4.0.2"
},
"nbsphinx": {
"execute": "never"
}
},
"nbformat": 4,
"nbformat_minor": 5
}