{ "cells": [ { "cell_type": "markdown", "id": "welcome-sharing", "metadata": {}, "source": [ "# Integration with R\n", "\n", "This tutorial illustrates how to use **pyam** features\n", "in an **R workflow** using the [reticulate package](https://rstudio.github.io/reticulate/).\n", "\n", "\n", "\n", "The source code of the tutorial is available in the folder\n", "[docs/R_tutorials](https://github.com/IAMconsortium/pyam/tree/master/docs/R_tutorials)\n", "of the **pyam** GitHub repository.\n", "\n", "## Installation and requirements\n", "\n", "Assuming you have a working and current R installation,\n", "you can follow the steps below to install the requirements to run this tutorial:\n", "\n", "0. Install Python using [Anaconda](https://www.anaconda.com) \n", " *This is the recommended approach for users that aren't already experienced with Python...* \n", "\n", "\n", "1. Install **pyam** ([read the docs](https://pyam-iamc.readthedocs.io/en/stable/install.html)\n", " for more information) \n", " ```\n", " conda install -c conda-forge pyam\n", " ```\n", "\n", "2. To use R in a Jupyter notebook as in this tutorial,\n", " install Jupyter and the [IRkernel](https://irkernel.github.io) via conda \n", " ```\n", " conda install jupyter ir-kernel\n", " ```\n", "\n", "3. In R, install the **reticulate** package from CRAN \n", " ``` r\n", " install.packages(\"reticulate\")\n", " ```\n", "\n", "
\n", "\n", "This tutorial is an illustration on how to get started,\n", "not a full-fledged documentation of a **pyam-R** interface... \n", "The integration between Python and R using **reticulate** is a bit fickle -\n", "having multiple Python installations/environments\n", "or outdated versions of **reticulate**/**pandas** can cause issues. \n", "Please [read the docs](https://rstudio.github.io/reticulate/) if you experience any problems!\n", "\n", "
\n", "\n", "This notebook was run with R version 4.0 and Python 3.8 on Mac OS.\n", "\n", "****\n", "\n", "**Developers note:**\n", "Running this notebook on CI and RTD is currently not supported. \n", "For the time being, this notebook is not executed by **nbsphinx**\n", "and has to be saved with output." ] }, { "cell_type": "markdown", "id": "surprised-charles", "metadata": {}, "source": [ "## Setting up the session" ] }, { "cell_type": "code", "execution_count": 1, "id": "specified-administration", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Loading required package: reticulate\n", "\n" ] } ], "source": [ "require(reticulate)\n" ] }, { "cell_type": "markdown", "id": "durable-longer", "metadata": {}, "source": [ "The next line instructs **reticulate** to use the conda 'base' environment. \n", "[Read the docs](https://rstudio.github.io/reticulate/reference/use_python.html)\n", "in case you want to use an alternative installation or environment!" ] }, { "cell_type": "code", "execution_count": 2, "id": "varied-functionality", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "use_condaenv()\n" ] }, { "cell_type": "markdown", "id": "developing-collect", "metadata": {}, "source": [ "Now, we import **pyam** to the R session." ] }, { "cell_type": "code", "execution_count": 3, "id": "registered-stack", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "pyam <- import(\"pyam\")\n" ] }, { "cell_type": "markdown", "id": "christian-share", "metadata": {}, "source": [ "## Passing an R dataframe to pyam\n", "\n", "The first cell of this section creates a simple timeseries data table\n", "following the IAMC format as an R dataframe." ] }, { "cell_type": "code", "execution_count": 4, "id": "increased-substance", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.frame: 3 × 7
modelscenarioregionvariableunitX2005X2010
<chr><chr><chr><chr><chr><dbl><dbl>
model_ascen_aWorldPrimary Energy EJ/yr1.06
model_ascen_aWorldPrimary Energy|CoalEJ/yr0.53
model_ascen_bWorldPrimary Energy EJ/yr2.07
\n" ], "text/latex": [ "A data.frame: 3 × 7\n", "\\begin{tabular}{lllllll}\n", " model & scenario & region & variable & unit & X2005 & X2010\\\\\n", " & & & & & & \\\\\n", "\\hline\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 1.0 & 6\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 0.5 & 3\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2.0 & 7\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.frame: 3 × 7\n", "\n", "| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | X2005 <dbl> | X2010 <dbl> |\n", "|---|---|---|---|---|---|---|\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 1.0 | 6 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 0.5 | 3 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2.0 | 7 |\n", "\n" ], "text/plain": [ " model scenario region variable unit X2005 X2010\n", "1 model_a scen_a World Primary Energy EJ/yr 1.0 6 \n", "2 model_a scen_a World Primary Energy|Coal EJ/yr 0.5 3 \n", "3 model_a scen_b World Primary Energy EJ/yr 2.0 7 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "data <- data.frame(\n", " model = c(\"model_a\", \"model_a\", \"model_a\"),\n", " scenario = c(\"scen_a\", \"scen_a\", \"scen_b\"),\n", " region = c(\"World\", \"World\", \"World\"),\n", " variable = c(\"Primary Energy\", \"Primary Energy|Coal\", \"Primary Energy\"),\n", " unit = c(\"EJ/yr\", \"EJ/yr\", \"EJ/yr\"),\n", " \"2005\" = c(1, 0.5, 2),\n", " \"2010\" = c(6, 3, 7)\n", ")\n", "data\n" ] }, { "cell_type": "markdown", "id": "chemical-prophet", "metadata": {}, "source": [ "The following cell casts the data to a **pyam.IamDataFrame**.\n", "\n", "
\n", "\n", "When migrating code from **Python** to **R**, keep in mind that \n", "you have to use `$` instead of `.` to call an object's methods!\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 5, "id": "spread-spring", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/plain": [ "\n", "Index:\n", " * model : model_a (1)\n", " * scenario : scen_a, scen_b (2)\n", "Timeseries data coordinates:\n", " region : World (1)\n", " variable : Primary Energy, Primary Energy|Coal (2)\n", " unit : EJ/yr (1)\n", " year : 2005, 2010 (2)\n", "Meta indicators:\n", " exclude (bool) False (1)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df <- pyam$IamDataFrame(data)\n", "df\n" ] }, { "cell_type": "markdown", "id": "figured-client", "metadata": {}, "source": [ "*Ending a cell of a Jupyter notebook with the **IamDataFrame** instance\n", "shows the same overview as if you were in a Python environment!*\n", "\n", "The last cell of this section retrieves the timeseries data from the **IamDataFrame** in long form\n", "and returns it again as an **R data.frame**." ] }, { "cell_type": "code", "execution_count": 6, "id": "different-consensus", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.frame: 6 × 7
modelscenarioregionvariableunityearvalue
<chr><chr><chr><chr><chr><dbl><dbl>
model_ascen_aWorldPrimary Energy EJ/yr20051.0
model_ascen_aWorldPrimary Energy EJ/yr20106.0
model_ascen_aWorldPrimary Energy|CoalEJ/yr20050.5
model_ascen_aWorldPrimary Energy|CoalEJ/yr20103.0
model_ascen_bWorldPrimary Energy EJ/yr20052.0
model_ascen_bWorldPrimary Energy EJ/yr20107.0
\n" ], "text/latex": [ "A data.frame: 6 × 7\n", "\\begin{tabular}{lllllll}\n", " model & scenario & region & variable & unit & year & value\\\\\n", " & & & & & & \\\\\n", "\\hline\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2005 & 1.0\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2010 & 6.0\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2005 & 0.5\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2010 & 3.0\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2005 & 2.0\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2010 & 7.0\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.frame: 6 × 7\n", "\n", "| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | year <dbl> | value <dbl> |\n", "|---|---|---|---|---|---|---|\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |\n", "\n" ], "text/plain": [ " model scenario region variable unit year value\n", "1 model_a scen_a World Primary Energy EJ/yr 2005 1.0 \n", "2 model_a scen_a World Primary Energy EJ/yr 2010 6.0 \n", "3 model_a scen_a World Primary Energy|Coal EJ/yr 2005 0.5 \n", "4 model_a scen_a World Primary Energy|Coal EJ/yr 2010 3.0 \n", "5 model_a scen_b World Primary Energy EJ/yr 2005 2.0 \n", "6 model_a scen_b World Primary Energy EJ/yr 2010 7.0 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df$data\n" ] }, { "cell_type": "markdown", "id": "angry-tragedy", "metadata": {}, "source": [ "## Calling a pyam method\n", "\n", "For illustration, we will use the [interpolate()](https://pyam-iamc.readthedocs.io/en/stable/api/iamdataframe.html#pyam.IamDataFrame.interpolate) method\n", "on the **IamDataFrame** instance.\n", "\n", "
\n", "\n", "One key difference between **R** and **Python** are the default types: \n", "whereas Python will interpret \"2007\" as an integer, R will treat the same input as a float. \n", "In the example below, we have to explicitly instruct R to pass an integer to the **pyam** function.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 7, "id": "urban-eclipse", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "x <- df$interpolate(as.integer(2007), inplace = TRUE)\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "confident-lebanon", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.frame: 9 × 7
modelscenarioregionvariableunityearvalue
<chr><chr><chr><chr><chr><dbl><dbl>
model_ascen_aWorldPrimary Energy EJ/yr20051.0
model_ascen_aWorldPrimary Energy EJ/yr20073.0
model_ascen_aWorldPrimary Energy EJ/yr20106.0
model_ascen_aWorldPrimary Energy|CoalEJ/yr20050.5
model_ascen_aWorldPrimary Energy|CoalEJ/yr20071.5
model_ascen_aWorldPrimary Energy|CoalEJ/yr20103.0
model_ascen_bWorldPrimary Energy EJ/yr20052.0
model_ascen_bWorldPrimary Energy EJ/yr20074.0
model_ascen_bWorldPrimary Energy EJ/yr20107.0
\n" ], "text/latex": [ "A data.frame: 9 × 7\n", "\\begin{tabular}{lllllll}\n", " model & scenario & region & variable & unit & year & value\\\\\n", " & & & & & & \\\\\n", "\\hline\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2005 & 1.0\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2007 & 3.0\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy & EJ/yr & 2010 & 6.0\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2005 & 0.5\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2007 & 1.5\\\\\n", "\t model\\_a & scen\\_a & World & Primary Energy\\textbar{}Coal & EJ/yr & 2010 & 3.0\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2005 & 2.0\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2007 & 4.0\\\\\n", "\t model\\_a & scen\\_b & World & Primary Energy & EJ/yr & 2010 & 7.0\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.frame: 9 × 7\n", "\n", "| model <chr> | scenario <chr> | region <chr> | variable <chr> | unit <chr> | year <dbl> | value <dbl> |\n", "|---|---|---|---|---|---|---|\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 2005 | 1.0 |\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 2007 | 3.0 |\n", "| model_a | scen_a | World | Primary Energy | EJ/yr | 2010 | 6.0 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2005 | 0.5 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2007 | 1.5 |\n", "| model_a | scen_a | World | Primary Energy|Coal | EJ/yr | 2010 | 3.0 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2005 | 2.0 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2007 | 4.0 |\n", "| model_a | scen_b | World | Primary Energy | EJ/yr | 2010 | 7.0 |\n", "\n" ], "text/plain": [ " model scenario region variable unit year value\n", "1 model_a scen_a World Primary Energy EJ/yr 2005 1.0 \n", "2 model_a scen_a World Primary Energy EJ/yr 2007 3.0 \n", "3 model_a scen_a World Primary Energy EJ/yr 2010 6.0 \n", "4 model_a scen_a World Primary Energy|Coal EJ/yr 2005 0.5 \n", "5 model_a scen_a World Primary Energy|Coal EJ/yr 2007 1.5 \n", "6 model_a scen_a World Primary Energy|Coal EJ/yr 2010 3.0 \n", "7 model_a scen_b World Primary Energy EJ/yr 2005 2.0 \n", "8 model_a scen_b World Primary Energy EJ/yr 2007 4.0 \n", "9 model_a scen_b World Primary Energy EJ/yr 2010 7.0 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df$data\n" ] }, { "cell_type": "markdown", "id": "d4fa2637", "metadata": {}, "source": [ "## Query data from an IIASA data resource\n", "\n", "The IIASA *Energy, Climate, and Environment* Program hosts a suite of **Scenario Explorer** instances and related infrastructure to support analysis of integrated-assessment pathways in IPCC reports and model comparison projects. \n", "High-profile use cases include the [IAMC 1.5°C Scenario Explorer hosted by IIASA](https://data.ece.iiasa.ac.at/iamc-1.5c-explorer) supporting the *IPCC Special Report on Global Warming of 1.5°C* (SR15) and the Horizon 2020 project [CD-LINKS](https://data.ece.iiasa.ac.at/cd-links).\n", "\n", "The **pyam** package can retrieve scenario data directly from any **Scenario Explorer** database instance\n", "hosted by IIASA for use in your processing and analysis workflows." ] }, { "cell_type": "code", "execution_count": 9, "id": "91c7fddf", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "df <- pyam$read_iiasa(\n", " \"iamc15\",\n", " model = \"MESSAGEix*\",\n", " variable = c(\"Emissions|CO2\", \"Primary Energy|Coal\"),\n", " region = \"World\",\n", " meta = c(\"category\")\n", ")\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "377eac4c", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/plain": [ "\n", "Index:\n", " * model : MESSAGEix-GLOBIOM 1.0 (1)\n", " * scenario : CD-LINKS_INDCi, CD-LINKS_NPi, CD-LINKS_NPi2020_1000, ... LowEnergyDemand (7)\n", "Timeseries data coordinates:\n", " region : World (1)\n", " variable : Emissions|CO2, Primary Energy|Coal (2)\n", " unit : EJ/yr, Mt CO2/yr (2)\n", " year : 2000, 2005, 2010, 2020, 2030, 2040, 2050, 2060, ... 2100 (12)\n", "Meta indicators:\n", " version (object) 1 (1)\n", " carbon price|Avg NPV (2030-2100) (object) 0.9123249940000001, ... 30.1503013256 (7)\n", " year of netzero CO2 emissions (object) nan, 2065, 2078, 2053, 2060.0 (5)\n", " carbon price|2100 (NPV) (object) 0.0426419715, 0.012143570900000001, ... 14.2051784281 (7)\n", " cumulative CO2 emissions (2016 to peak warming, Gt CO2) (object) ... (7)\n", " ..." ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df\n" ] }, { "cell_type": "markdown", "id": "a0eb9b43", "metadata": {}, "source": [ "See the [pyam-IIASA-database tutorial](https://pyam-iamc.readthedocs.io/en/stable/tutorials/iiasa.html)\n", "or the [API documentation](https://pyam-iamc.readthedocs.io/en/stable/api/iiasa.html)\n", "for more information and a complete list of features!" ] }, { "cell_type": "markdown", "id": "isolated-whole", "metadata": {}, "source": [ "## Next steps\n", "\n", "
\n", " \n", "**New to Jupyter notebooks?** \n", "Read [this page](https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/) for helpful tips and tricks when working with Jupyter notebooks.\n", "\n", "
\n", "\n", "### Questions?\n", "\n", "Take a look at the other tutorials to see the scope of **pyam** features - then join our [mailing list](https://groups.io/g/pyam)!\n", "\n", "### Problems?\n", "\n", "If you encounter any functions or methods that don't work as expected,\n", "please check whether there are already any [issues](https://github.com/IAMconsortium/pyam/labels/R) in our GitHub repo. \n", "If not, start a [new one](https://github.com/IAMconsortium/pyam/issues/new)!" ] } ], "metadata": { "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "4.0.2" }, "nbsphinx": { "execute": "never" } }, "nbformat": 4, "nbformat_minor": 5 }