Importing results from a GAMS model to an IamDataFrame

This tutorial illustrates how to extract results from any GAMS model (via the gdx file) and cast them to an IamDataFrame for further processing.

09500dd0219547318c257f042316ba01

The workflow presented here uses the GAMS Python API (read the docs) and the gamstransfer module.

Follow these steps to install the requirements:

  1. Manually install the GAMS API, follow one of the options in this description.

    TL;DR: seach for the folder “api_38” in the GAMS installation folder on your machine and run

    python setup.py install
    
  2. Install the schema package via pip install schema This is a dependency of the gamstransfer module.

This notebook was run with Python 3.8 and GAMS 33.2 on Mac OS.


Developers note: Running this notebook on CI and RTD is a challenge due to the need to download and install the GAMS API.
For the time being, this notebook is not executed by nbsphinx and has to be saved with output.

The transport model

The model used for this illustration is the transport model by Rosenthal, frequently used for tutorials and examples.

The complete model is repeated here for clarity - note the last line which exports the results to a file in the GAMS Data Exchange (gdx) format.

*Basic example of transport model from GAMS model library

$Title  A Transportation Problem (TRNSPORT,SEQ=1)
$Ontext

This problem finds a least cost shipping schedule that meets
requirements at markets and supplies at factories.

Dantzig, G B, Chapter 3.3. In Linear Programming and Extensions.
Princeton University Press, Princeton, New Jersey, 1963.

This formulation is described in detail in:
Rosenthal, R E, Chapter 2: A GAMS Tutorial. In GAMS: A User's Guide.
The Scientific Press, Redwood City, California, 1988.

$Offtext

Sets
     i   canning plants   / seattle, san-diego /
     j   markets          / new-york, chicago, topeka / ;
Parameters
     a(i)  capacity of plant i in cases
       /    seattle     350
            san-diego   600  /
     b(j)  demand at market j in cases
       /    new-york    325
            chicago     300
            topeka      275  / ;
Table d(i,j)  distance in thousands of miles
                  new-york       chicago      topeka
    seattle          2.5           1.7          1.8
    san-diego        2.5           1.8          1.4  ;
Scalar f  freight in dollars per case per thousand miles  /90/ ;
Parameter c(i,j)  transport cost in thousands of dollars per case ;
          c(i,j) = f * d(i,j) / 1000 ;
Variables
     x(i,j)  shipment quantities in cases
     z       total transportation costs in thousands of dollars ;

Positive Variable x ;

Equations
     cost        define objective function
     supply(i)   observe supply limit at plant i
     demand(j)   satisfy demand at market j ;

cost ..        z  =e=  sum((i,j), c(i,j)*x(i,j)) ;

supply(i) ..   sum(j, x(i,j))  =l=  a(i) ;

demand(j) ..   sum(i, x(i,j))  =g=  b(j) ;

Model transport /all/ ;

Solve transport using lp minimizing z ;

Display x.l, x.m ;

Execute_unload 'transport_out.gdx';

Import results from the gdx file

The first cell imports pyam and the two GAMS packages used in this tutorial.

[1]:
from pyam import IamDataFrame
from gams import GamsWorkspace
from gamstransfer import GdxContainer
This cell loads the transport model results gdx file and reads all data into memory.
In a larger application, one could load the data step by step (and only load the data that is actually needed).
[2]:
gdx = GdxContainer(GamsWorkspace().system_directory,'transport_out.gdx')
gdx.rgdx()

Cast results to an IamDataFrame

The first cell in this section defines the model and scenario identifiers, which are identical for all results.

[3]:
args = dict(model='transport', scenario='baseline')
The basic transport model does not have a temporal resolution, but an IamDataFrame requires some information on the time domain.
We therefore assume that the year is 2020 and add it to the results. In a more elaborate model, “time” or “year” would be in the symbol domain.

The objective value

The first cell in this section reads the value of the objective function, adds the year and displays the data.

The next cell casts the data read from the gdx to an IamDataFrame using the variable name cost.
In the following sections, we will add more results from the gdx.

The last cell displays the data of the IamDataFrame.

[4]:
z = gdx.to_dataframe('z')['elements']
z['year'] = 2020
z
[4]:
L year
0 153.675 2020
[5]:
df = IamDataFrame(z, variable='cost', region='USA', unit='$', value='L', **args)
[6]:
df.timeseries()
[6]:
2020
model scenario region variable unit
transport baseline USA cost $ 153.675

Optimal shipment of quantities

The shipments determined by the model can be read from the value of the decision variable x.
The value (GAMS-speak: “level” or field L) is read by default by the GdxContainer.to_dataframe() function.
[7]:
x = gdx.to_dataframe('x')['elements']
x['year'] = 2020
x
[7]:
i j L year
0 seattle new-york 50.0 2020
1 seattle chicago 300.0 2020
2 seattle topeka 0.0 2020
3 san-diego new-york 275.0 2020
4 san-diego chicago 0.0 2020
5 san-diego topeka 275.0 2020

There are several ways to coerce the “from-to” dimension of the mathematical formulation to the “variable/region” format used in the IAMC data standard. In this example, we define a variable supply|* where * is the supply location and we use the demand center as the region.

This is implemented by adding a column ‘type’ to the shipment-dataframe and appending a new object to the results IamDataFrame concatening the ‘type’ column and the orgin column ‘i’.

[8]:
x['type'] = 'supply'
[9]:
df.append(x, variable=['type', 'i'], value='L',
          region='j', unit='cases', **args, inplace=True)

Market prices at the demand centers

The market price can be determined from the marginal value (field: M) of the demand-constraint equations.

[10]:
demand = gdx.to_dataframe('demand', fields='M')['elements']
demand['year'] = 2020
demand
[10]:
j M year
0 new-york 0.225 2020
1 chicago 0.153 2020
2 topeka 0.126 2020
[11]:
df.append(demand, variable='market price', value='M',
          region='j', unit='$/case', **args, inplace=True)
[12]:
df.timeseries()
[12]:
2020
model scenario region variable unit
transport baseline USA cost $ 153.675
chicago market price $/case 0.153
supply|san-diego cases 0.000
supply|seattle cases 300.000
new-york market price $/case 0.225
supply|san-diego cases 275.000
supply|seattle cases 50.000
topeka market price $/case 0.126
supply|san-diego cases 275.000
supply|seattle cases 0.000

Postprocessing to compute aggregate results

It is often practical to not only have results at the most disaggregated level, but to also add aggregated results to the output. The pyam package offers several utilities to perform aggregation or validation; the following cell computes the total supply to each demand city and appends it to the IamDataFrame.

The second cell again displays the complete data to illustrate the aggregation feature.

[13]:
df.aggregate('supply', append=True)
[14]:
df.timeseries()
[14]:
2020
model scenario region variable unit
transport baseline USA cost $ 153.675
chicago market price $/case 0.153
supply cases 300.000
supply|san-diego cases 0.000
supply|seattle cases 300.000
new-york market price $/case 0.225
supply cases 325.000
supply|san-diego cases 275.000
supply|seattle cases 50.000
topeka market price $/case 0.126
supply cases 275.000
supply|san-diego cases 275.000
supply|seattle cases 0.000

Visualization of results

The pyam package includes a powerful plotting and visualization library. Most features are geared for timeseries data (for plotting development over time), but some are useful even for this small, stylized application: the next cell shows the regional share of supply from the two supply locations to the three demand centers.

[15]:
df.filter(variable='supply|*').plot.bar(x='region')
[15]:
<AxesSubplot:title={'center':'model: transport scenario: baseline'}, xlabel='Region', ylabel='cases'>
../_images/tutorials_GAMS_to_pyam_26_1.png

Visit the pyam plotting gallery for more features and options!

Exporting to different file formats

You can use pyam to export the processed data in the IAMC format as an Excel table.

[16]:
df.to_excel('transport.xlsx')

pyam also supports export to different file formats, for example the frictionless data package!

[17]:
df.to_datapackage('transport.zip')
[17]:
<datapackage.package.Package at 0x7fd68e73d970>

Questions? Take a look at this tutorial for first steps with pyam - then join our mailing list!