05 Error Metrics¶

[Estimated execution time: 15 min]

The toolkit allows to evaluate standard error metrics automatically. This notebook will continue the air quality data set used in the previous tutorial 04 Model Training.

In [1]:
import mogptk
import numpy as np
import pandas as pd

Air Quality MOGP¶

For this tutorial we will use the air quality dataset. The data set contains hourly averaged responses from an array of five metal oxide chemical sensors embedded in an Air Quality Chemical Multisensor Device. The device was located in the field in a significantly polluted area in an Italian city. Data were recorded for one year from March 2004 representing the longest freely available recordings of a deployed air quality chemical sensor device.

We will only use five columns: CO(GT), NMHC(GT), C6H6(GT), NOx(GT), NO2(GT). For more information on data loading check out the tutorial 01 Data Loading. For more information on data handling check out the tutorial 02 Data Preparation.

For each sensor the minimum value is -200, which is also the default value when there is an error in the measurements. We will ignore them by converting them to NaN.

In [2]:
df = pd.read_csv('data/AirQualityUCI.csv', delimiter=';')

# Replace missing values with NaN
df.replace(-200.0, np.nan, inplace=True)

# First two columns are date and time
# We convert it to a single column with datetime format
df['Date'] = pd.to_datetime(df['Date'] + ' ' + df['Time'], format='%d/%m/%Y %H.%M.%S')

# Define an initial date to compare all other to it
ini_date = pd.Timestamp('2004-03-10 00:00:00.0')

# Get elapsed hours
df['Time'] = (df['Date'] - ini_date) / pd.Timedelta(hours=1)

# Use only the first eight days of data
df2 = df[df['Date'] < pd.Timestamp('2004-03-19 00:00:00.0')]

Remove aditional data to simulate sensor failure. In this case for each channel we will first remove 50% of the observations and then remove complete sectors in order to get reconstructions from the other channels through learned cross correlations.

We will also use data transformation as defined in the data preparation and transformation tutorial 02 Data Preparation where each channel is normalized so it has zero mean and unit variance.

In [3]:
cols = ['CO(GT)', 'NMHC(GT)', 'C6H6(GT)', 'NOx(GT)', 'NO2(GT)']
dataset = mogptk.LoadDataFrame(df2, x_col='Time', y_col=cols)

for channel in dataset:
    channel.remove_randomly(pct=0.5)

# drop relative ranges to simulate sensor failure
dataset[0].remove_relative_range(0.2, 0.3)
dataset[1].remove_relative_range(0.8, 1.0)
dataset[2].remove_relative_range(0.9, 1.0)
dataset[3].remove_relative_range(0.8, 1.0)
dataset[4].remove_relative_range(0.0, 0.2)

for channel in dataset:
    channel.transform(mogptk.TransformDetrend(degree=1))
    channel.transform(mogptk.TransformStandard())
dataset.plot();

Models¶

We will use one of each of the models in the toolkit, starting with independent Gaussian processes with an SM kernel.

In [4]:
method = 'Adam'
iters = 500
lr = 0.05

Independent channels with spectral mixture kernels¶

For each channel we will use four mixtures.

In [5]:
sm = mogptk.SM(dataset, Q=4)
sm.init_parameters('BNSE')
sm.train(method=method, lr=lr, iters=iters,
         verbose=True, error='MAE', plot=True);
Starting optimization using Adam
‣ Model: SM
‣ Channels: 5
‣ Parameters: 65
‣ Training points: 402
‣ Initial loss: 503.226
‣ Initial error: 33.2931

Start Adam:
    0/500   0:00:00  loss=     503.226  error=     33.2931
    5/500   0:00:00  loss=     497.896  error=     33.9372
   10/500   0:00:00  loss=     489.217  error=     33.0414
   15/500   0:00:00  loss=     482.743  error=     33.5371
   20/500   0:00:00  loss=      476.72  error=     32.7491
   25/500   0:00:01  loss=     471.127  error=     33.3041
   30/500   0:00:01  loss=     465.836  error=     32.6956
   35/500   0:00:01  loss=     460.844  error=     32.8632
   40/500   0:00:01  loss=     456.154  error=     32.8857
   45/500   0:00:01  loss=     451.728  error=      32.892
   50/500   0:00:02  loss=     447.548  error=     32.6396
   55/500   0:00:02  loss=     443.593  error=     32.7133
   60/500   0:00:02  loss=     439.867  error=     32.6249
   65/500   0:00:02  loss=     436.374  error=     32.5903
   70/500   0:00:03  loss=     433.106  error=     32.6223
   75/500   0:00:03  loss=     430.056  error=     32.5878
   80/500   0:00:03  loss=     427.222  error=     32.5342
   85/500   0:00:04  loss=     424.593  error=     32.5085
   90/500   0:00:04  loss=     422.161  error=     32.5081
   95/500   0:00:04  loss=     419.917  error=     32.4625
  100/500   0:00:04  loss=     417.983  error=     32.5256
  105/500   0:00:05  loss=     416.003  error=     32.4347
  110/500   0:00:05  loss=     414.251  error=     32.3536
  115/500   0:00:05  loss=     412.657  error=     32.3897
  120/500   0:00:05  loss=     411.209  error=     32.3412
  125/500   0:00:06  loss=     409.899  error=     32.3658
  130/500   0:00:06  loss=      408.72  error=     32.2846
  135/500   0:00:06  loss=     407.654  error=     32.3208
  140/500   0:00:06  loss=     406.692  error=     32.2682
  145/500   0:00:07  loss=     405.829  error=     32.2485
  150/500   0:00:07  loss=     405.057  error=      32.247
  155/500   0:00:07  loss=     404.368  error=     32.2387
  160/500   0:00:07  loss=      403.75  error=     32.2123
  165/500   0:00:08  loss=       403.2  error=     32.1803
  170/500   0:00:08  loss=     402.901  error=     32.1957
  175/500   0:00:08  loss=      402.42  error=     32.1256
  180/500   0:00:08  loss=       401.9  error=     32.1056
  185/500   0:00:09  loss=     401.544  error=     32.0876
  190/500   0:00:09  loss=     401.234  error=     32.1236
  195/500   0:00:09  loss=     400.936  error=     32.0909
  200/500   0:00:09  loss=      400.68  error=     32.0324
  205/500   0:00:10  loss=     400.459  error=     32.0969
  210/500   0:00:10  loss=     400.277  error=      32.072
  215/500   0:00:10  loss=      400.04  error=     31.9964
  220/500   0:00:10  loss=     399.919  error=     32.0881
  225/500   0:00:11  loss=     399.694  error=     32.0356
  230/500   0:00:11  loss=     399.483  error=      31.975
  235/500   0:00:11  loss=     399.334  error=     31.9825
  240/500   0:00:11  loss=     399.606  error=     32.0317
  245/500   0:00:11  loss=     399.143  error=     31.9178
  250/500   0:00:12  loss=     398.898  error=     31.9293
  255/500   0:00:12  loss=     398.805  error=     32.0061
  260/500   0:00:12  loss=     398.661  error=     31.9302
  265/500   0:00:12  loss=     398.513  error=     31.9132
  270/500   0:00:13  loss=      398.57  error=     31.9563
  275/500   0:00:13  loss=     398.345  error=     31.8706
  280/500   0:00:13  loss=     398.178  error=     31.8736
  285/500   0:00:13  loss=     398.057  error=     31.8946
  290/500   0:00:14  loss=     397.935  error=     31.8739
  295/500   0:00:14  loss=     397.814  error=      31.782
  300/500   0:00:14  loss=     397.733  error=     31.8633
  305/500   0:00:14  loss=     397.584  error=     31.8412
  310/500   0:00:15  loss=     397.479  error=     31.8405
  315/500   0:00:15  loss=     397.378  error=       31.84
  320/500   0:00:15  loss=     397.337  error=     31.8265
  325/500   0:00:15  loss=     397.207  error=     31.8603
  330/500   0:00:16  loss=     397.316  error=     31.6685
  335/500   0:00:16  loss=     397.002  error=     31.7665
  340/500   0:00:16  loss=     396.865  error=     31.7645
  345/500   0:00:16  loss=     396.783  error=     31.7222
  350/500   0:00:17  loss=     396.692  error=     31.7742
  355/500   0:00:17  loss=     396.671  error=     31.6457
  360/500   0:00:17  loss=     396.547  error=     31.7301
  365/500   0:00:17  loss=     396.403  error=      31.699
  370/500   0:00:18  loss=     396.284  error=     31.7633
  375/500   0:00:18  loss=     396.223  error=      31.778
  380/500   0:00:18  loss=     396.144  error=       31.79
  385/500   0:00:19  loss=     396.023  error=     31.6705
  390/500   0:00:19  loss=      395.98  error=     31.7925
  395/500   0:00:19  loss=     396.179  error=     32.0185
  400/500   0:00:19  loss=     395.874  error=     31.9131
  405/500   0:00:20  loss=     395.726  error=     31.8448
  410/500   0:00:20  loss=     395.577  error=     31.8701
  415/500   0:00:20  loss=     395.511  error=     31.8069
  420/500   0:00:21  loss=     395.479  error=     31.8941
  425/500   0:00:21  loss=     395.543  error=     31.5627
  430/500   0:00:21  loss=     395.235  error=     31.7371
  435/500   0:00:21  loss=     395.114  error=     31.8124
  440/500   0:00:21  loss=     395.045  error=     31.8659
  445/500   0:00:22  loss=      394.96  error=     31.8842
  450/500   0:00:22  loss=     394.871  error=     31.8631
  455/500   0:00:22  loss=     394.765  error=     31.7916
  460/500   0:00:23  loss=     394.698  error=     31.7108
  465/500   0:00:23  loss=     394.971  error=     31.7429
  470/500   0:00:23  loss=     394.641  error=      31.795
  475/500   0:00:23  loss=     394.584  error=     31.6968
  480/500   0:00:24  loss=     394.473  error=     31.8388
  485/500   0:00:24  loss=     394.457  error=       31.59
  490/500   0:00:24  loss=     394.352  error=      31.561
  495/500   0:00:24  loss=      394.21  error=     31.8005
  500/500   0:00:25  loss=     394.102  error=     31.8406
Finished

Optimization finished in 25.234 seconds
‣ Iterations: 500
‣ Final loss: 394.102
‣ Final error: 31.8406

Evaluating errors¶

Using the mogptk.error function given a test input and output it calculates:

  • Mean absolute error (MAE)
  • Mean absolute percentage error (MAPE)
  • Root mean squared error (RMSE)

If only the raw values $y_{true} - y_{pred}$ are desired, the flag simple must be passed.

Multiple models for the same test set (X, Y) can be passed at once, the result will be a list with one element for each model passed, where each element is another list of length equal to the number or channels where each element is the error for said model and channel. This enables to obtain errors for multiple models for the same test set where each channel can have different numbers of test points.

In [6]:
sm.plot_prediction(title='SM on Air Quality Data');
In [7]:
mogptk.error(sm, disp=True, per_channel=False);
MAE MAPE RMSE
Name
SM 31.840617 47.75155 58.443636

Multi Output Spectral Mixture (MOSM)¶

Next we use the multi output spectral mixture kernel (Parra et al, 2016).

In [8]:
mosm = mogptk.MOSM(dataset, Q=4)
mosm.init_parameters('BNSE')
mosm.train(method=method, lr=lr, iters=iters,
           verbose=True, error='MAE', plot=True)
mosm.plot_prediction(title='MOSM on Air Quality Data');
Starting optimization using Adam
‣ Model: MOSM
‣ Channels: 5
‣ Parameters: 105
‣ Training points: 402
‣ Initial loss: 538.303
‣ Initial error: 42.0905

Start Adam:
    0/500   0:00:00  loss=     538.303  error=     42.0905
    5/500   0:00:00  loss=     546.787  error=     44.5344
   10/500   0:00:00  loss=     540.718  error=     43.6076
   15/500   0:00:01  loss=     534.985  error=     43.1139
   20/500   0:00:01  loss=      531.09  error=     42.8474
   25/500   0:00:02  loss=      526.55  error=      42.187
   30/500   0:00:02  loss=     523.114  error=     41.9008
   35/500   0:00:03  loss=     520.795  error=     41.7139
   40/500   0:00:03  loss=     517.502  error=     41.3093
   45/500   0:00:04  loss=     514.388  error=     41.0597
   50/500   0:00:05  loss=     511.372  error=     40.7386
   55/500   0:00:05  loss=     508.364  error=     40.4829
   60/500   0:00:06  loss=      504.94  error=     40.2014
   65/500   0:00:06  loss=     501.645  error=     39.9121
   70/500   0:00:07  loss=     498.452  error=     39.6749
   75/500   0:00:08  loss=     494.789  error=     39.4123
   80/500   0:00:08  loss=     491.358  error=     39.1528
   85/500   0:00:09  loss=     487.642  error=     38.9089
   90/500   0:00:10  loss=     484.072  error=     38.7074
   95/500   0:00:11  loss=     480.332  error=     38.4734
  100/500   0:00:12  loss=      476.73  error=     38.2615
  105/500   0:00:13  loss=     473.164  error=     38.1123
  110/500   0:00:13  loss=     469.698  error=     37.9166
  115/500   0:00:14  loss=     466.379  error=     37.7805
  120/500   0:00:15  loss=     463.516  error=     37.7519
  125/500   0:00:16  loss=      460.28  error=     37.5855
  130/500   0:00:17  loss=      457.34  error=     37.4908
  135/500   0:00:17  loss=     454.585  error=     37.4072
  140/500   0:00:18  loss=     452.003  error=       37.32
  145/500   0:00:19  loss=     449.619  error=     37.2138
  150/500   0:00:20  loss=     447.644  error=     37.2355
  155/500   0:00:21  loss=     445.339  error=     37.1561
  160/500   0:00:21  loss=     443.414  error=     37.1142
  165/500   0:00:22  loss=     441.683  error=     37.0765
  170/500   0:00:23  loss=     439.972  error=      37.066
  175/500   0:00:24  loss=      438.45  error=     37.0508
  180/500   0:00:24  loss=     437.189  error=     37.0065
  185/500   0:00:25  loss=     435.684  error=     37.0052
  190/500   0:00:26  loss=      434.47  error=     37.0528
  195/500   0:00:27  loss=     433.592  error=     37.0057
  200/500   0:00:28  loss=     432.205  error=     36.9967
  205/500   0:00:29  loss=     431.362  error=     36.9996
  210/500   0:00:30  loss=     430.342  error=     37.0004
  215/500   0:00:31  loss=     429.532  error=     36.9872
  220/500   0:00:31  loss=      428.85  error=      36.985
  225/500   0:00:32  loss=     428.005  error=     36.9951
  230/500   0:00:33  loss=     427.383  error=     36.9939
  235/500   0:00:34  loss=      426.61  error=     37.0344
  240/500   0:00:35  loss=     426.427  error=     37.0018
  245/500   0:00:36  loss=     425.396  error=     36.9854
  250/500   0:00:37  loss=     424.878  error=     36.9649
  255/500   0:00:37  loss=     424.588  error=     37.0682
  260/500   0:00:38  loss=     424.192  error=     36.9702
  265/500   0:00:38  loss=     423.294  error=     36.9399
  270/500   0:00:39  loss=     423.203  error=      37.046
  275/500   0:00:40  loss=      423.07  error=     37.0962
  280/500   0:00:40  loss=     422.045  error=     36.9919
  285/500   0:00:41  loss=     421.647  error=      36.983
  290/500   0:00:41  loss=     421.352  error=      37.006
  295/500   0:00:42  loss=      421.05  error=     36.9945
  300/500   0:00:43  loss=     420.935  error=     36.9453
  305/500   0:00:44  loss=     420.095  error=      36.908
  310/500   0:00:45  loss=     420.057  error=     37.0039
  315/500   0:00:45  loss=     420.381  error=      37.172
  320/500   0:00:46  loss=     419.304  error=     37.0697
  325/500   0:00:47  loss=     419.062  error=     37.0237
  330/500   0:00:48  loss=     418.795  error=     37.0373
  335/500   0:00:48  loss=     418.543  error=     37.0972
  340/500   0:00:49  loss=     418.262  error=     37.0616
  345/500   0:00:50  loss=     418.088  error=     37.0067
  350/500   0:00:51  loss=     417.784  error=      37.052
  355/500   0:00:52  loss=     417.329  error=     37.0571
  360/500   0:00:53  loss=     417.962  error=      37.248
  365/500   0:00:55  loss=     417.751  error=      37.148
  370/500   0:00:56  loss=     416.983  error=     37.0826
  375/500   0:00:57  loss=     416.665  error=     37.0765
  380/500   0:00:57  loss=     416.702  error=     37.0803
  385/500   0:00:58  loss=     416.452  error=     37.0439
  390/500   0:01:00  loss=     416.059  error=     37.0657
  395/500   0:01:01  loss=     416.317  error=     37.3335
  400/500   0:01:02  loss=     416.149  error=     37.2948
  405/500   0:01:03  loss=     416.153  error=     37.2517
  410/500   0:01:04  loss=     415.959  error=     37.2569
  415/500   0:01:05  loss=     416.404  error=     37.2878
  420/500   0:01:06  loss=     416.618  error=     37.3746
  425/500   0:01:07  loss=     418.033  error=      37.078
  430/500   0:01:07  loss=     416.969  error=     37.4217
  435/500   0:01:08  loss=     417.138  error=     37.2567
  440/500   0:01:09  loss=     418.292  error=      37.559
  445/500   0:01:10  loss=     416.931  error=     37.1602
  450/500   0:01:11  loss=     415.315  error=     37.4222
  455/500   0:01:12  loss=     415.624  error=     37.5189
  460/500   0:01:13  loss=     415.115  error=     37.2232
  465/500   0:01:13  loss=     415.208  error=     37.1509
  470/500   0:01:14  loss=     415.071  error=     37.3978
  475/500   0:01:15  loss=      414.42  error=      37.149
  480/500   0:01:16  loss=     414.266  error=     37.3492
  485/500   0:01:17  loss=     414.003  error=     37.1281
  490/500   0:01:18  loss=     413.718  error=     37.2468
  495/500   0:01:18  loss=     413.784  error=     37.3052
  500/500   0:01:19  loss=      413.58  error=     37.1608
Finished

Optimization finished in 1 minute 19 seconds
‣ Iterations: 500
‣ Final loss: 413.58
‣ Final error: 37.1608

Cross Spectral Mixture (CSM)¶

Then we use the cross spectral mixture kernel (Ulrich et al, 2015).

In [9]:
csm = mogptk.CSM(dataset, Q=4)
csm.init_parameters()
csm.train(method=method, lr=lr, iters=iters,
          verbose=True, error='MAE', plot=True)
csm.plot_prediction(title='CSM on Air Quality Data');
Starting optimization using Adam
‣ Model: CSM
‣ Channels: 5
‣ Parameters: 53
‣ Training points: 402
‣ Initial loss: 473.578
‣ Initial error: 28.4435

Start Adam:
    0/500   0:00:00  loss=     473.578  error=     28.4435
    5/500   0:00:02  loss=     462.267  error=     28.5251
   10/500   0:00:04  loss=     452.686  error=     29.2235
   15/500   0:00:06  loss=     447.663  error=      29.157
   20/500   0:00:09  loss=      441.75  error=     28.4208
   25/500   0:00:11  loss=     435.853  error=     28.9513
   30/500   0:00:13  loss=     430.834  error=     28.8211
   35/500   0:00:15  loss=     425.952  error=     28.7363
   40/500   0:00:17  loss=     421.358  error=     28.6236
   45/500   0:00:19  loss=     417.136  error=      28.743
   50/500   0:00:21  loss=     413.228  error=     28.7319
   55/500   0:00:23  loss=     409.583  error=      28.707
   60/500   0:00:25  loss=     406.229  error=      28.588
   65/500   0:00:27  loss=     403.133  error=     28.5921
   70/500   0:00:29  loss=     400.274  error=     28.5655
   75/500   0:00:31  loss=     397.653  error=     28.5313
   80/500   0:00:33  loss=     395.247  error=       28.52
   85/500   0:00:36  loss=     393.048  error=     28.4893
   90/500   0:00:38  loss=     391.037  error=     28.4821
   95/500   0:00:40  loss=     389.204  error=     28.4498
  100/500   0:00:42  loss=     387.532  error=     28.4391
  105/500   0:00:44  loss=     386.011  error=     28.4216
  110/500   0:00:46  loss=     384.627  error=     28.4105
  115/500   0:00:48  loss=     383.369  error=     28.3959
  120/500   0:00:50  loss=     382.225  error=     28.3788
  125/500   0:00:52  loss=     381.186  error=     28.3663
  130/500   0:00:53  loss=     380.242  error=     28.3581
  135/500   0:00:55  loss=     379.383  error=     28.3486
  140/500   0:00:56  loss=     378.601  error=     28.3382
  145/500   0:00:57  loss=     377.889  error=     28.3314
  150/500   0:00:58  loss=      377.24  error=     28.3243
  155/500   0:00:59  loss=     376.651  error=     28.3295
  160/500   0:01:00  loss=     376.165  error=      28.267
  165/500   0:01:01  loss=     375.681  error=     28.3391
  170/500   0:01:02  loss=     375.153  error=     28.2708
  175/500   0:01:03  loss=      374.73  error=     28.2717
  180/500   0:01:04  loss=     374.344  error=     28.2795
  185/500   0:01:05  loss=     373.967  error=     28.2587
  190/500   0:01:06  loss=      373.68  error=     28.2476
  195/500   0:01:07  loss=     373.423  error=     28.2499
  200/500   0:01:08  loss=     373.104  error=     28.2445
  205/500   0:01:09  loss=     372.791  error=     28.2271
  210/500   0:01:10  loss=     372.504  error=      28.227
  215/500   0:01:11  loss=     372.235  error=     28.2284
  220/500   0:01:12  loss=     371.987  error=     28.2253
  225/500   0:01:13  loss=     371.772  error=     28.2254
  230/500   0:01:15  loss=     371.564  error=     28.2211
  235/500   0:01:16  loss=     371.374  error=     28.2211
  240/500   0:01:17  loss=     371.195  error=     28.2194
  245/500   0:01:18  loss=     371.089  error=     28.2226
  250/500   0:01:19  loss=     371.196  error=     28.1996
  255/500   0:01:20  loss=     370.782  error=     28.2092
  260/500   0:01:22  loss=     370.565  error=     28.2283
  265/500   0:01:23  loss=     370.462  error=     28.2416
  270/500   0:01:24  loss=      370.34  error=     28.2376
  275/500   0:01:25  loss=     370.185  error=     28.2165
  280/500   0:01:27  loss=     370.101  error=     28.2035
  285/500   0:01:28  loss=     369.986  error=     28.2254
  290/500   0:01:29  loss=     369.887  error=     28.2163
  295/500   0:01:30  loss=      369.89  error=     28.2491
  300/500   0:01:31  loss=     370.359  error=     28.1823
  305/500   0:01:32  loss=     369.907  error=     28.1886
  310/500   0:01:33  loss=     369.712  error=     28.2032
  315/500   0:01:34  loss=     369.589  error=     28.2144
  320/500   0:01:36  loss=     369.486  error=     28.2176
  325/500   0:01:37  loss=     369.398  error=     28.2244
  330/500   0:01:39  loss=      369.32  error=     28.2307
  335/500   0:01:40  loss=     369.255  error=     28.2344
  340/500   0:01:42  loss=     369.212  error=     28.2402
  345/500   0:01:43  loss=     369.163  error=     28.2419
  350/500   0:01:44  loss=     369.113  error=     28.2347
  355/500   0:01:45  loss=     369.066  error=     28.2349
  360/500   0:01:46  loss=     369.034  error=     28.2408
  365/500   0:01:47  loss=     369.104  error=     28.2177
  370/500   0:01:48  loss=     369.805  error=      28.304
  375/500   0:01:49  loss=     369.291  error=     28.2803
  380/500   0:01:51  loss=     369.098  error=     28.2817
  385/500   0:01:53  loss=     368.984  error=     28.2697
  390/500   0:01:54  loss=     368.898  error=      28.269
  395/500   0:01:56  loss=     368.834  error=     28.2649
  400/500   0:01:57  loss=     368.787  error=     28.2561
  405/500   0:01:59  loss=     368.753  error=     28.2602
  410/500   0:02:01  loss=     368.723  error=     28.2559
  415/500   0:02:03  loss=      368.69  error=     28.2565
  420/500   0:02:05  loss=      368.66  error=     28.2544
  425/500   0:02:08  loss=     368.634  error=     28.2523
  430/500   0:02:10  loss=      368.61  error=      28.254
  435/500   0:02:12  loss=     368.589  error=     28.2507
  440/500   0:02:14  loss=     368.569  error=     28.2518
  445/500   0:02:16  loss=     368.568  error=     28.2608
  450/500   0:02:18  loss=      368.83  error=     28.2305
  455/500   0:02:20  loss=     368.925  error=     28.2834
  460/500   0:02:23  loss=      368.81  error=     28.2869
  465/500   0:02:25  loss=     368.657  error=     28.2848
  470/500   0:02:27  loss=     368.557  error=     28.2857
  475/500   0:02:29  loss=     368.494  error=     28.2827
  480/500   0:02:32  loss=     368.452  error=      28.277
  485/500   0:02:34  loss=     368.416  error=     28.2743
  490/500   0:02:37  loss=     368.387  error=     28.2729
  495/500   0:02:39  loss=      368.36  error=     28.2691
  500/500   0:02:42  loss=     368.342  error=     28.2634
Finished

Optimization finished in 2 minutes 42 seconds
‣ Iterations: 500
‣ Final loss: 368.342
‣ Final error: 28.2634

Spectral Mixture - Linear Model of Corregionalization (SM-LMC)¶

Lastly we fit the spectral mixture linear model of corregionalization (Wilson, 2014).

In [10]:
smlmc = mogptk.SM_LMC(dataset, Q=4)
smlmc.init_parameters()
smlmc.train(method=method, lr=lr, iters=iters,
            verbose=True, error='MAE', plot=True)
smlmc.plot_prediction(title='SM-LMC on Air Quality Data');
Starting optimization using Adam
‣ Model: SM-LMC
‣ Channels: 5
‣ Parameters: 33
‣ Training points: 402
‣ Initial loss: 471.894
‣ Initial error: 27.767

Start Adam:
    0/500   0:00:00  loss=     471.894  error=      27.767
    5/500   0:00:00  loss=     463.752  error=     27.9928
   10/500   0:00:01  loss=     456.346  error=     28.0933
   15/500   0:00:01  loss=     449.342  error=     28.1606
   20/500   0:00:02  loss=     443.058  error=     28.5212
   25/500   0:00:02  loss=     437.238  error=     28.4841
   30/500   0:00:03  loss=     431.349  error=     28.2738
   35/500   0:00:03  loss=     426.032  error=     28.3295
   40/500   0:00:04  loss=     420.709  error=     28.3723
   45/500   0:00:05  loss=     415.791  error=     28.3594
   50/500   0:00:05  loss=      411.02  error=      28.368
   55/500   0:00:06  loss=     406.518  error=     28.3358
   60/500   0:00:06  loss=     402.237  error=     28.3715
   65/500   0:00:07  loss=     398.174  error=     28.3898
   70/500   0:00:08  loss=     394.216  error=     28.4864
   75/500   0:00:08  loss=     390.261  error=     28.5489
   80/500   0:00:09  loss=     386.495  error=     28.4288
   85/500   0:00:09  loss=     383.012  error=     28.3982
   90/500   0:00:10  loss=      379.82  error=     28.3663
   95/500   0:00:10  loss=     376.864  error=     28.3116
  100/500   0:00:11  loss=     374.101  error=     28.3491
  105/500   0:00:11  loss=     371.581  error=     28.3249
  110/500   0:00:12  loss=     369.296  error=      28.262
  115/500   0:00:13  loss=      367.21  error=      28.245
  120/500   0:00:13  loss=      365.33  error=     28.2108
  125/500   0:00:14  loss=     363.633  error=     28.1621
  130/500   0:00:15  loss=     362.109  error=      28.124
  135/500   0:00:15  loss=     360.738  error=     28.0959
  140/500   0:00:16  loss=     359.509  error=     28.0657
  145/500   0:00:16  loss=     358.406  error=     28.0306
  150/500   0:00:17  loss=     357.414  error=     27.9978
  155/500   0:00:17  loss=     356.519  error=     27.9688
  160/500   0:00:18  loss=     355.708  error=     27.9401
  165/500   0:00:18  loss=     354.969  error=     27.9086
  170/500   0:00:19  loss=     354.291  error=     27.8743
  175/500   0:00:20  loss=     353.662  error=     27.8369
  180/500   0:00:20  loss=     353.074  error=     27.7965
  185/500   0:00:21  loss=     352.519  error=     27.7524
  190/500   0:00:21  loss=     351.992  error=     27.7064
  195/500   0:00:22  loss=     351.487  error=       27.66
  200/500   0:00:23  loss=     351.005  error=      27.616
  205/500   0:00:23  loss=      350.55  error=     27.5489
  210/500   0:00:24  loss=      350.06  error=     27.5107
  215/500   0:00:25  loss=     349.543  error=     27.4326
  220/500   0:00:25  loss=     349.014  error=     27.3409
  225/500   0:00:26  loss=      348.39  error=     27.2457
  230/500   0:00:26  loss=     347.655  error=     27.0773
  235/500   0:00:27  loss=     346.856  error=      26.923
  240/500   0:00:28  loss=      346.05  error=     26.7454
  245/500   0:00:29  loss=      345.28  error=     26.6056
  250/500   0:00:29  loss=     344.571  error=     26.4746
  255/500   0:00:30  loss=     343.935  error=      26.365
  260/500   0:00:30  loss=     343.374  error=     26.2718
  265/500   0:00:31  loss=     342.878  error=     26.1882
  270/500   0:00:32  loss=     342.441  error=     26.1184
  275/500   0:00:32  loss=     342.052  error=     26.0604
  280/500   0:00:33  loss=     341.703  error=     26.0114
  285/500   0:00:34  loss=     341.384  error=     25.9699
  290/500   0:00:34  loss=      341.09  error=     25.9345
  295/500   0:00:35  loss=     340.815  error=     25.9045
  300/500   0:00:36  loss=     340.553  error=     25.8789
  305/500   0:00:36  loss=       340.3  error=     25.8565
  310/500   0:00:37  loss=     340.054  error=     25.8379
  315/500   0:00:37  loss=      339.81  error=     25.8249
  320/500   0:00:38  loss=     339.566  error=       25.81
  325/500   0:00:38  loss=     339.396  error=      25.824
  330/500   0:00:39  loss=     339.071  error=      25.813
  335/500   0:00:40  loss=     338.873  error=     25.8221
  340/500   0:00:40  loss=     338.502  error=     25.8026
  345/500   0:00:41  loss=     338.235  error=     25.7953
  350/500   0:00:41  loss=     337.914  error=     25.8093
  355/500   0:00:42  loss=     337.616  error=     25.8053
  360/500   0:00:42  loss=     337.327  error=     25.7907
  365/500   0:00:43  loss=     337.051  error=     25.7849
  370/500   0:00:43  loss=     336.794  error=     25.7821
  375/500   0:00:44  loss=     336.558  error=     25.7796
  380/500   0:00:45  loss=     336.337  error=     25.7734
  385/500   0:00:45  loss=     336.135  error=     25.7671
  390/500   0:00:46  loss=     335.947  error=     25.7605
  395/500   0:00:46  loss=     335.769  error=     25.7536
  400/500   0:00:47  loss=     335.601  error=     25.7457
  405/500   0:00:48  loss=     335.439  error=     25.7358
  410/500   0:00:48  loss=     335.281  error=     25.7232
  415/500   0:00:49  loss=     335.124  error=     25.7077
  420/500   0:00:49  loss=     334.965  error=     25.6877
  425/500   0:00:50  loss=       334.8  error=     25.6606
  430/500   0:00:51  loss=     334.618  error=     25.6211
  435/500   0:00:51  loss=     334.403  error=     25.5548
  440/500   0:00:52  loss=     334.058  error=     25.3722
  445/500   0:00:53  loss=     331.974  error=     24.1401
  450/500   0:00:54  loss=     323.693  error=      23.621
  455/500   0:00:55  loss=     321.113  error=     23.2902
  460/500   0:00:56  loss=     317.299  error=     23.0304
  465/500   0:00:57  loss=     314.528  error=      22.718
  470/500   0:00:58  loss=     312.673  error=     22.4864
  475/500   0:00:59  loss=     311.368  error=     22.3612
  480/500   0:01:00  loss=     310.393  error=     22.2992
  485/500   0:01:01  loss=     309.647  error=     22.2548
  490/500   0:01:01  loss=     309.041  error=     22.2114
  495/500   0:01:02  loss=     308.531  error=     22.1914
  500/500   0:01:02  loss=     308.087  error=     22.1675
Finished

Optimization finished in 1 minute 2 seconds
‣ Iterations: 500
‣ Final loss: 308.087
‣ Final error: 22.1675

Convolutional Gaussian (CONV)¶

In [11]:
conv = mogptk.CONV(dataset, Q=4)
conv.init_parameters()
conv.train(method=method, lr=lr, iters=iters,
           verbose=True, error='MAE', plot=True)
conv.plot_prediction(title='CONV on Air Quality Data');
Starting optimization using Adam
‣ Model: CONV
‣ Channels: 5
‣ Parameters: 49
‣ Training points: 402
‣ Initial loss: 514.873
‣ Initial error: 31.721

Start Adam:
    0/500   0:00:00  loss=     514.873  error=      31.721
    5/500   0:00:01  loss=      508.99  error=     30.9761
   10/500   0:00:03  loss=     503.065  error=     30.2364
   15/500   0:00:05  loss=     497.097  error=      29.502
   20/500   0:00:07  loss=     491.082  error=     28.7707
   25/500   0:00:10  loss=     485.019  error=       28.04
   30/500   0:00:12  loss=     478.906  error=     27.3331
   35/500   0:00:14  loss=      472.74  error=     26.6875
   40/500   0:00:16  loss=     466.519  error=     26.0872
   45/500   0:00:17  loss=     460.243  error=     25.5193
   50/500   0:00:19  loss=     453.912  error=     24.9642
   55/500   0:00:20  loss=     447.532  error=     24.4283
   60/500   0:00:23  loss=     441.108  error=      23.922
   65/500   0:00:25  loss=     434.647  error=     23.4217
   70/500   0:00:27  loss=      428.16  error=     22.9263
   75/500   0:00:29  loss=     421.658  error=     22.4357
   80/500   0:00:32  loss=     415.158  error=     21.9642
   85/500   0:00:34  loss=     408.677  error=     21.5079
   90/500   0:00:35  loss=     402.234  error=     21.0648
   95/500   0:00:36  loss=      395.85  error=     20.6434
  100/500   0:00:38  loss=     389.545  error=     20.2558
  105/500   0:00:39  loss=     383.337  error=     19.8972
  110/500   0:00:40  loss=     377.246  error=     19.5717
  115/500   0:00:42  loss=     371.288  error=     19.2781
  120/500   0:00:44  loss=     365.479  error=     19.0106
  125/500   0:00:46  loss=      359.83  error=     18.7768
  130/500   0:00:47  loss=     354.351  error=      18.571
  135/500   0:00:49  loss=     349.047  error=     18.3804
  140/500   0:00:50  loss=     343.923  error=     18.2142
  145/500   0:00:52  loss=     338.978  error=      18.064
  150/500   0:00:54  loss=     334.207  error=     17.9298
  155/500   0:00:57  loss=     329.602  error=      17.804
  160/500   0:00:59  loss=     325.143  error=      17.684
  165/500   0:01:01  loss=     320.798  error=     17.5579
  170/500   0:01:04  loss=     316.521  error=     17.4178
  175/500   0:01:06  loss=     312.262  error=       17.25
  180/500   0:01:07  loss=     307.993  error=     17.0495
  185/500   0:01:09  loss=     303.725  error=     16.8221
  190/500   0:01:11  loss=     299.501  error=     16.5971
  195/500   0:01:12  loss=     295.383  error=     16.4077
  200/500   0:01:14  loss=      291.42  error=     16.2511
  205/500   0:01:19  loss=     287.642  error=     16.1157
  210/500   0:01:24  loss=      284.05  error=     15.9991
  215/500   0:01:30  loss=     280.626  error=     15.9052
  220/500   0:01:34  loss=     277.347  error=     15.8252
  225/500   0:01:37  loss=     274.205  error=     15.7565
  230/500   0:01:39  loss=     271.203  error=     15.6895
  235/500   0:01:42  loss=     268.346  error=     15.6238
  240/500   0:01:44  loss=     265.635  error=     15.5609
  245/500   0:01:46  loss=     263.066  error=     15.5009
  250/500   0:01:47  loss=     260.631  error=     15.4437
  255/500   0:01:48  loss=     258.325  error=     15.3906
  260/500   0:01:50  loss=     256.142  error=     15.3388
  265/500   0:01:51  loss=     254.075  error=       15.29
  270/500   0:01:52  loss=     252.115  error=     15.2524
  275/500   0:01:53  loss=     250.255  error=     15.2156
  280/500   0:01:55  loss=     248.484  error=     15.1785
  285/500   0:01:56  loss=     246.793  error=     15.1411
  290/500   0:01:57  loss=     245.172  error=     15.1031
  295/500   0:01:58  loss=     243.611  error=     15.0645
  300/500   0:02:00  loss=     242.099  error=     15.0229
  305/500   0:02:01  loss=     240.627  error=     14.9783
  310/500   0:02:02  loss=     239.185  error=     14.9295
  315/500   0:02:04  loss=     237.764  error=     14.8755
  320/500   0:02:05  loss=     236.358  error=     14.8168
  325/500   0:02:06  loss=     234.961  error=     14.7552
  330/500   0:02:08  loss=      233.57  error=     14.6913
  335/500   0:02:09  loss=     232.186  error=     14.6225
  340/500   0:02:11  loss=     230.809  error=     14.5496
  345/500   0:02:12  loss=     229.445  error=     14.4756
  350/500   0:02:13  loss=     228.096  error=     14.3995
  355/500   0:02:15  loss=      226.77  error=     14.3221
  360/500   0:02:16  loss=      225.47  error=     14.2464
  365/500   0:02:17  loss=     224.199  error=     14.1753
  370/500   0:02:19  loss=     222.959  error=     14.1111
  375/500   0:02:20  loss=      221.75  error=      14.052
  380/500   0:02:21  loss=      220.57  error=     13.9998
  385/500   0:02:23  loss=     219.421  error=     13.9595
  390/500   0:02:24  loss=       218.3  error=     13.9282
  395/500   0:02:25  loss=     217.208  error=        13.9
  400/500   0:02:27  loss=     216.145  error=     13.8727
  405/500   0:02:28  loss=      215.11  error=     13.8448
  410/500   0:02:29  loss=     214.101  error=     13.8151
  415/500   0:02:30  loss=     213.117  error=     13.7857
  420/500   0:02:32  loss=     212.158  error=     13.7564
  425/500   0:02:33  loss=     211.223  error=     13.7277
  430/500   0:02:34  loss=      210.31  error=      13.701
  435/500   0:02:35  loss=     209.419  error=     13.6745
  440/500   0:02:36  loss=     208.549  error=     13.6479
  445/500   0:02:38  loss=     207.699  error=      13.621
  450/500   0:02:39  loss=     206.869  error=     13.5935
  455/500   0:02:40  loss=     206.059  error=     13.5653
  460/500   0:02:42  loss=     205.267  error=     13.5362
  465/500   0:02:43  loss=     204.494  error=     13.5063
  470/500   0:02:44  loss=      203.74  error=     13.4757
  475/500   0:02:46  loss=     203.004  error=     13.4447
  480/500   0:02:47  loss=     202.288  error=     13.4134
  485/500   0:02:48  loss=     201.591  error=     13.3824
  490/500   0:02:49  loss=     200.913  error=     13.3521
  495/500   0:02:51  loss=     200.256  error=     13.3227
  500/500   0:02:52  loss=     199.619  error=     13.2947
Finished

Optimization finished in 2 minutes 52 seconds
‣ Iterations: 500
‣ Final loss: 199.619
‣ Final error: 13.2947

Compare errors¶

We will take the mean MAE, MAPE, and RMSE for all the channels and compare all the models.

In [12]:
mogptk.error(sm, mosm, csm, smlmc, conv, disp=True)
MAE MAPE RMSE
Name
SM 31.840617 47.751550 58.443636
MOSM 37.160805 65.353113 71.388803
CSM 28.263449 42.516146 53.322366
SM-LMC 22.167511 30.401371 43.084923
CONV 13.294705 18.211582 25.173724