Currency exchange example¶

Dataset used is the currency enchange with respect to USD dollars, from daily measurements from the last 2 years (2017-2018).

Currencies used:

  • Reference: USA
  • Canadian dollar (CAD)
  • Euro (EUR)
  • Japanese yen (JPY)
  • Great british pound (GBP)
  • Swiss franc (CHF)
  • Austrial dollar (AUD)
  • Hong Kong dollar (HKD)
  • New zealand dollar (NZD)
  • South korean won (KRW)
  • Mexican peso (MXN)

Please see our publication at https://doi.org/10.1109/ICASSP40776.2020.9054102 for a full description of this data set and the interpretations of our results.

In [1]:
import mogptk
import numpy as np
import pandas as pd

Data loading¶

In [2]:
column_names = ['EUR/USD', 'CAD/USD', 'JPY/USD', 'GBP/USD', 'CHF/USD',
                'AUD/USD', 'HKD/USD','NZD/USD', 'KRW/USD','MXN/USD']

dataset = mogptk.DataSet()
for names in column_names:
    dataset.append(mogptk.LoadCSV('data/currency_exchange/final_dataset.csv',
                                    x_col='Date', y_col=names))

dataset.filter('2017-01-03', '2018-01-01')
In [3]:
# Preprocess by randomly removing points and detrending
for i, channel in enumerate(dataset):
    channel.transform(mogptk.TransformDetrend)
    channel.transform(mogptk.TransformNormalize())
    channel.remove_randomly(pct=0.3)
    
    if i not in [0, 2, 5]:
        channel.remove_range('2017-11-17', None)
    
# simulate sensor failure
dataset[1].remove_range('2017-03-31', '2017-05-01')
dataset[2].remove_range('2017-12-28', None)
dataset[3].remove_range('2017-07-20', '2017-09-08')
dataset[4].remove_range(None, '2017-01-31')
dataset[5].remove_range('2017-12-28', None)
dataset[7].remove_range(None, '2017-01-31')
In [4]:
dataset.plot();
In [5]:
n_trials = 3
Q = 3
init_method = 'BNSE'
method = 'Adam'
lr = 0.2
iters = 500

Multi-output spectral mixture kernel¶

In [6]:
mosm_models = []
mosm_mae = np.zeros((n_trials,10))
mosm_rmse = np.zeros((n_trials,10))
mosm_mape = np.zeros((n_trials,10))

# experiment trials
for n in range(n_trials):
    mosm_dataset = dataset.copy()
    for i, channel in enumerate(mosm_dataset):
        channel.remove_randomly(pct=0.3)
        
    print('\nTrial', n+1, 'of', n_trials)
    mosm = mogptk.MOSM(mosm_dataset, Q=Q)
    mosm.init_parameters(init_method)
    mosm.train(method=method, lr=lr, iters=iters, verbose=True)
    mosm_models.append(mosm)
    print('=' * 50)
    
    error = mogptk.error(mosm, per_channel=True)[0]
    mosm_mae[n,:] = np.array([item['MAE'] for item in error])
    mosm_rmse[n,:] = np.array([item['RMSE'] for item in error])
    mosm_mape[n,:] = np.array([item['MAPE'] for item in error])
Trial 1 of 3

Starting optimization using Adam
‣ Model: MOSM
‣ Channels: 10
‣ Parameters: 160
‣ Training points: 1089
‣ Initial loss: 1119.45

Start Adam:
    0/500   0:00:00  loss=     1119.45
    5/500   0:00:03  loss=     1032.38
   10/500   0:00:05  loss=     942.463
   15/500   0:00:08  loss=     850.966
   20/500   0:00:10  loss=     761.964
   25/500   0:00:13  loss=     671.378
   30/500   0:00:16  loss=     584.492
   35/500   0:00:19  loss=     496.408
   40/500   0:00:22  loss=     414.771
   45/500   0:00:24  loss=     332.073
   50/500   0:00:27  loss=     264.347
   55/500   0:00:30  loss=     206.143
   60/500   0:00:32  loss=     139.012
   65/500   0:00:35  loss=     94.6481
   70/500   0:00:38  loss=     39.3152
   75/500   0:00:41  loss=     -6.4878
   80/500   0:00:43  loss=    -43.2379
   85/500   0:00:46  loss=    -74.7615
   90/500   0:00:49  loss=     -90.153
   95/500   0:00:52  loss=    -80.3697
  100/500   0:00:56  loss=    -105.275
  105/500   0:01:00  loss=    -125.538
  110/500   0:01:04  loss=     -131.22
  115/500   0:01:09  loss=     -131.37
  120/500   0:01:15  loss=    -127.628
  125/500   0:01:20  loss=    -124.316
  130/500   0:01:25  loss=    -131.703
  135/500   0:01:30  loss=    -136.957
  140/500   0:01:35  loss=     -148.45
  145/500   0:01:42  loss=    -156.846
  150/500   0:01:49  loss=    -180.487
  155/500   0:01:55  loss=    -159.741
  160/500   0:02:00  loss=    -169.674
  165/500   0:02:06  loss=    -186.394
  170/500   0:02:11  loss=    -186.207
  175/500   0:02:15  loss=     -190.75
  180/500   0:02:20  loss=    -175.553
  185/500   0:02:24  loss=    -163.363
  190/500   0:02:29  loss=    -185.728
  195/500   0:02:33  loss=    -204.926
  200/500   0:02:36  loss=    -206.977
  205/500   0:02:39  loss=    -208.343
  210/500   0:02:42  loss=      -212.1
  215/500   0:02:45  loss=    -214.811
  220/500   0:02:48  loss=    -213.697
  225/500   0:02:51  loss=      -215.2
  230/500   0:02:54  loss=    -219.701
  235/500   0:02:56  loss=    -220.603
  240/500   0:02:59  loss=    -221.848
  245/500   0:03:02  loss=    -215.747
  250/500   0:03:05  loss=    -223.323
  255/500   0:03:09  loss=    -224.724
  260/500   0:03:13  loss=    -220.794
  265/500   0:03:17  loss=    -222.207
  270/500   0:03:21  loss=    -225.767
  275/500   0:03:25  loss=    -226.321
  280/500   0:03:30  loss=    -226.185
  285/500   0:03:33  loss=    -218.146
  290/500   0:03:37  loss=    -228.446
  295/500   0:03:41  loss=    -221.412
  300/500   0:03:44  loss=    -228.881
  305/500   0:03:48  loss=    -229.893
  310/500   0:03:51  loss=    -229.576
  315/500   0:03:55  loss=    -230.749
  320/500   0:03:58  loss=    -231.064
  325/500   0:04:02  loss=     -219.83
  330/500   0:04:05  loss=    -224.737
  335/500   0:04:09  loss=    -229.942
  340/500   0:04:12  loss=     -225.13
  345/500   0:04:16  loss=    -231.787
  350/500   0:04:20  loss=    -231.079
  355/500   0:04:24  loss=    -229.624
  360/500   0:04:27  loss=    -233.144
  365/500   0:04:31  loss=    -234.925
  370/500   0:04:35  loss=    -235.542
  375/500   0:04:38  loss=    -233.904
  380/500   0:04:42  loss=    -226.756
  385/500   0:04:45  loss=    -221.856
  390/500   0:04:49  loss=    -221.694
  395/500   0:04:53  loss=    -229.821
  400/500   0:04:57  loss=     -228.18
  405/500   0:05:01  loss=    -231.198
  410/500   0:05:05  loss=    -233.401
  415/500   0:05:09  loss=    -230.255
  420/500   0:05:13  loss=    -235.232
  425/500   0:05:17  loss=    -236.238
  430/500   0:05:21  loss=    -235.472
  435/500   0:05:24  loss=    -234.094
  440/500   0:05:28  loss=    -235.631
  445/500   0:05:32  loss=    -231.631
  450/500   0:05:36  loss=    -224.774
  455/500   0:05:40  loss=    -224.034
  460/500   0:05:43  loss=    -227.291
  465/500   0:05:47  loss=    -232.861
  470/500   0:05:51  loss=    -235.109
  475/500   0:05:54  loss=    -236.317
  480/500   0:05:58  loss=    -238.428
  485/500   0:06:02  loss=    -238.941
  490/500   0:06:06  loss=    -226.963
  495/500   0:06:10  loss=     -230.41
  500/500   0:06:13  loss=    -228.684
Finished

Optimization finished in 6 minutes 13 seconds
‣ Iterations: 500
‣ Final loss: -228.684
==================================================

Trial 2 of 3

Starting optimization using Adam
‣ Model: MOSM
‣ Channels: 10
‣ Parameters: 160
‣ Training points: 1087
‣ Initial loss: 1120.7

Start Adam:
    0/500   0:00:00  loss=      1120.7
    5/500   0:00:04  loss=     1034.13
   10/500   0:00:08  loss=     949.438
   15/500   0:00:12  loss=     862.129
   20/500   0:00:16  loss=     764.767
   25/500   0:00:21  loss=     678.391
   30/500   0:00:25  loss=     593.353
   35/500   0:00:30  loss=     514.829
   40/500   0:00:35  loss=     440.801
   45/500   0:00:39  loss=     358.773
   50/500   0:00:44  loss=     282.177
   55/500   0:00:49  loss=     219.742
   60/500   0:00:53  loss=     141.468
   65/500   0:00:58  loss=     108.973
   70/500   0:01:03  loss=     65.8549
   75/500   0:01:08  loss=      9.6963
   80/500   0:01:13  loss=    -18.6547
   85/500   0:01:18  loss=    -46.7035
   90/500   0:01:23  loss=    -63.9162
   95/500   0:01:28  loss=    -61.9074
  100/500   0:01:33  loss=    -78.6609
  105/500   0:01:38  loss=      -103.4
  110/500   0:01:44  loss=    -125.731
  115/500   0:01:49  loss=    -142.674
  120/500   0:01:56  loss=    -154.741
  125/500   0:02:02  loss=    -161.407
  130/500   0:02:08  loss=    -165.476
  135/500   0:02:15  loss=    -170.267
  140/500   0:02:20  loss=    -171.978
  145/500   0:02:26  loss=    -176.126
  150/500   0:02:31  loss=    -178.851
  155/500   0:02:35  loss=    -178.436
  160/500   0:02:41  loss=     -179.41
  165/500   0:02:45  loss=    -182.173
  170/500   0:02:51  loss=     -182.68
  175/500   0:02:55  loss=    -182.567
  180/500   0:03:01  loss=    -191.916
  185/500   0:03:06  loss=    -185.762
  190/500   0:03:11  loss=    -185.686
  195/500   0:03:16  loss=    -179.165
  200/500   0:03:20  loss=    -182.096
  205/500   0:03:25  loss=    -186.383
  210/500   0:03:31  loss=    -187.286
  215/500   0:03:36  loss=    -189.244
  220/500   0:03:41  loss=    -184.949
  225/500   0:03:46  loss=    -177.218
  230/500   0:03:50  loss=    -160.539
  235/500   0:03:56  loss=    -186.169
  240/500   0:04:01  loss=    -179.433
  245/500   0:04:06  loss=    -180.997
  250/500   0:04:11  loss=    -178.151
  255/500   0:04:16  loss=    -179.537
  260/500   0:04:21  loss=     -183.59
  265/500   0:04:26  loss=    -186.103
  270/500   0:04:31  loss=    -184.808
  275/500   0:04:36  loss=    -188.303
  280/500   0:04:40  loss=    -189.655
  285/500   0:04:45  loss=    -188.109
  290/500   0:04:51  loss=    -187.866
  295/500   0:04:56  loss=     -178.93
  300/500   0:05:01  loss=    -178.064
  305/500   0:05:05  loss=    -177.282
  310/500   0:05:08  loss=    -165.291
  315/500   0:05:12  loss=    -172.423
  320/500   0:05:15  loss=    -183.599
  325/500   0:05:17  loss=    -188.886
  330/500   0:05:20  loss=    -190.137
  335/500   0:05:23  loss=    -193.757
  340/500   0:05:25  loss=    -195.386
  345/500   0:05:28  loss=    -190.371
  350/500   0:05:31  loss=    -194.234
  355/500   0:05:33  loss=    -188.746
  360/500   0:05:36  loss=    -196.782
  365/500   0:05:39  loss=    -190.287
  370/500   0:05:41  loss=    -193.314
  375/500   0:05:44  loss=    -193.656
  380/500   0:05:46  loss=    -200.047
  385/500   0:05:49  loss=    -199.775
  390/500   0:05:52  loss=    -176.933
  395/500   0:05:54  loss=     -182.32
  400/500   0:05:57  loss=    -191.489
  405/500   0:06:00  loss=    -196.838
  410/500   0:06:02  loss=    -199.728
  415/500   0:06:05  loss=     -201.63
  420/500   0:06:07  loss=    -203.023
  425/500   0:06:10  loss=    -204.019
  430/500   0:06:13  loss=    -204.657
  435/500   0:06:15  loss=    -205.239
  440/500   0:06:18  loss=    -205.558
  445/500   0:06:20  loss=    -205.927
  450/500   0:06:23  loss=    -205.732
  455/500   0:06:26  loss=    -187.551
  460/500   0:06:28  loss=    -201.434
  465/500   0:06:31  loss=    -202.272
  470/500   0:06:34  loss=    -204.675
  475/500   0:06:37  loss=    -203.515
  480/500   0:06:40  loss=    -205.697
  485/500   0:06:44  loss=    -204.708
  490/500   0:06:47  loss=    -207.998
  495/500   0:06:50  loss=    -208.282
  500/500   0:06:53  loss=    -202.224
Finished

Optimization finished in 6 minutes 53 seconds
‣ Iterations: 500
‣ Final loss: -202.224
==================================================

Trial 3 of 3

Starting optimization using Adam
‣ Model: MOSM
‣ Channels: 10
‣ Parameters: 160
‣ Training points: 1076
‣ Initial loss: 1100.24

Start Adam:
    0/500   0:00:00  loss=     1100.24
    5/500   0:00:03  loss=     1022.11
   10/500   0:00:06  loss=     941.098
   15/500   0:00:08  loss=     855.806
   20/500   0:00:11  loss=     770.328
   25/500   0:00:14  loss=     678.253
   30/500   0:00:16  loss=     591.095
   35/500   0:00:19  loss=     514.362
   40/500   0:00:22  loss=     432.431
   45/500   0:00:24  loss=     346.632
   50/500   0:00:27  loss=     263.339
   55/500   0:00:30  loss=     200.484
   60/500   0:00:33  loss=     151.572
   65/500   0:00:35  loss=       79.91
   70/500   0:00:38  loss=     54.1625
   75/500   0:00:41  loss=     48.0171
   80/500   0:00:43  loss=     35.7453
   85/500   0:00:46  loss=    -23.8541
   90/500   0:00:49  loss=    -69.2523
   95/500   0:00:51  loss=    -73.9301
  100/500   0:00:54  loss=    -99.4781
  105/500   0:00:57  loss=    -116.066
  110/500   0:01:00  loss=    -121.918
  115/500   0:01:03  loss=    -129.967
  120/500   0:01:05  loss=    -136.275
  125/500   0:01:08  loss=    -141.532
  130/500   0:01:11  loss=    -146.859
  135/500   0:01:14  loss=    -150.026
  140/500   0:01:17  loss=    -157.607
  145/500   0:01:20  loss=    -160.959
  150/500   0:01:23  loss=    -146.466
  155/500   0:01:25  loss=    -151.614
  160/500   0:01:28  loss=    -170.226
  165/500   0:01:31  loss=    -172.563
  170/500   0:01:34  loss=     -172.49
  175/500   0:01:37  loss=    -179.466
  180/500   0:01:39  loss=    -182.586
  185/500   0:01:42  loss=    -162.372
  190/500   0:01:45  loss=     -180.75
  195/500   0:01:48  loss=    -176.408
  200/500   0:01:50  loss=    -181.165
  205/500   0:01:53  loss=    -185.768
  210/500   0:01:56  loss=    -187.945
  215/500   0:01:58  loss=    -187.111
  220/500   0:02:01  loss=    -185.809
  225/500   0:02:04  loss=    -191.502
  230/500   0:02:07  loss=     -182.58
  235/500   0:02:09  loss=    -184.653
  240/500   0:02:12  loss=    -188.252
  245/500   0:02:15  loss=    -191.708
  250/500   0:02:17  loss=    -192.447
  255/500   0:02:20  loss=    -192.772
  260/500   0:02:23  loss=    -192.296
  265/500   0:02:25  loss=    -194.016
  270/500   0:02:28  loss=    -179.396
  275/500   0:02:31  loss=    -193.289
  280/500   0:02:34  loss=    -184.023
  285/500   0:02:36  loss=    -186.605
  290/500   0:02:39  loss=    -190.488
  295/500   0:02:42  loss=    -190.895
  300/500   0:02:45  loss=    -192.945
  305/500   0:02:47  loss=    -191.744
  310/500   0:02:50  loss=    -177.546
  315/500   0:02:53  loss=    -188.683
  320/500   0:02:56  loss=    -190.041
  325/500   0:02:58  loss=    -197.086
  330/500   0:03:01  loss=     -187.05
  335/500   0:03:04  loss=    -198.159
  340/500   0:03:07  loss=    -192.132
  345/500   0:03:09  loss=    -202.583
  350/500   0:03:12  loss=    -204.805
  355/500   0:03:15  loss=    -195.989
  360/500   0:03:18  loss=    -202.418
  365/500   0:03:21  loss=    -208.037
  370/500   0:03:23  loss=    -210.833
  375/500   0:03:26  loss=    -217.299
  380/500   0:03:29  loss=    -219.152
  385/500   0:03:32  loss=    -220.254
  390/500   0:03:34  loss=    -219.952
  395/500   0:03:37  loss=     -219.98
  400/500   0:03:40  loss=    -221.815
  405/500   0:03:42  loss=    -220.367
  410/500   0:03:45  loss=    -220.181
  415/500   0:03:48  loss=    -221.641
  420/500   0:03:51  loss=    -224.179
  425/500   0:03:53  loss=      -221.4
  430/500   0:03:56  loss=    -223.827
  435/500   0:03:59  loss=    -224.348
  440/500   0:04:02  loss=    -224.344
  445/500   0:04:04  loss=    -216.475
  450/500   0:04:07  loss=    -219.133
  455/500   0:04:10  loss=    -221.072
  460/500   0:04:12  loss=    -207.929
  465/500   0:04:15  loss=     -206.72
  470/500   0:04:18  loss=    -194.691
  475/500   0:04:21  loss=     -192.23
  480/500   0:04:24  loss=    -199.222
  485/500   0:04:27  loss=    -197.543
  490/500   0:04:30  loss=    -184.291
  495/500   0:04:32  loss=    -205.117
  500/500   0:04:35  loss=    -232.248
Finished

Optimization finished in 4 minutes 35 seconds
‣ Iterations: 500
‣ Final loss: -232.248
==================================================
In [7]:
pd.DataFrame(np.c_[mosm_mae.mean(1), mosm_rmse.mean(1), mosm_mape.mean(1),
             mosm_mae.std(1), mosm_rmse.std(1), mosm_mape.std(1)],
             columns=['MAE', 'RMSE', 'MAPE', 'MAE std', 'RMSE std', 'MAPE std'])
Out[7]:
MAE RMSE MAPE MAE std RMSE std MAPE std
0 1.122094 1.362972 0.744078 3.086486 3.727266 0.341182
1 1.138951 1.345307 0.770069 3.150050 3.691730 0.335431
2 1.152736 1.341146 0.722918 3.132835 3.616437 0.322265
In [8]:
best_mosm = mosm_models[np.argmin(mosm_mape.mean(1))]
best_mosm.predict()
best_mosm.plot_prediction();

Cross spectral mixture kernel¶

In [9]:
csm_models = []
csm_mae = np.zeros((n_trials,10))
csm_rmse = np.zeros((n_trials,10))
csm_mape = np.zeros((n_trials,10))

for n in range(n_trials):
    csm_dataset = dataset.copy()
    for i, channel in enumerate(csm_dataset):
        channel.remove_randomly(pct=0.3)
        
    print('\nTrial', n+1, 'of', n_trials)
    csm = mogptk.CSM(csm_dataset, Q=Q)
    csm.init_parameters(init_method)    
    csm.train(method=method, lr=lr, iters=iters, verbose=True)
    csm_models.append(csm)
    print('=' * 50)
    
    error = mogptk.error(csm, per_channel=True)[0]
    csm_mae[n,:] = np.array([item['MAE'] for item in error])
    csm_rmse[n,:] = np.array([item['RMSE'] for item in error])
    csm_mape[n,:] = np.array([item['MAPE'] for item in error])
Trial 1 of 3

Starting optimization using Adam
‣ Model: CSM
‣ Channels: 10
‣ Parameters: 76
‣ Training points: 1080
‣ Initial loss: 1104.66

Start Adam:
    0/500   0:00:01  loss=     1104.66
    5/500   0:00:07  loss=     967.154
   10/500   0:00:13  loss=     875.214
   15/500   0:00:18  loss=     777.076
   20/500   0:00:23  loss=     684.597
   25/500   0:00:29  loss=     597.686
   30/500   0:00:34  loss=     513.449
   35/500   0:00:39  loss=     435.087
   40/500   0:00:45  loss=     362.404
   45/500   0:00:50  loss=     296.356
   50/500   0:00:56  loss=     237.544
   55/500   0:01:01  loss=     187.677
   60/500   0:01:07  loss=     150.239
   65/500   0:01:12  loss=     107.381
   70/500   0:01:17  loss=     77.5491
   75/500   0:01:22  loss=     45.2565
   80/500   0:01:28  loss=     27.1807
   85/500   0:01:33  loss=     20.0989
   90/500   0:01:39  loss=    0.386542
   95/500   0:01:44  loss=    -24.4887
  100/500   0:01:49  loss=    -35.5421
  105/500   0:01:55  loss=    -44.7956
  110/500   0:02:00  loss=    -51.3644
  115/500   0:02:05  loss=    -51.5075
  120/500   0:02:11  loss=    -69.9832
  125/500   0:02:16  loss=    -62.8513
  130/500   0:02:22  loss=    -57.7233
  135/500   0:02:27  loss=    -65.2968
  140/500   0:02:32  loss=    -75.6087
  145/500   0:02:38  loss=    -79.7484
  150/500   0:02:43  loss=    -79.9018
  155/500   0:02:49  loss=    -82.9261
  160/500   0:02:54  loss=    -83.6699
  165/500   0:02:59  loss=    -82.0538
  170/500   0:03:05  loss=    -72.0986
  175/500   0:03:10  loss=    -85.9892
  180/500   0:03:16  loss=    -78.9211
  185/500   0:03:21  loss=    -84.0515
  190/500   0:03:27  loss=    -83.8045
  195/500   0:03:32  loss=    -75.1414
  200/500   0:03:38  loss=    -86.4495
  205/500   0:03:43  loss=    -80.0601
  210/500   0:03:49  loss=    -79.2002
  215/500   0:03:54  loss=    -86.6912
  220/500   0:04:00  loss=    -76.6325
  225/500   0:04:05  loss=    -80.9937
  230/500   0:04:10  loss=    -83.8536
  235/500   0:04:16  loss=    -77.6687
  240/500   0:04:21  loss=    -87.0174
  245/500   0:04:26  loss=    -77.5109
  250/500   0:04:32  loss=    -81.0671
  255/500   0:04:37  loss=    -85.5635
  260/500   0:04:43  loss=    -75.7962
  265/500   0:04:48  loss=     -86.647
  270/500   0:04:53  loss=    -82.8575
  275/500   0:04:59  loss=    -74.0881
  280/500   0:05:04  loss=    -87.5633
  285/500   0:05:10  loss=    -75.1352
  290/500   0:05:16  loss=    -86.3883
  295/500   0:05:21  loss=    -83.3956
  300/500   0:05:26  loss=    -73.8982
  305/500   0:05:31  loss=    -88.0426
  310/500   0:05:37  loss=    -80.1117
  315/500   0:05:42  loss=     -78.016
  320/500   0:05:47  loss=    -86.6205
  325/500   0:05:53  loss=    -76.0196
  330/500   0:05:58  loss=    -90.2824
  335/500   0:06:04  loss=    -83.4247
  340/500   0:06:09  loss=    -76.6221
  345/500   0:06:14  loss=    -95.8849
  350/500   0:06:20  loss=    -90.3963
  355/500   0:06:25  loss=    -110.711
  360/500   0:06:31  loss=    -132.365
  365/500   0:06:36  loss=    -137.473
  370/500   0:06:41  loss=    -182.173
  375/500   0:06:47  loss=    -177.348
  380/500   0:06:52  loss=    -170.906
  385/500   0:06:58  loss=    -193.234
  390/500   0:07:03  loss=    -209.579
  395/500   0:07:09  loss=    -219.721
  400/500   0:07:14  loss=    -190.122
  405/500   0:07:20  loss=    -185.508
  410/500   0:07:26  loss=    -185.006
  415/500   0:07:31  loss=    -195.993
  420/500   0:07:37  loss=    -232.392
  425/500   0:07:42  loss=    -218.339
  430/500   0:07:47  loss=    -242.561
  435/500   0:07:52  loss=    -233.098
  440/500   0:07:58  loss=     -265.36
  445/500   0:08:03  loss=    -297.155
  450/500   0:08:08  loss=    -302.505
  455/500   0:08:14  loss=    -309.213
  460/500   0:08:20  loss=    -321.099
  465/500   0:08:25  loss=    -270.408
  470/500   0:08:31  loss=     -137.74
  475/500   0:08:36  loss=    -44.5384
  480/500   0:08:42  loss=    -129.887
  485/500   0:08:47  loss=    -133.967
  490/500   0:08:52  loss=    -175.371
  495/500   0:08:58  loss=    -193.693
  500/500   0:09:03  loss=    -225.148
Finished

Optimization finished in 9 minutes 3 seconds
‣ Iterations: 500
‣ Final loss: -225.148
==================================================

Trial 2 of 3

Starting optimization using Adam
‣ Model: CSM
‣ Channels: 10
‣ Parameters: 76
‣ Training points: 1083
‣ Initial loss: 1110.26

Start Adam:
    0/500   0:00:01  loss=     1110.26
    5/500   0:00:06  loss=      988.87
   10/500   0:00:11  loss=     883.387
   15/500   0:00:17  loss=     784.216
   20/500   0:00:23  loss=     692.055
   25/500   0:00:28  loss=     603.785
   30/500   0:00:33  loss=     519.114
   35/500   0:00:39  loss=     440.344
   40/500   0:00:44  loss=     367.491
   45/500   0:00:50  loss=     300.353
   50/500   0:00:55  loss=     245.237
   55/500   0:01:01  loss=      192.35
   60/500   0:01:06  loss=     146.645
   65/500   0:01:11  loss=     126.815
   70/500   0:01:17  loss=     85.8179
   75/500   0:01:22  loss=     57.5969
   80/500   0:01:27  loss=     29.4822
   85/500   0:01:33  loss=     19.0807
   90/500   0:01:38  loss=     14.4175
   95/500   0:01:43  loss=    -11.0922
  100/500   0:01:49  loss=    -25.5476
  105/500   0:01:54  loss=    -44.4351
  110/500   0:02:00  loss=    -46.9498
  115/500   0:02:05  loss=    -57.7846
  120/500   0:02:11  loss=     -50.644
  125/500   0:02:16  loss=    -12.2196
  130/500   0:02:21  loss=    -10.9353
  135/500   0:02:27  loss=    -34.2323
  140/500   0:02:32  loss=    -50.2671
  145/500   0:02:38  loss=    -54.6134
  150/500   0:02:43  loss=    -72.2288
  155/500   0:02:49  loss=    -78.0813
  160/500   0:02:54  loss=    -80.9181
  165/500   0:03:00  loss=    -85.4733
  170/500   0:03:05  loss=    -93.4026
  175/500   0:03:10  loss=    -97.2637
  180/500   0:03:16  loss=    -103.641
  185/500   0:03:21  loss=    -108.364
  190/500   0:03:26  loss=    -109.647
  195/500   0:03:32  loss=    -109.972
  200/500   0:03:37  loss=    -114.518
  205/500   0:03:42  loss=    -114.139
  210/500   0:03:48  loss=    -113.306
  215/500   0:03:53  loss=    -119.676
  220/500   0:03:58  loss=      -120.7
  225/500   0:04:04  loss=    -113.792
  230/500   0:04:09  loss=    -128.241
  235/500   0:04:14  loss=    -127.302
  240/500   0:04:20  loss=    -127.142
  245/500   0:04:25  loss=    -126.063
  250/500   0:04:31  loss=    -127.978
  255/500   0:04:36  loss=    -128.658
  260/500   0:04:41  loss=    -133.891
  265/500   0:04:47  loss=    -126.523
  270/500   0:04:52  loss=    -126.943
  275/500   0:04:58  loss=    -115.709
  280/500   0:05:03  loss=    -123.483
  285/500   0:05:08  loss=    -137.557
  290/500   0:05:13  loss=    -139.376
  295/500   0:05:19  loss=    -146.385
  300/500   0:05:24  loss=    -147.777
  305/500   0:05:29  loss=    -141.332
  310/500   0:05:35  loss=    -138.864
  315/500   0:05:40  loss=     -152.41
  320/500   0:05:46  loss=    -154.105
  325/500   0:05:51  loss=    -155.257
  330/500   0:05:56  loss=    -158.856
  335/500   0:06:01  loss=    -161.226
  340/500   0:06:07  loss=    -163.542
  345/500   0:06:12  loss=     -163.78
  350/500   0:06:17  loss=    -155.875
  355/500   0:06:23  loss=    -149.504
  360/500   0:06:28  loss=    -137.199
  365/500   0:06:34  loss=    -131.041
  370/500   0:06:39  loss=    -160.336
  375/500   0:06:45  loss=    -178.874
  380/500   0:06:50  loss=    -186.308
  385/500   0:06:56  loss=    -143.127
  390/500   0:07:01  loss=     -166.65
  395/500   0:07:06  loss=    -88.6465
  400/500   0:07:12  loss=    -145.125
  405/500   0:07:17  loss=    -166.042
  410/500   0:07:22  loss=    -169.613
  415/500   0:07:28  loss=    -187.315
  420/500   0:07:33  loss=    -205.405
  425/500   0:07:39  loss=    -215.848
  430/500   0:07:44  loss=     -224.54
  435/500   0:07:50  loss=    -228.396
  440/500   0:07:55  loss=    -230.439
  445/500   0:08:00  loss=    -234.536
  450/500   0:08:06  loss=    -224.048
  455/500   0:08:11  loss=    -96.0072
  460/500   0:08:16  loss=    -154.576
  465/500   0:08:22  loss=    -180.054
  470/500   0:08:27  loss=    -180.952
  475/500   0:08:32  loss=    -210.572
  480/500   0:08:37  loss=    -214.414
  485/500   0:08:43  loss=    -225.681
  490/500   0:08:48  loss=    -238.371
  495/500   0:08:53  loss=    -246.441
  500/500   0:08:59  loss=    -252.729
Finished

Optimization finished in 8 minutes 59 seconds
‣ Iterations: 500
‣ Final loss: -252.729
==================================================

Trial 3 of 3

Starting optimization using Adam
‣ Model: CSM
‣ Channels: 10
‣ Parameters: 76
‣ Training points: 1093
‣ Initial loss: 1120.56

Start Adam:
    0/500   0:00:01  loss=     1120.56
    5/500   0:00:06  loss=     984.519
   10/500   0:00:11  loss=     881.915
   15/500   0:00:17  loss=     788.589
   20/500   0:00:22  loss=     691.043
   25/500   0:00:27  loss=     600.375
   30/500   0:00:33  loss=     514.865
   35/500   0:00:38  loss=     433.917
   40/500   0:00:44  loss=     358.929
   45/500   0:00:49  loss=     295.498
   50/500   0:00:54  loss=     232.598
   55/500   0:01:00  loss=     179.408
   60/500   0:01:05  loss=     133.943
   65/500   0:01:11  loss=     102.069
   70/500   0:01:16  loss=     70.4565
   75/500   0:01:21  loss=     45.2425
   80/500   0:01:27  loss=     31.2734
   85/500   0:01:32  loss=     4.31543
   90/500   0:01:38  loss=    -14.8691
   95/500   0:01:43  loss=     17.8004
  100/500   0:01:49  loss=     6.23479
  105/500   0:01:54  loss=    -45.4577
  110/500   0:02:00  loss=    -41.8609
  115/500   0:02:05  loss=    -47.8622
  120/500   0:02:10  loss=    -53.9507
  125/500   0:02:15  loss=    -67.0061
  130/500   0:02:21  loss=    -35.7663
  135/500   0:02:26  loss=    -58.5743
  140/500   0:02:32  loss=    -60.8647
  145/500   0:02:37  loss=    -54.7714
  150/500   0:02:42  loss=    -53.9035
  155/500   0:02:48  loss=    -58.5041
  160/500   0:02:53  loss=    -71.7191
  165/500   0:02:59  loss=    -73.6996
  170/500   0:03:05  loss=    -74.8439
  175/500   0:03:10  loss=    -76.0674
  180/500   0:03:16  loss=    -71.8609
  185/500   0:03:21  loss=    -75.9761
  190/500   0:03:27  loss=     -63.518
  195/500   0:03:32  loss=    -78.7726
  200/500   0:03:38  loss=     -66.993
  205/500   0:03:43  loss=     -79.001
  210/500   0:03:48  loss=    -74.6051
  215/500   0:03:54  loss=    -67.8313
  220/500   0:03:59  loss=    -72.4115
  225/500   0:04:05  loss=    -76.9558
  230/500   0:04:10  loss=    -77.3285
  235/500   0:04:16  loss=    -77.6405
  240/500   0:04:21  loss=    -85.4137
  245/500   0:04:26  loss=    -88.5865
  250/500   0:04:32  loss=    -95.1426
  255/500   0:04:37  loss=    -101.819
  260/500   0:04:42  loss=    -103.337
  265/500   0:04:48  loss=    -81.7684
  270/500   0:04:53  loss=    -89.8302
  275/500   0:04:59  loss=    -102.072
  280/500   0:05:04  loss=    -100.587
  285/500   0:05:10  loss=    -104.401
  290/500   0:05:15  loss=    -102.323
  295/500   0:05:20  loss=    -107.903
  300/500   0:05:26  loss=     -106.79
  305/500   0:05:31  loss=     -109.07
  310/500   0:05:36  loss=     -111.84
  315/500   0:05:42  loss=    -99.3657
  320/500   0:05:47  loss=    -107.291
  325/500   0:05:53  loss=    -110.426
  330/500   0:05:58  loss=    -111.499
  335/500   0:06:03  loss=    -113.501
  340/500   0:06:09  loss=    -113.976
  345/500   0:06:14  loss=    -113.095
  350/500   0:06:19  loss=      -113.5
  355/500   0:06:24  loss=    -117.915
  360/500   0:06:30  loss=    -113.327
  365/500   0:06:35  loss=    -119.409
  370/500   0:06:40  loss=    -118.705
  375/500   0:06:45  loss=    -123.312
  380/500   0:06:51  loss=    -125.494
  385/500   0:06:56  loss=      -125.4
  390/500   0:07:01  loss=    -126.517
  395/500   0:07:07  loss=    -126.932
  400/500   0:07:12  loss=    -127.472
  405/500   0:07:18  loss=    -127.914
  410/500   0:07:23  loss=     -127.86
  415/500   0:07:28  loss=    -115.308
  420/500   0:07:34  loss=    -123.577
  425/500   0:07:39  loss=    -123.019
  430/500   0:07:44  loss=    -127.631
  435/500   0:07:49  loss=    -127.157
  440/500   0:07:55  loss=    -127.838
  445/500   0:08:00  loss=    -127.877
  450/500   0:08:05  loss=    -127.974
  455/500   0:08:11  loss=     -128.64
  460/500   0:08:16  loss=    -127.086
  465/500   0:08:21  loss=    -122.859
  470/500   0:08:26  loss=    -128.852
  475/500   0:08:32  loss=    -127.959
  480/500   0:08:37  loss=    -129.345
  485/500   0:08:43  loss=    -128.172
  490/500   0:08:48  loss=    -127.446
  495/500   0:08:54  loss=    -126.285
  500/500   0:08:59  loss=    -128.872
Finished

Optimization finished in 8 minutes 59 seconds
‣ Iterations: 500
‣ Final loss: -128.872
==================================================
In [10]:
pd.DataFrame(np.c_[csm_mae.mean(1), csm_rmse.mean(1), csm_mape.mean(1),
             csm_mae.std(1), csm_rmse.std(1), csm_mape.std(1)],
             columns=['MAE', 'RMSE', 'MAPE', 'MAE std', 'RMSE std', 'MAPE std'])
Out[10]:
MAE RMSE MAPE MAE std RMSE std MAPE std
0 0.599223 0.765870 0.627079 1.487277 1.879569 0.277313
1 1.180773 1.372929 0.707619 3.219390 3.675202 0.306940
2 1.194325 1.444053 0.754181 3.214573 3.857320 0.295204
In [11]:
best_csm = csm_models[np.argmin(csm_mape.mean(1))]
best_csm.predict()
best_csm.plot_prediction();

Linear model of coregionalization with spectral mixture kernels¶

In [12]:
smlmc_models = []
smlmc_mae = np.zeros((n_trials,10))
smlmc_rmse = np.zeros((n_trials,10))
smlmc_mape = np.zeros((n_trials,10))

for n in range(n_trials):
    smlmc_dataset = dataset.copy()
    for i, channel in enumerate(smlmc_dataset):
        channel.remove_randomly(pct=0.3)
        
    print('\nTrial', n+1, 'of', n_trials)
    smlmc = mogptk.SM_LMC(smlmc_dataset, Q=Q)
    smlmc.init_parameters(init_method)
    smlmc.train(method=method, lr=lr, iters=iters, verbose=True)
    smlmc_models.append(smlmc)
    print('=' * 50)

    error = mogptk.error(smlmc, per_channel=True)[0]
    smlmc_mae[n,:] = np.array([item['MAE'] for item in error])
    smlmc_rmse[n,:] = np.array([item['RMSE'] for item in error])
    smlmc_mape[n,:] = np.array([item['MAPE'] for item in error])
Trial 1 of 3

Starting optimization using Adam
‣ Model: SM-LMC
‣ Channels: 10
‣ Parameters: 46
‣ Training points: 1102
‣ Initial loss: 1127.37

Start Adam:
    0/500   0:00:00  loss=     1127.37
    5/500   0:00:03  loss=     1022.73
   10/500   0:00:06  loss=      931.47
   15/500   0:00:09  loss=     835.228
   20/500   0:00:11  loss=     749.111
   25/500   0:00:14  loss=      666.16
   30/500   0:00:17  loss=      588.59
   35/500   0:00:20  loss=     517.691
   40/500   0:00:23  loss=      453.26
   45/500   0:00:25  loss=     396.232
   50/500   0:00:28  loss=     346.554
   55/500   0:00:31  loss=     304.455
   60/500   0:00:34  loss=     270.139
   65/500   0:00:37  loss=     243.549
   70/500   0:00:40  loss=     223.488
   75/500   0:00:43  loss=     208.551
   80/500   0:00:46  loss=     198.032
   85/500   0:00:48  loss=     190.729
   90/500   0:00:51  loss=     185.635
   95/500   0:00:54  loss=     182.288
  100/500   0:00:57  loss=      179.37
  105/500   0:01:00  loss=     177.556
  110/500   0:01:03  loss=     175.766
  115/500   0:01:05  loss=     174.472
  120/500   0:01:08  loss=     173.517
  125/500   0:01:11  loss=     172.399
  130/500   0:01:14  loss=     171.992
  135/500   0:01:17  loss=     171.048
  140/500   0:01:20  loss=     170.825
  145/500   0:01:22  loss=     170.135
  150/500   0:01:25  loss=     169.764
  155/500   0:01:28  loss=     169.293
  160/500   0:01:31  loss=      168.68
  165/500   0:01:34  loss=     168.161
  170/500   0:01:37  loss=     167.195
  175/500   0:01:40  loss=     165.592
  180/500   0:01:42  loss=     163.867
  185/500   0:01:45  loss=     161.476
  190/500   0:01:48  loss=      160.07
  195/500   0:01:51  loss=     157.698
  200/500   0:01:54  loss=     156.658
  205/500   0:01:57  loss=     155.173
  210/500   0:01:59  loss=     154.826
  215/500   0:02:02  loss=     153.951
  220/500   0:02:05  loss=     154.407
  225/500   0:02:08  loss=     153.574
  230/500   0:02:11  loss=     153.839
  235/500   0:02:14  loss=     153.369
  240/500   0:02:17  loss=     153.587
  245/500   0:02:19  loss=     153.241
  250/500   0:02:22  loss=      153.05
  255/500   0:02:25  loss=     153.381
  260/500   0:02:28  loss=     153.252
  265/500   0:02:31  loss=     153.134
  270/500   0:02:34  loss=     152.681
  275/500   0:02:37  loss=     152.365
  280/500   0:02:39  loss=     152.303
  285/500   0:02:42  loss=     152.067
  290/500   0:02:45  loss=     151.936
  295/500   0:02:48  loss=      151.69
  300/500   0:02:51  loss=     151.594
  305/500   0:02:54  loss=     152.081
  310/500   0:02:56  loss=     150.923
  315/500   0:02:59  loss=     150.871
  320/500   0:03:02  loss=     150.596
  325/500   0:03:05  loss=     150.181
  330/500   0:03:08  loss=      150.04
  335/500   0:03:11  loss=     149.701
  340/500   0:03:14  loss=     149.524
  345/500   0:03:16  loss=      149.41
  350/500   0:03:19  loss=     149.414
  355/500   0:03:22  loss=     149.009
  360/500   0:03:25  loss=     148.742
  365/500   0:03:28  loss=     148.612
  370/500   0:03:31  loss=     148.366
  375/500   0:03:34  loss=     148.167
  380/500   0:03:36  loss=     148.076
  385/500   0:03:39  loss=     148.087
  390/500   0:03:42  loss=     148.146
  395/500   0:03:45  loss=     147.662
  400/500   0:03:48  loss=      147.65
  405/500   0:03:51  loss=     147.442
  410/500   0:03:54  loss=     147.359
  415/500   0:03:57  loss=     147.262
  420/500   0:03:59  loss=     147.141
  425/500   0:04:02  loss=      147.07
  430/500   0:04:05  loss=      147.02
  435/500   0:04:08  loss=     147.046
  440/500   0:04:11  loss=     147.238
  445/500   0:04:14  loss=     146.843
  450/500   0:04:17  loss=     146.786
  455/500   0:04:19  loss=     146.656
  460/500   0:04:22  loss=     146.607
  465/500   0:04:25  loss=     146.521
  470/500   0:04:28  loss=      146.46
  475/500   0:04:31  loss=     146.413
  480/500   0:04:34  loss=     146.345
  485/500   0:04:37  loss=     146.288
  490/500   0:04:39  loss=     146.237
  495/500   0:04:42  loss=     146.193
  500/500   0:04:45  loss=     146.187
Finished

Optimization finished in 4 minutes 45 seconds
‣ Iterations: 500
‣ Final loss: 146.187
==================================================

Trial 2 of 3

Starting optimization using Adam
‣ Model: SM-LMC
‣ Channels: 10
‣ Parameters: 46
‣ Training points: 1087
‣ Initial loss: 1112.71

Start Adam:
    0/500   0:00:00  loss=     1112.71
    5/500   0:00:03  loss=     1007.89
   10/500   0:00:06  loss=     916.594
   15/500   0:00:08  loss=     824.624
   20/500   0:00:11  loss=     740.702
   25/500   0:00:14  loss=     660.723
   30/500   0:00:17  loss=     585.539
   35/500   0:00:20  loss=     515.862
   40/500   0:00:22  loss=     452.568
   45/500   0:00:25  loss=     396.636
   50/500   0:00:28  loss=     347.768
   55/500   0:00:31  loss=     306.739
   60/500   0:00:33  loss=     273.708
   65/500   0:00:36  loss=      247.72
   70/500   0:00:39  loss=     228.906
   75/500   0:00:42  loss=     215.571
   80/500   0:00:45  loss=     206.489
   85/500   0:00:47  loss=     200.927
   90/500   0:00:50  loss=     197.581
   95/500   0:00:53  loss=     194.917
  100/500   0:00:55  loss=     193.093
  105/500   0:00:58  loss=     192.018
  110/500   0:01:01  loss=     191.151
  115/500   0:01:03  loss=     190.076
  120/500   0:01:06  loss=     189.726
  125/500   0:01:09  loss=     188.711
  130/500   0:01:12  loss=     188.222
  135/500   0:01:15  loss=     187.618
  140/500   0:01:17  loss=     187.294
  145/500   0:01:20  loss=     186.609
  150/500   0:01:22  loss=     186.447
  155/500   0:01:25  loss=     185.812
  160/500   0:01:28  loss=      185.64
  165/500   0:01:30  loss=     185.212
  170/500   0:01:33  loss=     184.871
  175/500   0:01:36  loss=      184.91
  180/500   0:01:39  loss=     184.362
  185/500   0:01:42  loss=     184.234
  190/500   0:01:44  loss=     183.777
  195/500   0:01:47  loss=     183.678
  200/500   0:01:50  loss=     183.307
  205/500   0:01:52  loss=     182.998
  210/500   0:01:55  loss=     182.833
  215/500   0:01:58  loss=     183.591
  220/500   0:02:00  loss=     182.751
  225/500   0:02:03  loss=     182.191
  230/500   0:02:06  loss=     181.633
  235/500   0:02:09  loss=     181.146
  240/500   0:02:11  loss=     180.695
  245/500   0:02:14  loss=     180.188
  250/500   0:02:17  loss=     179.578
  255/500   0:02:19  loss=     178.829
  260/500   0:02:22  loss=     177.854
  265/500   0:02:25  loss=     176.737
  270/500   0:02:28  loss=     175.563
  275/500   0:02:30  loss=     174.714
  280/500   0:02:33  loss=     171.992
  285/500   0:02:36  loss=     169.182
  290/500   0:02:39  loss=     167.187
  295/500   0:02:41  loss=     166.017
  300/500   0:02:44  loss=     165.067
  305/500   0:02:47  loss=     164.328
  310/500   0:02:50  loss=     163.549
  315/500   0:02:53  loss=     161.971
  320/500   0:02:56  loss=     158.621
  325/500   0:02:59  loss=     155.348
  330/500   0:03:01  loss=     152.471
  335/500   0:03:04  loss=     149.803
  340/500   0:03:07  loss=     147.991
  345/500   0:03:10  loss=     146.681
  350/500   0:03:12  loss=      145.78
  355/500   0:03:15  loss=     144.775
  360/500   0:03:18  loss=     143.955
  365/500   0:03:20  loss=     143.236
  370/500   0:03:23  loss=     142.602
  375/500   0:03:26  loss=     141.996
  380/500   0:03:28  loss=     141.397
  385/500   0:03:31  loss=     140.803
  390/500   0:03:34  loss=     140.221
  395/500   0:03:37  loss=     139.637
  400/500   0:03:39  loss=      139.04
  405/500   0:03:42  loss=     138.359
  410/500   0:03:45  loss=     137.604
  415/500   0:03:48  loss=     136.895
  420/500   0:03:50  loss=     136.346
  425/500   0:03:53  loss=     135.907
  430/500   0:03:56  loss=     135.525
  435/500   0:03:58  loss=     135.182
  440/500   0:04:01  loss=     134.849
  445/500   0:04:04  loss=     134.432
  450/500   0:04:07  loss=     133.482
  455/500   0:04:09  loss=     131.872
  460/500   0:04:12  loss=     130.642
  465/500   0:04:15  loss=     129.874
  470/500   0:04:18  loss=     129.607
  475/500   0:04:20  loss=     129.542
  480/500   0:04:23  loss=     129.404
  485/500   0:04:26  loss=      129.24
  490/500   0:04:29  loss=     129.088
  495/500   0:04:32  loss=     128.968
  500/500   0:04:35  loss=     128.856
Finished

Optimization finished in 4 minutes 35 seconds
‣ Iterations: 500
‣ Final loss: 128.856
==================================================

Trial 3 of 3

Starting optimization using Adam
‣ Model: SM-LMC
‣ Channels: 10
‣ Parameters: 46
‣ Training points: 1079
‣ Initial loss: 1104.81

Start Adam:
    0/500   0:00:00  loss=     1104.81
    5/500   0:00:03  loss=     1003.53
   10/500   0:00:05  loss=     912.808
   15/500   0:00:08  loss=     822.026
   20/500   0:00:11  loss=     739.157
   25/500   0:00:14  loss=     663.403
   30/500   0:00:16  loss=     590.237
   35/500   0:00:19  loss=     521.726
   40/500   0:00:22  loss=     462.038
   45/500   0:00:25  loss=     411.304
   50/500   0:00:27  loss=     365.981
   55/500   0:00:30  loss=     327.793
   60/500   0:00:33  loss=     295.435
   65/500   0:00:36  loss=     267.946
   70/500   0:00:39  loss=     244.918
   75/500   0:00:41  loss=     225.768
   80/500   0:00:44  loss=      210.27
   85/500   0:00:47  loss=     198.149
   90/500   0:00:50  loss=     189.064
   95/500   0:00:52  loss=     182.347
  100/500   0:00:55  loss=     177.107
  105/500   0:00:58  loss=     172.585
  110/500   0:01:01  loss=     168.414
  115/500   0:01:04  loss=     163.825
  120/500   0:01:06  loss=     158.387
  125/500   0:01:09  loss=     153.724
  130/500   0:01:12  loss=     150.409
  135/500   0:01:14  loss=     146.936
  140/500   0:01:17  loss=     142.488
  145/500   0:01:20  loss=     132.709
  150/500   0:01:23  loss=     125.463
  155/500   0:01:25  loss=     121.001
  160/500   0:01:28  loss=     117.605
  165/500   0:01:31  loss=     114.571
  170/500   0:01:34  loss=     110.833
  175/500   0:01:36  loss=      106.67
  180/500   0:01:39  loss=     103.013
  185/500   0:01:42  loss=     98.8749
  190/500   0:01:45  loss=      90.231
  195/500   0:01:47  loss=     72.8991
  200/500   0:01:50  loss=     50.3819
  205/500   0:01:53  loss=     24.7077
  210/500   0:01:56  loss=     6.66079
  215/500   0:01:58  loss=    -6.88354
  220/500   0:02:01  loss=    -18.1042
  225/500   0:02:04  loss=     -29.017
  230/500   0:02:07  loss=    -39.5909
  235/500   0:02:10  loss=    -48.0308
  240/500   0:02:12  loss=    -55.5923
  245/500   0:02:15  loss=    -63.0714
  250/500   0:02:18  loss=    -69.5319
  255/500   0:02:21  loss=    -71.8793
  260/500   0:02:23  loss=    -77.6852
  265/500   0:02:26  loss=    -80.8007
  270/500   0:02:29  loss=    -83.2123
  275/500   0:02:31  loss=    -86.2337
  280/500   0:02:34  loss=    -91.1443
  285/500   0:02:37  loss=    -93.8234
  290/500   0:02:40  loss=    -96.4279
  295/500   0:02:43  loss=    -98.4109
  300/500   0:02:45  loss=    -99.8681
  305/500   0:02:48  loss=    -101.013
  310/500   0:02:51  loss=     -102.12
  315/500   0:02:54  loss=    -104.244
  320/500   0:02:56  loss=    -106.813
  325/500   0:02:59  loss=    -110.096
  330/500   0:03:02  loss=    -112.681
  335/500   0:03:05  loss=    -114.636
  340/500   0:03:07  loss=    -115.951
  345/500   0:03:11  loss=    -116.715
  350/500   0:03:13  loss=    -117.133
  355/500   0:03:16  loss=    -117.367
  360/500   0:03:19  loss=     -117.53
  365/500   0:03:22  loss=    -117.675
  370/500   0:03:25  loss=    -117.812
  375/500   0:03:28  loss=    -117.939
  380/500   0:03:30  loss=    -118.053
  385/500   0:03:33  loss=    -118.157
  390/500   0:03:36  loss=    -118.252
  395/500   0:03:38  loss=     -118.34
  400/500   0:03:41  loss=     -118.42
  405/500   0:03:44  loss=    -118.494
  410/500   0:03:47  loss=    -118.562
  415/500   0:03:49  loss=    -118.624
  420/500   0:03:52  loss=    -118.682
  425/500   0:03:55  loss=    -118.737
  430/500   0:03:58  loss=    -118.789
  435/500   0:04:01  loss=    -118.839
  440/500   0:04:03  loss=    -118.888
  445/500   0:04:06  loss=    -118.935
  450/500   0:04:09  loss=    -118.981
  455/500   0:04:12  loss=    -119.027
  460/500   0:04:15  loss=    -119.072
  465/500   0:04:17  loss=    -119.116
  470/500   0:04:20  loss=     -119.16
  475/500   0:04:23  loss=    -119.202
  480/500   0:04:26  loss=    -119.244
  485/500   0:04:28  loss=    -119.286
  490/500   0:04:31  loss=    -119.326
  495/500   0:04:34  loss=    -119.366
  500/500   0:04:37  loss=    -119.406
Finished

Optimization finished in 4 minutes 37 seconds
‣ Iterations: 500
‣ Final loss: -119.406
==================================================
In [13]:
pd.DataFrame(np.c_[smlmc_mae.mean(1), smlmc_rmse.mean(1), smlmc_mape.mean(1),
             smlmc_mae.std(1), smlmc_rmse.std(1), smlmc_mape.std(1)],
             columns=['MAE', 'RMSE', 'MAPE', 'MAE std', 'RMSE std', 'MAPE std'])
Out[13]:
MAE RMSE MAPE MAE std RMSE std MAPE std
0 1.300352 1.651520 0.906478 3.447964 4.384061 0.311815
1 1.151461 1.412515 0.862733 3.057899 3.718161 0.277745
2 1.312147 1.558453 0.711142 3.574198 4.228392 0.299729
In [14]:
best_smlmc = smlmc_models[np.argmin(smlmc_mape.mean(1))]
best_smlmc.predict()
best_smlmc.plot_prediction();

Independent spectral mixture kernels¶

In [15]:
sm_models = []
sm_mae = np.zeros((n_trials,10))
sm_rmse = np.zeros((n_trials,10))
sm_mape = np.zeros((n_trials,10))

for n in range(n_trials):
    sm_dataset = dataset.copy()
    for i, channel in enumerate(sm_dataset):
        channel.remove_randomly(pct=0.3)
    
    print('\nTrial', n+1, 'of', n_trials)
    sm = mogptk.SM(sm_dataset, Q=Q)
    sm.init_parameters(init_method)
    sm.train(method=method, lr=lr, iters=iters, verbose=True)
    sm_models.append(sm)
    print('=' * 50)
        
    error = mogptk.error(sm, per_channel=True)[0]
    sm_mae[n,:] = np.array([item['MAE'] for item in error])
    sm_rmse[n,:] = np.array([item['RMSE'] for item in error])
    sm_mape[n,:] = np.array([item['MAPE'] for item in error])
Trial 1 of 3

Starting optimization using Adam
‣ Model: SM
‣ Channels: 10
‣ Parameters: 100
‣ Training points: 1081
‣ Initial loss: 1091.95

Start Adam:
    0/500   0:00:00  loss=     1091.95
    5/500   0:00:02  loss=     990.352
   10/500   0:00:04  loss=     894.648
   15/500   0:00:06  loss=     799.661
   20/500   0:00:08  loss=     708.003
   25/500   0:00:10  loss=     617.989
   30/500   0:00:12  loss=     530.441
   35/500   0:00:15  loss=     446.143
   40/500   0:00:17  loss=      365.64
   45/500   0:00:19  loss=     289.013
   50/500   0:00:21  loss=     216.623
   55/500   0:00:23  loss=     150.409
   60/500   0:00:25  loss=     91.1761
   65/500   0:00:27  loss=     39.0437
   70/500   0:00:30  loss=    -7.55423
   75/500   0:00:32  loss=    -49.7618
   80/500   0:00:34  loss=    -82.4165
   85/500   0:00:36  loss=    -110.847
   90/500   0:00:38  loss=    -131.547
   95/500   0:00:40  loss=    -147.092
  100/500   0:00:42  loss=    -157.296
  105/500   0:00:44  loss=    -164.487
  110/500   0:00:46  loss=    -171.726
  115/500   0:00:49  loss=    -174.586
  120/500   0:00:51  loss=    -179.122
  125/500   0:00:53  loss=    -183.813
  130/500   0:00:55  loss=     -191.42
  135/500   0:00:57  loss=    -200.184
  140/500   0:00:59  loss=    -206.819
  145/500   0:01:02  loss=    -211.274
  150/500   0:01:04  loss=    -213.316
  155/500   0:01:06  loss=    -215.672
  160/500   0:01:08  loss=    -219.953
  165/500   0:01:10  loss=    -222.953
  170/500   0:01:12  loss=    -225.229
  175/500   0:01:15  loss=    -227.776
  180/500   0:01:17  loss=    -229.026
  185/500   0:01:19  loss=    -230.324
  190/500   0:01:21  loss=    -230.947
  195/500   0:01:23  loss=    -231.629
  200/500   0:01:25  loss=    -232.376
  205/500   0:01:27  loss=    -230.824
  210/500   0:01:29  loss=    -232.699
  215/500   0:01:31  loss=    -232.756
  220/500   0:01:34  loss=    -232.297
  225/500   0:01:36  loss=    -233.603
  230/500   0:01:38  loss=    -230.292
  235/500   0:01:40  loss=    -236.595
  240/500   0:01:42  loss=    -233.509
  245/500   0:01:44  loss=    -237.396
  250/500   0:01:47  loss=    -239.599
  255/500   0:01:49  loss=     -241.97
  260/500   0:01:51  loss=     -242.51
  265/500   0:01:53  loss=    -242.893
  270/500   0:01:55  loss=    -243.288
  275/500   0:01:57  loss=    -243.628
  280/500   0:01:59  loss=    -243.855
  285/500   0:02:01  loss=    -243.858
  290/500   0:02:04  loss=    -243.305
  295/500   0:02:06  loss=    -243.812
  300/500   0:02:08  loss=    -243.921
  305/500   0:02:10  loss=    -243.989
  310/500   0:02:12  loss=    -244.126
  315/500   0:02:14  loss=    -244.274
  320/500   0:02:16  loss=    -244.284
  325/500   0:02:18  loss=    -244.476
  330/500   0:02:21  loss=    -244.374
  335/500   0:02:23  loss=    -244.455
  340/500   0:02:25  loss=    -244.423
  345/500   0:02:27  loss=      -244.6
  350/500   0:02:29  loss=    -244.154
  355/500   0:02:31  loss=    -244.515
  360/500   0:02:33  loss=    -244.398
  365/500   0:02:36  loss=    -244.633
  370/500   0:02:38  loss=    -244.796
  375/500   0:02:40  loss=    -244.831
  380/500   0:02:42  loss=    -244.753
  385/500   0:02:44  loss=    -244.682
  390/500   0:02:46  loss=    -244.147
  395/500   0:02:49  loss=     -244.54
  400/500   0:02:51  loss=    -245.032
  405/500   0:02:53  loss=    -244.061
  410/500   0:02:55  loss=    -244.258
  415/500   0:02:57  loss=    -244.622
  420/500   0:02:59  loss=    -245.199
  425/500   0:03:01  loss=    -245.217
  430/500   0:03:04  loss=      -245.2
  435/500   0:03:06  loss=    -245.169
  440/500   0:03:08  loss=    -245.524
  445/500   0:03:10  loss=    -245.953
  450/500   0:03:13  loss=    -246.697
  455/500   0:03:15  loss=    -248.317
  460/500   0:03:17  loss=    -250.413
  465/500   0:03:19  loss=    -252.845
  470/500   0:03:21  loss=    -254.981
  475/500   0:03:23  loss=    -256.157
  480/500   0:03:25  loss=    -257.021
  485/500   0:03:27  loss=    -256.749
  490/500   0:03:30  loss=    -257.456
  495/500   0:03:32  loss=    -257.411
  500/500   0:03:34  loss=    -257.795
Finished

Optimization finished in 3 minutes 34 seconds
‣ Iterations: 500
‣ Final loss: -257.795
==================================================

Trial 2 of 3

Starting optimization using Adam
‣ Model: SM
‣ Channels: 10
‣ Parameters: 100
‣ Training points: 1101
‣ Initial loss: 1111.76

Start Adam:
    0/500   0:00:00  loss=     1111.76
    5/500   0:00:02  loss=     1007.97
   10/500   0:00:04  loss=     910.902
   15/500   0:00:07  loss=     814.982
   20/500   0:00:09  loss=      720.87
   25/500   0:00:11  loss=     628.554
   30/500   0:00:14  loss=     538.712
   35/500   0:00:16  loss=     452.266
   40/500   0:00:18  loss=      369.26
   45/500   0:00:20  loss=     290.747
   50/500   0:00:23  loss=     217.359
   55/500   0:00:25  loss=     149.939
   60/500   0:00:27  loss=     88.4323
   65/500   0:00:29  loss=     33.8108
   70/500   0:00:32  loss=    -14.9666
   75/500   0:00:34  loss=    -58.4961
   80/500   0:00:36  loss=    -92.3959
   85/500   0:00:38  loss=    -123.683
   90/500   0:00:40  loss=    -148.408
   95/500   0:00:43  loss=    -165.492
  100/500   0:00:45  loss=    -180.188
  105/500   0:00:47  loss=    -190.975
  110/500   0:00:49  loss=    -199.558
  115/500   0:00:52  loss=    -204.281
  120/500   0:00:54  loss=    -207.188
  125/500   0:00:56  loss=    -212.803
  130/500   0:00:58  loss=    -218.864
  135/500   0:01:01  loss=    -227.056
  140/500   0:01:03  loss=    -231.749
  145/500   0:01:05  loss=    -235.541
  150/500   0:01:07  loss=    -238.505
  155/500   0:01:09  loss=    -240.973
  160/500   0:01:12  loss=    -243.139
  165/500   0:01:14  loss=     -242.93
  170/500   0:01:16  loss=    -244.602
  175/500   0:01:19  loss=    -253.269
  180/500   0:01:21  loss=    -255.276
  185/500   0:01:23  loss=    -257.707
  190/500   0:01:25  loss=    -259.211
  195/500   0:01:28  loss=    -260.651
  200/500   0:01:30  loss=    -261.806
  205/500   0:01:32  loss=    -264.438
  210/500   0:01:34  loss=    -268.144
  215/500   0:01:37  loss=    -272.702
  220/500   0:01:39  loss=    -277.229
  225/500   0:01:41  loss=    -280.944
  230/500   0:01:43  loss=    -283.838
  235/500   0:01:45  loss=    -285.883
  240/500   0:01:48  loss=    -291.699
  245/500   0:01:50  loss=    -295.424
  250/500   0:01:52  loss=    -298.515
  255/500   0:01:55  loss=    -299.635
  260/500   0:01:57  loss=    -301.229
  265/500   0:01:59  loss=    -301.652
  270/500   0:02:01  loss=    -303.667
  275/500   0:02:04  loss=    -307.472
  280/500   0:02:06  loss=    -311.567
  285/500   0:02:08  loss=    -313.404
  290/500   0:02:11  loss=    -314.075
  295/500   0:02:13  loss=    -315.545
  300/500   0:02:15  loss=    -315.996
  305/500   0:02:17  loss=    -316.031
  310/500   0:02:20  loss=    -316.848
  315/500   0:02:22  loss=    -316.877
  320/500   0:02:24  loss=    -316.567
  325/500   0:02:26  loss=    -317.079
  330/500   0:02:29  loss=    -317.371
  335/500   0:02:31  loss=    -316.759
  340/500   0:02:33  loss=    -316.889
  345/500   0:02:35  loss=    -317.293
  350/500   0:02:37  loss=    -317.185
  355/500   0:02:40  loss=    -317.363
  360/500   0:02:42  loss=    -317.679
  365/500   0:02:44  loss=    -317.624
  370/500   0:02:46  loss=    -317.087
  375/500   0:02:49  loss=    -317.262
  380/500   0:02:51  loss=    -317.396
  385/500   0:02:53  loss=    -317.525
  390/500   0:02:55  loss=    -317.354
  395/500   0:02:58  loss=    -317.831
  400/500   0:03:00  loss=    -317.817
  405/500   0:03:02  loss=    -317.474
  410/500   0:03:04  loss=    -317.675
  415/500   0:03:07  loss=     -317.89
  420/500   0:03:09  loss=    -317.831
  425/500   0:03:11  loss=    -317.537
  430/500   0:03:13  loss=    -317.616
  435/500   0:03:16  loss=    -317.878
  440/500   0:03:18  loss=    -317.834
  445/500   0:03:20  loss=    -317.481
  450/500   0:03:22  loss=    -318.009
  455/500   0:03:25  loss=    -318.085
  460/500   0:03:27  loss=    -317.775
  465/500   0:03:29  loss=    -317.426
  470/500   0:03:31  loss=    -317.862
  475/500   0:03:34  loss=    -318.095
  480/500   0:03:36  loss=    -317.851
  485/500   0:03:38  loss=    -317.761
  490/500   0:03:41  loss=    -318.274
  495/500   0:03:43  loss=    -318.241
  500/500   0:03:45  loss=    -318.128
Finished

Optimization finished in 3 minutes 45 seconds
‣ Iterations: 500
‣ Final loss: -318.128
==================================================

Trial 3 of 3

Starting optimization using Adam
‣ Model: SM
‣ Channels: 10
‣ Parameters: 100
‣ Training points: 1086
‣ Initial loss: 1095.58

Start Adam:
    0/500   0:00:00  loss=     1095.58
    5/500   0:00:02  loss=     994.364
   10/500   0:00:04  loss=     897.103
   15/500   0:00:07  loss=     802.113
   20/500   0:00:09  loss=     709.339
   25/500   0:00:11  loss=     618.273
   30/500   0:00:13  loss=     530.009
   35/500   0:00:15  loss=     444.738
   40/500   0:00:18  loss=     363.362
   45/500   0:00:20  loss=     284.074
   50/500   0:00:22  loss=     211.048
   55/500   0:00:24  loss=     142.124
   60/500   0:00:26  loss=     80.6439
   65/500   0:00:29  loss=     25.7677
   70/500   0:00:31  loss=    -22.8297
   75/500   0:00:33  loss=    -64.0164
   80/500   0:00:35  loss=      -98.31
   85/500   0:00:38  loss=    -128.035
   90/500   0:00:40  loss=    -154.261
   95/500   0:00:42  loss=    -173.647
  100/500   0:00:44  loss=    -188.341
  105/500   0:00:46  loss=    -196.478
  110/500   0:00:49  loss=    -201.892
  115/500   0:00:51  loss=    -205.573
  120/500   0:00:53  loss=    -207.741
  125/500   0:00:55  loss=    -209.202
  130/500   0:00:57  loss=    -209.632
  135/500   0:01:00  loss=    -211.493
  140/500   0:01:02  loss=      -212.9
  145/500   0:01:04  loss=    -214.296
  150/500   0:01:06  loss=    -215.198
  155/500   0:01:08  loss=    -216.125
  160/500   0:01:11  loss=    -217.559
  165/500   0:01:13  loss=    -212.481
  170/500   0:01:15  loss=    -216.976
  175/500   0:01:17  loss=    -216.552
  180/500   0:01:20  loss=    -223.311
  185/500   0:01:22  loss=    -222.872
  190/500   0:01:24  loss=    -225.329
  195/500   0:01:27  loss=    -227.412
  200/500   0:01:29  loss=    -229.622
  205/500   0:01:31  loss=    -231.494
  210/500   0:01:33  loss=    -233.426
  215/500   0:01:35  loss=    -234.658
  220/500   0:01:38  loss=    -235.649
  225/500   0:01:40  loss=    -235.941
  230/500   0:01:42  loss=    -236.758
  235/500   0:01:44  loss=    -236.989
  240/500   0:01:47  loss=    -237.102
  245/500   0:01:49  loss=    -237.441
  250/500   0:01:51  loss=    -237.834
  255/500   0:01:53  loss=    -238.018
  260/500   0:01:56  loss=    -238.081
  265/500   0:01:58  loss=    -238.189
  270/500   0:02:00  loss=    -238.279
  275/500   0:02:02  loss=    -238.306
  280/500   0:02:04  loss=    -238.429
  285/500   0:02:07  loss=    -238.194
  290/500   0:02:09  loss=     -238.64
  295/500   0:02:11  loss=    -238.528
  300/500   0:02:13  loss=    -238.308
  305/500   0:02:15  loss=    -237.864
  310/500   0:02:18  loss=     -238.13
  315/500   0:02:20  loss=    -238.305
  320/500   0:02:22  loss=     -238.44
  325/500   0:02:24  loss=    -238.478
  330/500   0:02:26  loss=    -238.778
  335/500   0:02:29  loss=    -238.886
  340/500   0:02:31  loss=    -238.796
  345/500   0:02:33  loss=    -238.865
  350/500   0:02:35  loss=    -238.924
  355/500   0:02:37  loss=    -238.793
  360/500   0:02:40  loss=    -238.878
  365/500   0:02:42  loss=    -239.155
  370/500   0:02:44  loss=    -238.728
  375/500   0:02:46  loss=    -238.639
  380/500   0:02:48  loss=    -238.916
  385/500   0:02:51  loss=    -239.131
  390/500   0:02:53  loss=    -238.881
  395/500   0:02:55  loss=    -238.926
  400/500   0:02:57  loss=    -238.955
  405/500   0:03:00  loss=    -239.121
  410/500   0:03:02  loss=    -238.894
  415/500   0:03:04  loss=     -239.22
  420/500   0:03:06  loss=    -239.295
  425/500   0:03:09  loss=    -239.163
  430/500   0:03:11  loss=    -239.101
  435/500   0:03:13  loss=    -239.249
  440/500   0:03:15  loss=    -239.014
  445/500   0:03:18  loss=    -238.912
  450/500   0:03:20  loss=    -239.404
  455/500   0:03:22  loss=     -239.17
  460/500   0:03:24  loss=    -239.333
  465/500   0:03:27  loss=    -238.969
  470/500   0:03:29  loss=    -239.344
  475/500   0:03:31  loss=    -239.206
  480/500   0:03:33  loss=    -238.746
  485/500   0:03:35  loss=    -238.673
  490/500   0:03:38  loss=    -239.343
  495/500   0:03:40  loss=     -239.03
  500/500   0:03:42  loss=    -239.115
Finished

Optimization finished in 3 minutes 42 seconds
‣ Iterations: 500
‣ Final loss: -239.115
==================================================
In [16]:
pd.DataFrame(np.c_[sm_mae.mean(1), sm_rmse.mean(1), sm_mape.mean(1),
             sm_mae.std(1), sm_rmse.std(1), sm_mape.std(1)],
             columns=['MAE', 'RMSE', 'MAPE', 'MAE std', 'RMSE std', 'MAPE std'])
Out[16]:
MAE RMSE MAPE MAE std RMSE std MAPE std
0 0.926913 1.233118 0.709221 2.539383 3.378007 0.316391
1 1.036391 1.336773 0.636743 2.897598 3.732074 0.289083
2 0.896370 1.203064 0.734213 2.431880 3.280786 0.347998
In [17]:
best_sm = sm_models[np.argmin(sm_mape.mean(1))]
best_sm.predict()
best_sm.plot_prediction();

Results¶

In [18]:
print('MAPE errors per model')
print('MOSM:    %g ± %g' % (mosm_mape.mean(1).mean(), mosm_mape.mean(1).std()))
print('CSM:     %g ± %g' % (csm_mape.mean(1).mean(), csm_mape.mean(1).std()))
print('SM-LMC:  %g ± %g' % (smlmc_mape.mean(1).mean(), smlmc_mape.mean(1).std()))
print('SM:      %g ± %g' % (sm_mape.mean(1).mean(), sm_mape.mean(1).std()))
MAPE errors per model
MOSM:    0.745688 ± 0.0192832
CSM:     0.696293 ± 0.0525037
SM-LMC:  0.826785 ± 0.083699
SM:      0.693392 ± 0.041336
In [ ]: