--- title: Title keywords: fastai sidebar: home_sidebar nb_path: "nbs/experiments__auto.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

auto[source]

auto(config_dict:dict, Y_df:DataFrame, X_df:DataFrame, S_df:DataFrame, loss_function_val:callable, loss_functions_test:dict, forecast_horizon:int, ts_in_val:int, ts_in_test:int, return_forecasts:bool=False, return_model:bool=True, test_auto:bool=False, verbose:bool=False)

Auto hyperparameter tuning function.

Parameters

config_dict: Dict Dictionary with configuration. Keys should be name of models. For each model specify the hyperparameter space (None will use default suggested space), hyperopt steps and timeout. Y_df: pd.DataFrame Target time series with columns ['unique_id', 'ds', 'y']. X_df: pd.DataFrame Exogenous time series with columns ['unique_id', 'ds', 'y']. S_df: pd.DataFrame Static exogenous variables with columns ['unique_id', 'ds']. and static variables. loss_function_val: function Loss function used for validation. loss_functions_test: Dictionary Loss functions used for test. (function name: string, function: fun) forecast_horizon: int Forecast horizon ts_in_val: int Number of timestamps in validation. ts_in_test: int Number of timestamps in test. return_forecasts: bool If true return forecast on test. return_model: bool If true return model. test_auto: bool If true, will only run one training step and hyperopt iteration for each model. Use it for testing purposes. verbose: If true, will print summary of dataset, model and training.

{% endraw %} {% raw %}
{% endraw %} {% raw %}

instantiate_space[source]

instantiate_space(model, n_time_out, n_series, n_x, n_s, frequency, test)

{% endraw %} {% raw %}
{% endraw %}

Example

{% raw %}
import matplotlib.pyplot as plt
{% endraw %} {% raw %}
dataset = ['NP']
Y_df, X_df, S_df = nf.data.datasets.epf.EPF.load_groups(directory='data', groups=dataset)
Y_df['y'] = Y_df['y']/Y_df['y'].max()
X_df['Exogenous1'] = X_df['Exogenous1']/X_df['Exogenous1'].max()
X_df['Exogenous2'] = X_df['Exogenous2']/X_df['Exogenous2'].max()
{% endraw %}

Config dictionary

{% raw %}
config_dict = {
               'nbeats':
                       {'space': None, # Use default
                        'hyperopt_steps': 5,
                        'timeout': 60*1
                       },
               'nhits':
                       {'space': None,  # Use default
                        'hyperopt_steps': 5,
                        'timeout': 60*1
                       },
                'rnn':
                       {'space': None,  # Use default
                        'hyperopt_steps': 5,
                        'timeout': 60*1
                       }
              }
{% endraw %}

Automl

{% raw %}
best_model, results = auto(config_dict=config_dict,
                           Y_df=Y_df, X_df=X_df, S_df=None,
                           loss_function_val=nf.losses.numpy.mae, loss_functions_test={'mae':nf.losses.numpy.mae,'mse':nf.losses.numpy.mse},
                           forecast_horizon=24, ts_in_val=168, ts_in_test=168,
                           return_forecasts=True, return_model=True,
                           test_auto=True,
                           verbose=False)
INFO:hyperopt.tpe:build_posterior_wrapper took 0.012829 seconds
INFO:hyperopt.tpe:TPE using 0 trials
WARNING: THIS IS A TEST, MODELS WILL NOT BE TRAINED PROPERLY!
/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/data/tsdataset.py:207: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  X.drop(['unique_id', 'ds'], 1, inplace=True)
INFO:pytorch_lightning.utilities.distributed:GPU available: False, used: False
INFO:pytorch_lightning.utilities.distributed:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.distributed:IPU available: False, using: 0 IPUs
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 12 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 12 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/data/tsloader.py:47: UserWarning: This class wraps the pytorch `DataLoader` with a special collate function. If you want to use yours simply use `DataLoader`. Removing collate_fn
  'This class wraps the pytorch `DataLoader` with a '
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 12 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
INFO:hyperopt.tpe:build_posterior_wrapper took 0.015187 seconds
INFO:hyperopt.tpe:TPE using 0 trials
INFO:pytorch_lightning.utilities.distributed:GPU available: False, used: False
INFO:pytorch_lightning.utilities.distributed:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.distributed:IPU available: False, using: 0 IPUs
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/torch/nn/functional.py:3635: UserWarning: Default upsampling behavior when mode=linear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode)
INFO:hyperopt.tpe:build_posterior_wrapper took 0.011127 seconds
INFO:hyperopt.tpe:TPE using 0 trials
INFO:pytorch_lightning.utilities.distributed:GPU available: False, used: False
INFO:pytorch_lightning.utilities.distributed:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.distributed:IPU available: False, using: 0 IPUs
{% endraw %} {% raw %}
time = results['nbeats']['optimization_times']
losses = results['nbeats']['optimization_losses']
plt.plot(time, losses)
plt.xlabel('segs')
plt.ylabel('val loss')
Text(0, 0.5, 'val loss')
{% endraw %}

Load forecasts for test

{% raw %}
best_nhits = results['nhits']['y_hat']
best_nbeats = results['nbeats']['y_hat']
best_rnn = results['rnn']['y_hat']
y_true = results['nbeats']['y_true']
{% endraw %} {% raw %}
plt.plot(y_true[::24,:].flatten(), label='True')
plt.plot(best_nhits[::24,:].flatten(), label='N-HiTS')
plt.plot(best_nbeats[::24,:].flatten(), label='N-BEATS')
plt.plot(best_rnn[::24,:].flatten(), label='RNN')
plt.legend()
<matplotlib.legend.Legend at 0x7f7a93392350>
{% endraw %}

Or forecast with best model

{% raw %}
best_model
RNN(
  (model): _RNN(
    (rnn_stack): Sequential(
      (0): DRNN(
        (cells): Sequential(
          (0): LSTM(768, 20)
          (1): LSTM(20, 20)
        )
      )
    )
    (adapterW): Linear(in_features=20, out_features=24, bias=True)
  )
)
{% endraw %} {% raw %}
Y_forecast_df = Y_df[Y_df['ds']<'2016-12-27']
X_forecast_df = X_df[X_df['ds']<'2016-12-28']
{% endraw %} {% raw %}
best_model.return_decomposition = False
forecast_df = best_model.forecast(Y_df=Y_forecast_df, X_df=X_forecast_df, S_df=None, batch_size=2)
forecast_df
/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/models/rnn/rnn.py:372: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  Y_df['ds'] = pd.to_datetime(Y_df['ds'])
/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/models/rnn/rnn.py:374: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  X_df['ds'] = pd.to_datetime(X_df['ds'])
INFO:root:Train Validation splits

INFO:root:                              ds                    
                             min                 max
unique_id sample_mask                               
NP        0           2013-01-01 2016-12-26 23:00:00
          1           2016-12-27 2016-12-27 23:00:00
INFO:root:
Total data 			34968 time stamps 
Available percentage=100.0, 	34968 time stamps 
Insample  percentage=0.07, 	24 time stamps 
Outsample percentage=99.93, 	34944 time stamps 

/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/data/tsdataset.py:207: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  X.drop(['unique_id', 'ds'], 1, inplace=True)
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:91: LightningDeprecationWarning: Setting `Trainer(progress_bar_refresh_rate=1)` is deprecated in v1.5 and will be removed in v1.7. Please pass `pytorch_lightning.callbacks.progress.TQDMProgressBar` with `refresh_rate` directly to the Trainer's `callbacks` argument instead. Or, to disable the progress bar pass `enable_progress_bar = False` to the Trainer.
  f"Setting `Trainer(progress_bar_refresh_rate={progress_bar_refresh_rate})` is deprecated in v1.5 and"
INFO:pytorch_lightning.utilities.distributed:GPU available: False, used: False
INFO:pytorch_lightning.utilities.distributed:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.distributed:IPU available: False, using: 0 IPUs
/Users/cchallu/NIXTLA/neuralforecast/neuralforecast/data/tsloader.py:47: UserWarning: This class wraps the pytorch `DataLoader` with a special collate function. If you want to use yours simply use `DataLoader`. Removing collate_fn
  'This class wraps the pytorch `DataLoader` with a '
/Users/cchallu/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 12 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
Predicting: 100%|██████████| 1/1 [00:00<00:00, 14.17it/s]
unique_id ds y
0 NP 2016-12-27 00:00:00 -0.019946
1 NP 2016-12-27 01:00:00 0.036935
2 NP 2016-12-27 02:00:00 -0.185193
3 NP 2016-12-27 03:00:00 -0.085634
4 NP 2016-12-27 04:00:00 -0.082543
5 NP 2016-12-27 05:00:00 -0.012673
6 NP 2016-12-27 06:00:00 0.160076
7 NP 2016-12-27 07:00:00 0.190466
8 NP 2016-12-27 08:00:00 -0.263867
9 NP 2016-12-27 09:00:00 -0.056118
10 NP 2016-12-27 10:00:00 0.112959
11 NP 2016-12-27 11:00:00 -0.087473
12 NP 2016-12-27 12:00:00 0.235862
13 NP 2016-12-27 13:00:00 0.043721
14 NP 2016-12-27 14:00:00 0.085159
15 NP 2016-12-27 15:00:00 0.207209
16 NP 2016-12-27 16:00:00 0.113367
17 NP 2016-12-27 17:00:00 -0.102256
18 NP 2016-12-27 18:00:00 -0.080079
19 NP 2016-12-27 19:00:00 -0.066619
20 NP 2016-12-27 20:00:00 -0.035557
21 NP 2016-12-27 21:00:00 0.256218
22 NP 2016-12-27 22:00:00 0.310515
23 NP 2016-12-27 23:00:00 -0.133059
{% endraw %}