--- title: N-HiTS keywords: fastai sidebar: home_sidebar nb_path: "nbs/models_nhits__nhits.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %}

A new model for long-horizon forecasting which incorporates novel hierarchical interpolation and multi-rate data sampling techniques to specialize blocks of its architecture to different frequency band of the time-series signal. It achieves SoTA performance on several benchmark datasets, outperforming current Transformer-based models by more than 25%. Paper available at https://arxiv.org/abs/2201.12886

{% raw %}

class NHITS[source]

NHITS(n_time_in:int, n_time_out:int, n_x:int, n_s:int, shared_weights:bool, activation:str, initialization:str, stack_types:List[str], n_blocks:List[int], n_layers:List[int], n_mlp_units:List[List[int]], n_x_hidden:int, n_s_hidden:int, n_pool_kernel_size:List[int], n_freq_downsample:List[int], pooling_mode:str, interpolation_mode:str, batch_normalization:bool, dropout_prob_theta:float, learning_rate:float, lr_decay:float, lr_decay_step_size:int, weight_decay:float, loss_train:str, loss_hypar:float, loss_valid:str, frequency:str, random_seed:int) :: LightningModule

Hooks to be used in LightningModule.

{% endraw %} {% raw %}
{% endraw %} {% raw %}

NHITS.forecast[source]

NHITS.forecast(Y_df:DataFrame, X_df:DataFrame=None, S_df:DataFrame=None, batch_size:int=1, trainer:Trainer=None)

Method for forecasting self.n_time_out periods after last timestamp of Y_df.

Parameters

Y_df: pd.DataFrame Dataframe with target time-series data, needs 'unique_id','ds' and 'y' columns. X_df: pd.DataFrame Dataframe with exogenous time-series data, needs 'unique_id' and 'ds' columns. Note that 'unique_id' and 'ds' must match Y_df plus the forecasting horizon. S_df: pd.DataFrame Dataframe with static data, needs 'unique_id' column. bath_size: int Batch size for forecasting. trainer: pl.Trainer Trainer object for model training and evaluation.

Returns

forecast_df: pd.DataFrame Dataframe with forecasts.

{% endraw %} {% raw %}
{% endraw %}

N-HITS Usage Example

Load Data

{% raw %}
import pandas as pd
from neuralforecast.data.datasets.epf import EPF
from neuralforecast.data.tsloader import TimeSeriesLoader

import pylab as plt
from pylab import rcParams
plt.style.use('seaborn-whitegrid')
plt.rcParams['font.family'] = 'serif'

FONTSIZE = 19

# Load and plot data
Y_df, X_df, S_df = EPF.load_groups(directory='./data', groups=['NP','FR'])

fig = plt.figure(figsize=(15, 6))
plt.plot(Y_df[Y_df['unique_id']=='NP'].ds, Y_df[Y_df['unique_id']=='NP'].y.values, color='#628793', linewidth=0.4)
plt.ylabel('Price [EUR/MWh]', fontsize=19)
plt.xlabel('Date', fontsize=15)
plt.show()
{% endraw %}

Declare Model and Data Parameters

{% raw %}
mc = {}
mc['model'] = 'n-hits'
mc['mode'] = 'simple'
mc['activation'] = 'SELU'

mc['n_time_in'] = 24*3
mc['n_time_out'] = 24
mc['n_x_hidden'] = 8
mc['n_s_hidden'] = 0

mc['stack_types'] = ['identity', 'identity', 'identity']
mc['constant_n_blocks'] = 1
mc['constant_n_layers'] = 2
mc['constant_n_mlp_units'] = 256
mc['n_pool_kernel_size'] = [4, 2, 1]
mc['n_freq_downsample'] = [24, 12, 1]
mc['pooling_mode'] = 'max'
mc['interpolation_mode'] = 'linear'
mc['shared_weights'] = False

# Optimization and regularization parameters
mc['initialization'] = 'lecun_normal'
mc['learning_rate'] = 0.001
mc['batch_size'] = 1
mc['n_windows'] = 32
mc['lr_decay'] = 0.5
mc['lr_decay_step_size'] = 2
mc['max_epochs'] = 1
mc['max_steps'] = None
mc['early_stop_patience'] = 20
mc['eval_freq'] = 500
mc['batch_normalization'] = False
mc['dropout_prob_theta'] = 0.0
mc['dropout_prob_exogenous'] = 0.0
mc['weight_decay'] = 0
mc['loss_train'] = 'MAE'
mc['loss_hypar'] = 0.5
mc['loss_valid'] = mc['loss_train']
mc['random_seed'] = 1

# Data Parameters
mc['idx_to_sample_freq'] = 1
mc['val_idx_to_sample_freq'] = 1
mc['n_val_weeks'] = 52
mc['normalizer_y'] = None
mc['normalizer_x'] = 'median'
mc['complete_windows'] = False
mc['frequency'] = 'H'

print(65*'=')
print(pd.Series(mc))
print(65*'=')

mc['n_mlp_units'] = len(mc['stack_types']) * [ mc['constant_n_layers'] * [int(mc['constant_n_mlp_units'])] ]
mc['n_blocks'] =  len(mc['stack_types']) * [ mc['constant_n_blocks'] ]
mc['n_layers'] =  len(mc['stack_types']) * [ mc['constant_n_layers'] ]
=================================================================
model                                             n-hits
mode                                              simple
activation                                          SELU
n_time_in                                             72
n_time_out                                            24
n_x_hidden                                             8
n_s_hidden                                             0
stack_types               [identity, identity, identity]
constant_n_blocks                                      1
constant_n_layers                                      2
constant_n_mlp_units                                 256
n_pool_kernel_size                             [4, 2, 1]
n_freq_downsample                            [24, 12, 1]
pooling_mode                                         max
interpolation_mode                                linear
shared_weights                                     False
initialization                              lecun_normal
learning_rate                                      0.001
batch_size                                             1
n_windows                                             32
lr_decay                                             0.5
lr_decay_step_size                                     2
max_epochs                                             1
max_steps                                           None
early_stop_patience                                   20
eval_freq                                            500
batch_normalization                                False
dropout_prob_theta                                   0.0
dropout_prob_exogenous                               0.0
weight_decay                                           0
loss_train                                           MAE
loss_hypar                                           0.5
loss_valid                                           MAE
random_seed                                            1
idx_to_sample_freq                                     1
val_idx_to_sample_freq                                 1
n_val_weeks                                           52
normalizer_y                                        None
normalizer_x                                      median
complete_windows                                   False
frequency                                              H
dtype: object
=================================================================
{% endraw %}

Instantiate Loaders and Model

{% raw %}
from neuralforecast.experiments.utils import create_datasets

train_dataset, val_dataset, test_dataset, scaler_y = create_datasets(mc=mc,
                                                                     S_df=S_df, Y_df=Y_df, X_df=X_df,
                                                                     f_cols=['Exogenous1', 'Exogenous2'],
                                                                     ds_in_val=294*24,
                                                                     ds_in_test=728*24)

train_loader = TimeSeriesLoader(dataset=train_dataset,
                                batch_size=int(mc['batch_size']),
                                n_windows=mc['n_windows'],
                                shuffle=True)

val_loader = TimeSeriesLoader(dataset=val_dataset,
                              batch_size=int(mc['batch_size']),
                              shuffle=False)

test_loader = TimeSeriesLoader(dataset=test_dataset,
                               batch_size=int(mc['batch_size']),
                               shuffle=False)

mc['n_x'], mc['n_s'] = train_dataset.get_n_variables()
{% endraw %} {% raw %}
model = NHITS(n_time_in=int(mc['n_time_in']),
              n_time_out=int(mc['n_time_out']),
              n_x=mc['n_x'],
              n_s=mc['n_s'],
              n_s_hidden=int(mc['n_s_hidden']),
              n_x_hidden=int(mc['n_x_hidden']),
              shared_weights=mc['shared_weights'],
              initialization=mc['initialization'],
              activation=mc['activation'],
              stack_types=mc['stack_types'],
              n_blocks=mc['n_blocks'],
              n_layers=mc['n_layers'],
              n_mlp_units=mc['n_mlp_units'],
              n_pool_kernel_size=mc['n_pool_kernel_size'],
              n_freq_downsample=mc['n_freq_downsample'],
              pooling_mode=mc['pooling_mode'],
              interpolation_mode=mc['interpolation_mode'],
              batch_normalization = mc['batch_normalization'],
              dropout_prob_theta=mc['dropout_prob_theta'],
              learning_rate=float(mc['learning_rate']),
              lr_decay=float(mc['lr_decay']),
              lr_decay_step_size=float(mc['lr_decay_step_size']),
              weight_decay=mc['weight_decay'],
              loss_train=mc['loss_train'],
              loss_hypar=float(mc['loss_hypar']),
              loss_valid=mc['loss_valid'],
              frequency=mc['frequency'],
              random_seed=int(mc['random_seed']))
{% endraw %}

Train Model

{% raw %}
from pytorch_lightning.callbacks import EarlyStopping

early_stopping = EarlyStopping(monitor="val_loss", 
                               min_delta=1e-4, 
                               patience=mc['early_stop_patience'],
                               verbose=False,
                               mode="min")

trainer = pl.Trainer(max_epochs=mc['max_epochs'], 
                     max_steps=mc['max_steps'],
                     gradient_clip_val=1.0,
                     progress_bar_refresh_rate=10, 
                     log_every_n_steps=500, 
                     check_val_every_n_epoch=1,
                     callbacks=[early_stopping])

trainer.fit(model, train_loader, val_loader)
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py:49: LightningDeprecationWarning: Setting `max_steps = None` is deprecated in v1.5 and will no longer be supported in v1.7. Use `max_steps = -1` instead.
  "Setting `max_steps = None` is deprecated in v1.5 and will no longer be supported in v1.7."
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:91: LightningDeprecationWarning: Setting `Trainer(progress_bar_refresh_rate=10)` is deprecated in v1.5 and will be removed in v1.7. Please pass `pytorch_lightning.callbacks.progress.TQDMProgressBar` with `refresh_rate` directly to the Trainer's `callbacks` argument instead. Or, to disable the progress bar pass `enable_progress_bar = False` to the Trainer.
  f"Setting `Trainer(progress_bar_refresh_rate={progress_bar_refresh_rate})` is deprecated in v1.5 and"
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs

  | Name  | Type   | Params
---------------------------------
0 | model | _NHITS | 1.0 M 
---------------------------------
1.0 M     Trainable params
0         Non-trainable params
1.0 M     Total params
4.121     Total estimated model params size (MB)
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/torch/nn/functional.py:3613: UserWarning: Default upsampling behavior when mode=linear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode)
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:433: UserWarning: The number of training samples (2) is smaller than the logging interval Trainer(log_every_n_steps=500). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  f"The number of training samples ({self.num_training_batches}) is smaller than the logging interval"
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/torch/nn/functional.py:652: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /tmp/pip-req-build-up3bu5e5/c10/core/TensorImpl.h:1156.)
  return torch.max_pool1d(input, kernel_size, stride, padding, dilation, ceil_mode)
{% endraw %}

Make Predictions

{% raw %}
model.return_decomposition = False
outputs = trainer.predict(model, val_loader)

print("outputs[0][0].shape", outputs[0][0].shape)
print("outputs[0][1].shape", outputs[0][1].shape)
print("outputs[0][2].shape", outputs[0][2].shape)
/Users/kingtzolivares/Desktop/neuralforecast/neuralforecast/data/tsloader.py:47: UserWarning: This class wraps the pytorch `DataLoader` with a special collate function. If you want to use yours simply use `DataLoader`. Removing collate_fn
  'This class wraps the pytorch `DataLoader` with a '
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
outputs[0][0].shape torch.Size([7033, 24])
outputs[0][1].shape torch.Size([7033, 24])
outputs[0][2].shape torch.Size([7033, 24])
{% endraw %}

Forecast

{% raw %}
Y_forecast_df = Y_df[Y_df['ds']<'2016-12-27']
Y_forecast_df.tail()
unique_id ds y
87355 NP 2016-12-26 19:00:00 27.44
87356 NP 2016-12-26 20:00:00 27.11
87357 NP 2016-12-26 21:00:00 26.82
87358 NP 2016-12-26 22:00:00 26.65
87359 NP 2016-12-26 23:00:00 25.68
{% endraw %} {% raw %}
X_forecast_df = X_df[X_df['ds']<'2016-12-28']
X_forecast_df.tail()
unique_id ds Exogenous1 Exogenous2 week_day day_0 day_1 day_2 day_3 day_4 day_5 day_6
87379 NP 2016-12-27 19:00:00 0.133135 -0.566365 -0.67449 0.0 1.927498 0.0 0.0 0.0 0.0 0.0
87380 NP 2016-12-27 20:00:00 0.010193 -0.569435 -0.67449 0.0 1.927498 0.0 0.0 0.0 0.0 0.0
87381 NP 2016-12-27 21:00:00 -0.088980 -0.572021 -0.67449 0.0 1.927498 0.0 0.0 0.0 0.0 0.0
87382 NP 2016-12-27 22:00:00 -0.221603 -0.576345 -0.67449 0.0 1.927498 0.0 0.0 0.0 0.0 0.0
87383 NP 2016-12-27 23:00:00 -0.426087 -0.583618 -0.67449 0.0 1.927498 0.0 0.0 0.0 0.0 0.0
{% endraw %} {% raw %}
model.return_decomposition = False
forecast_df = model.forecast(Y_df=Y_forecast_df, X_df=X_forecast_df, S_df=S_df, batch_size=2)
INFO:root:Train Validation splits

INFO:root:                              ds                    
                             min                 max
unique_id sample_mask                               
FR        0           2011-01-09 2016-12-26 23:00:00
          1           2016-12-27 2016-12-27 23:00:00
NP        0           2013-01-01 2016-12-26 23:00:00
          1           2016-12-27 2016-12-27 23:00:00
INFO:root:
Total data 			87288 time stamps 
Available percentage=100.0, 	87288 time stamps 
Insample  percentage=0.05, 	48 time stamps 
Outsample percentage=99.95, 	87240 time stamps 

/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:91: LightningDeprecationWarning: Setting `Trainer(progress_bar_refresh_rate=1)` is deprecated in v1.5 and will be removed in v1.7. Please pass `pytorch_lightning.callbacks.progress.TQDMProgressBar` with `refresh_rate` directly to the Trainer's `callbacks` argument instead. Or, to disable the progress bar pass `enable_progress_bar = False` to the Trainer.
  f"Setting `Trainer(progress_bar_refresh_rate={progress_bar_refresh_rate})` is deprecated in v1.5 and"
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
/Users/kingtzolivares/Desktop/neuralforecast/neuralforecast/data/tsloader.py:47: UserWarning: This class wraps the pytorch `DataLoader` with a special collate function. If you want to use yours simply use `DataLoader`. Removing collate_fn
  'This class wraps the pytorch `DataLoader` with a '
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, predict_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  f"The dataloader, {name}, does not have many workers which may be a bottleneck."
/opt/anaconda3/envs/neuralforecast/lib/python3.7/site-packages/torch/nn/functional.py:3613: UserWarning: Default upsampling behavior when mode=linear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode)
{% endraw %} {% raw %}
forecast_df.head()
unique_id ds y
0 FR 2016-12-27 00:00:00 49.371140
1 FR 2016-12-27 01:00:00 43.137184
2 FR 2016-12-27 02:00:00 50.837498
3 FR 2016-12-27 03:00:00 45.940487
4 FR 2016-12-27 04:00:00 46.389229
{% endraw %}