--- title: Long-Horizon Datasets keywords: fastai sidebar: home_sidebar summary: "Download and wrangling utility for long-horizon datasets." description: "Download and wrangling utility for long-horizon datasets." nb_path: "nbs/data_datasets__long_horizon.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %}

Long-Horizon Meta Information

{% raw %}

class ETTh1[source]

ETTh1(freq:str='H', name:str='ETTh1', n_ts:int=1)

The ETTh1 dataset monitors an electricity transformer from a region of a province of China including oil temperature and variants of load (such as high useful load and high useless load) from July 2016 to July 2018 at an hourly frequency.

{% endraw %} {% raw %}

class ETTh2[source]

ETTh2(freq:str='H', name:str='ETTh2', n_ts:int=1)

The ETTh2 dataset monitors an electricity transformer from a region of a province of China including oil temperature and variants of load (such as high useful load and high useless load) from July 2016 to July 2018 at an hourly frequency.

{% endraw %} {% raw %}

class ETTm1[source]

ETTm1(freq:str='15T', name:str='ETTm1', n_ts:int=7)

The ETTm1 dataset monitors an electricity transformer from a region of a province of China including oil temperature and variants of load (such as high useful load and high useless load) from July 2016 to July 2018 at a fifteen minute frequency.

{% endraw %} {% raw %}

class ETTm2[source]

ETTm2(freq:str='15T', name:str='ETTm2', n_ts:int=7)

The ETTm2 dataset monitors an electricity transformer from a region of a province of China including oil temperature and variants of load (such as high useful load and high useless load) from July 2016 to July 2018 at a fifteen minute frequency.

Reference:
Zhou, et al. Informer: Beyond Efficient Transformer
for Long Sequence Time-Series Forecasting. AAAI 2021.
https://arxiv.org/abs/2012.07436
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class ECL[source]

ECL(freq:str='15T', name:str='ECL', n_ts:int=321)

The Electricity dataset reports the fifteen minute electricity consumption (KWh) of 321 customers from 2012 to 2014. For comparability, we aggregate it hourly.

Reference:
Li, S et al. Enhancing the locality and breaking the memory
bottleneck of Transformer on time series forecasting.
NeurIPS 2019. http://arxiv.org/abs/1907.00235.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class Exchange[source]

Exchange(freq:str='D', name:str='Exchange', n_ts:int=8)

The Exchange dataset is a collection of daily exchange rates of eight countries relative to the US dollar. The countries include Australia, UK, Canada, Switzerland, China, Japan, New Zealand and Singapore from 1990 to 2016.

Reference:
Lai, G., Chang, W., Yang, Y., and Liu, H. Modeling Long and
Short-Term Temporal Patterns with Deep Neural Networks.
SIGIR 2018. http://arxiv.org/abs/1703.07015.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class TrafficL[source]

TrafficL(freq:str='H', name:str='traffic', n_ts:int=862)

This large Traffic dataset was collected by the California Department of Transportation, it reports road hourly occupancy rates of 862 sensors, from January 2015 to December 2016.

Reference:
Lai, G., Chang, W., Yang, Y., and Liu, H. Modeling Long and
Short-Term Temporal Patterns with Deep Neural Networks.
SIGIR 2018. http://arxiv.org/abs/1703.07015.

Wu, H., Xu, J., Wang, J., and Long, M. Autoformer:
Decomposition Transformers with auto-correlation for
long-term series forecasting. NeurIPS 2021.
https://arxiv.org/abs/2106.13008.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class ILI[source]

ILI(freq:str='W', name:str='ili', n_ts:int=7)

This dataset reports weekly recorded influenza-like illness (ILI) patients from Centers for Disease Control and Prevention of the United States from 2002 to 2021. It is measured as a ratio of ILI patients versus the total patients in the week.

Reference:
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer:
Decomposition Transformers with auto-correlation for
long-term series forecasting. NeurIPS 2021.
https://arxiv.org/abs/2106.13008.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class Weather[source]

Weather(freq:str='10M', name:str='weather', n_ts:int=21)

This Weather dataset contains the 2020 year of 21 meteorological measurements recorded every 10 minutes from the Weather Station of the Max Planck Biogeochemistry Institute in Jena, Germany.

Reference:
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer:
Decomposition Transformers with auto-correlation for
long-term series forecasting. NeurIPS 2021.
https://arxiv.org/abs/2106.13008.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class LongHorizon[source]

LongHorizon(source_url:str='https://nhits-experiments.s3.amazonaws.com/datasets.zip')

This Long-Horizon datasets wrapper class, provides with utility to download and wrangle the following datasets: ETT, ECL, Exchange, Traffic, ILI and Weather.

  • Each set is normalized with the train data mean and standard deviation.
  • Datasets are partitioned into train, validation and test splits.
  • For all datasets: 70%, 10%, and 20% of observations are train, validation, test, except ETT that uses 20% validation.
{% endraw %} {% raw %}
{% endraw %} {% raw %}
for group, meta in LongHorizonInfo:
    print(group)

group = 'ETTm2'
y_df, x_df, s_df = LongHorizon.load(directory='data', group=group, cache=False)
n_series = len(np.unique(y_df.unique_id.values))
ex_vars = x_df.columns.to_list()
ex_vars.remove('unique_id')
ex_vars.remove('ds')

display_str  = f'Group: {group} '
display_str += f'n_series: {n_series} '
print('\n' + display_str)
{% endraw %}