This is a method for time series forecasting. Short fixed-length inputs are the main bottleneck of deep learning methods in long time-series forecasting tasks. Prolonging input length causes overfitting, rapidly deteriorating accuracy(left pic). Our method can adapt to longer context, thereby achieving better performance(right pic).
However, the sampling approach has three main limitations: 1) Insufficient context: Shorter inputs lead to prediction errors as the model lacks sufficient context for effective learning. 2) Non-stationarity: Downsampling introduces additional non-stationary components, increasing complexity. 3) Limited applicability: Multiscale downsampling requires specialized modules (like PDM) for aggregation, adding overhead and reducing scalability.
We first need to understand the cause of overfitting, which is:
**A single, fixed patch size causes the model to focus only on the temporal patterns at a primary scale of the sequence (e.g., 24 points, or one day), thus not requiring a long context (512). **
Therefore, it is necessary to introduce multi-resolution analysis and shorten the context of smaller resolutions to avoid overfitting. This is a straightforward implementation, there are many better methods such as multi-resolution attention.
This method employs a wide, low-coupling architecture instead of a deep network, which intuitively seems suboptimal. However, some facts about current forecasting models are: (1) the number of layers is usually small (1~3), and (2) the encoder layers between the input and output layers have a relatively small impact on the prediction results (~10%). These facts compel us to consider adopting a wide structure and making improvements at the embedding layer and the prediction layer rather than the slightly-involved encoding layer(input-output layer).
The main results, MAE (Mean Absolute Error) and MSE (Mean Squared Error), with lower values indicating better predictive performance. Multivariate Benchmark
- install dependencies
pip install -r requirements.txt
-
prepare data You can obtain the well pre-processed datasets from [Google Drive] or [Baidu Drive], Then place the downloaded data in the folder
./dataset. thanks to [thuml] -
To reproduce all results in the paper, run following scripts to get corresponding results:
bash scripts/ETTm2.sh
bash scripts/ETTm1.sh
bash scripts/weather.sh
bash scripts/electricity.sh
bash scripts/solar.sh
bash scripts/traffic.sh
- To get result on single set such as etth1 of our model, you can run the following scripts:
python -u run_longExp.py --is_training 1 --model_id test --data 'ETTm1' --data_path 'ETTm1.csv' --pred_len 720 --n_heads 4 --d_model 16 --d_ff 128 --e_layer 1 --features 'M' --fc_dropout 0.8 --dropout 0.8 --pct_start 0.4 --batch_size 128 --model MTETST --patience 3 --seq_len 960 --period_list 96