TrackSSM: A General Motion Predictor by State-Space Model

Bin Hu, Run Luo, Zelin Liu, Cheng Wang, Wenyu Liu

arXiv 2409.00487

• Submitting the paper on Arxiv at Sep 4 2024.

Creating a new environment.

Running: pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118

Compiling mamba 1.2.0 post1, ensuring triton == 2.1.0

Running: pip install -r requirement.txt

cd external/YOLOX,run: pip install -r requirements.txt && python setup.py develop

Download MOT17, MOT20, DanceTrack, SportsMOT and put them under ROOT/ in the following structure. The structure of the MIX dataset follows the method used in DiffMOT:

ROOT
   |
   |——————TrackSSM(repo)
   |                         
   |——————mot(MIX)
   |        └——————train(MOT17 train set and MOT20 train set)
   |        └——————test(MOT17 test set and MOT20 test set)
   |——————DanceTrack
   |           └——————train
   |           └——————train_seqmap.txt
   |           └——————test
   |           └——————test_seqmap.txt
   |           └——————val
   |           └——————val_seqmap.txt
   └——————SportsMOT
              └——————train
              └——————test
              └——————val
              └——————splits_txt
                         └——————train.txt
                         └——————val.txt
                         └——————test.txt

and then, run

python dancetrack_data_process.py
python sports_data_process.py
python mot_data_process.py

Refer to Detection Model.

Refer to : MOT17-61.4 HOTA, DanceTrack-57.7 HOTA, SportsMOT-74.4 HOTA.

Refer to ByteTrack.

• Changing the data_dir in config
• Training on the MIX, DanceTrack and SportsMOT:

python main.py --config ./configs/dancetrack.yaml
python main.py --config ./configs/sportsmot.yaml
python main.py --config ./configs/mot.yaml

Notes:

• For MIX, we should unenable line 60 in motion_decoder.py.
• For MIX and DanceTrack, we should unenable GIoU loss in motion_decoder.py.
• For SportsMOT, we should use both GIoU loss and smooth L1 loss in motion_decoder.py.

python main.py --config ./configs/mot17_test.yaml

python main.py --config ./configs/dancetrack_test.yaml

python main.py --config ./configs/sportsmot_test.yaml

Notes:

• For tracking on MOT17, we should unenable line 60 in motion_decoder.py.
• Before perform tracking process, change det_dir, info_dir and save_dir in config files.
• The use_detection_model is an optional item. When making the use_detection_model project effective, the detector will participate in the process of tracking inference, not just the motion model.
• The interval the length of the historical trajectory involved in training and inference.

@misc{trackssm,
      title={TrackSSM: A General Motion Predictor by State-Space Model}, 
      author={Bin Hu and Run Luo and Zelin Liu and Cheng Wang and Wenyu Liu},
      year={2024},
      eprint={2409.00487},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2409.00487}, 
}

A large part of the code is borrowed from Mamba,DiffMOT, FairMOT, ByteTrack. Many thanks for their wonderful works.