This repository introduces Cross-D Conv, a novel convolutional operation designed to bridge the dimensional gap between 2D and 3D medical imaging datasets. By leveraging the Fourier domain for phase shifting, Cross-D Conv enables seamless weight transfer between 2D and 3D convolutional operations. This method addresses the challenge of multimodal data scarcity by utilizing abundant 2D data to enhance 3D model performance effectively.

@article{yavuz2024cross,
  title={Cross-D Conv: Cross-Dimensional Transferable Knowledge Base via Fourier Shifting Operation},
  author={Yavuz, Mehmet Can and Yang, Yang},
  journal={arXiv preprint arXiv:2411.02441},
  year={2024}
}

Access pretrained weights for Cross-D Conv on Hugging Face:

• Cross-D Conv Weights
• Enhanced MedMNIST Dataset

These weights are optimized for both 2D and 3D datasets, leveraging the Cross-D Conv operation for enhanced medical imaging performance. This dataset includes a curated collection of medical imaging data across multiple modalities (CT, MRI, Ultrasound) to support multimodal and cross-dimensional training.

• Cross-Dimensional Weight Transfer: Facilitates smooth transfer between 2D and 3D convolutional weights.
• Efficient Pretraining: Utilizes abundant 2D datasets (e.g., RadImagenet) to enhance 3D model performance.
• Multimodal Data Support: Designed for both 2D images and 3D volumetric medical datasets across CT, MRI, and ultrasound modalities.
• Superior Performance: Demonstrates statistically significant improvements over traditional convolution methods in diverse datasets.

To train the model using distributed training:

torchrun --nproc_per_node=8 --standalone train.py \
    --data-path /path/to/radimagenet \
    --workers 32 \
    --batch-size 32 \
    --sync-bn \
    --rot

The model achieves state-of-the-art performance in cross-dimensional medical image analysis, demonstrating superior feature quality assessment compared to conventional methods.

This project is licensed under the MIT License.