Benchmarking CERR Radiomics with other software

Comparing CERR's radiomic computations against IBSI benchmarks

IBSI 1

The IBSI 1 initiative established a standardized workflow and reference values to calibrate various software implementations of commonly-used radiomic features. We provide a test script to compare features computed using CERR against IBSI benchmarks.

• Dataset : The reference Lung CT phantom provided by the IBSI was imported to planC format, distributed with CERR.

• Settings: JSON-format files IBSIconfigC_avg and IBSIconfigC_merge for feature extraction using CERR and matching IBSI configuration C are provided.

• Run IBSI-1 calculations using CERR

config = 'C'; %Either 'A' or 'C'
cerrFeatS = calcIBSI1PhantomRadiomics(config);

• Compare calculations to benchmark: The following call returns features computed on the reference phantom for configuration 'C' using CERR, the corresponding benchmark values from the IBSI, and the percentage differences between the two.

[cerrFeatS,IBSIfeatS,pctDiffS] = compareRadiomicsWithIBSI1OrigImgWithInterp;

IBSI-1 computations using Pyradiomics (via CERR)

• Settings: YAML format files pyRadConfigC_avg.yaml and pyRadConfigC_merge.yaml matching IBSI configuration C are provided.

The following returns percentage differences in feature values computed using Pyradiomics from the IBSI benchmark values computed on the reference dataset for configuration 'C'.

diffPyRadS = comparePyradWithIBSIOrigImgWithInterp;

IBSI 2

The IBSI 2 initiative aims to standardize the implementation of commonly-used convolutional image filters.

Phase-1

• Dataset : Synthetic phantoms distributed by the IBSI were imported to planC archives

• Settings: JSON-format files (IBSI2-1ID*.json) are provided to apply convolutional filters matching phase 1 specifications.

• Unit testing: testConvolutionalFilters.m compares filter responses as currently implemented in CERR with those submitted to IBSI2 phase-1 to ensure continued compliance.

Phase-2

• Dataset : The CT phantom distributed by the IBSI was imported to planC format.

• Settings: JSON-format files(IBSI2-2ID*.json) are provided with settings for feature extraction matching phase 2 specifications.

• Unit testing: testIBSI2Features.m compares feature extraction as currently implemented in CERR with those submitted to IBSI2 phase-2 to ensure continued compliance.

Phase-3

• Settings: JSON-format files(IBSI2-3ID*.json) with settings for feature extraction following phase 3 specifications are provided.

Comparing CERR radiomics with PyRadiomics

See documentation for sample tests comparing radiomics calculations between CERR and Pyradiomics.

Citation

• Iyer, A., E. LoCastro, H. Veeraraghavan, J. Deasy, and A. Apte (2022). IBSI-Compatible Convolutional Image Texture Filters in CERR, Med. Phys. 49 (6), pp. E686.