Welcome to the SNMMI AI Challenge 2023

The SNMMI AI Task Force—with support from the Physics, Instrumentation, and Data Science Council—has organized a Radiomics and Machine Learning (ML) Challenge. The Radiomics Challenge consists of providing participants with a fully anonymized dataset containing radiomics features and outcome or survival data (csv file). See here to find more information about the dataset and the challenge. People who register can use the dataset to train a ML model to predict survival. After some time, you will be requested to externally validate your model with another dataset with blinded survival information and your model predictions will be evaluated by the AI challenge team. The models that perform best will recognized and the participant(s) will receive a small monetary prize and a certificate.

$7,000 in prize money!

• There will be 2 winners, 2 runners-up, and 2 honorary mentions based on the best performing model in terms of accuracy and based on best classification. For more details, also on the performance metrics used to assess best models, please have a look here.
• Two first places - $2,000
• Two second places - $1,000
• Honorable mentions (2) - $500

The top teams will be invited to present their work at the SNMMI Annual Meeting. A manuscript summarizing the challenge results will be submitted for publication following completion of the challenge which will summarize the approaches taken by participating teams. Note that participants will not be coauthors, but will be acknowledged (with consent) on the challenge report; the report will only briefly describe results and methods so that participants can publish their own manuscripts based on the AI Machine Learning Challenge.

The AI Challenge is sponsored by the Physics, Instrumentation, and Data Science Council and the Artificial Intelligence Task Force.

Proposed Timeline

• Challenge opens July 1, 2023. More detailed information will be posted shortly.
• Registration possible through the end of October 2023.
• Distribution of radiomics dataset for training starts mid-October 2023 (dataset will be provided to participant by the registered e-mail address).
• Distribution of (blinded) radiomics dataset for (external) validation will be distributed from November 1, 2023 (this dataset will also be provided to the registered participants by e-mail upon request. Reminders will be sent around mid-November).
• Reporting of results of external test to the challenge team until the end of 2023.
• December 2023/January 2024 evaluation of submitted external blinded test results by challenge team.
• Spring 2024, announcement of three best performing teams/participants.

Terms and Conditions

• Anonymous participation is not allowed.
• If you participate as a team, then registration may be done by one representative being responsible for the team participation and activities.
• Participants from the host institution and colleagues of the organizers may participate, but they will not be eligible for prizes or recognition as a ranking participant.
• Entry by commercial entities or companies is permitted but should be disclosed.
• Once participants submit their outputs to the Challenge organizers, they will be considered fully vested in the challenge, so that their performance results may become part of presentations, publications, or subsequent analyses derived from the Challenge at the discretion of the organizers.
• The downloaded datasets or any data derived from these datasets, may not be given or redistributed under any circumstances to persons not belonging to the registered team.
• The full data, including reference data associated with the test set cases, is expected to be made publicly available after the publication of the Challenge. Until the official public release of the data, data downloaded from this site or provided to you by the organizers of the challenge may only be used for the purpose of preparing an entry to be submitted for the AI Challenge. The data may not be used for other purposes in scientific studies and may not be used to train or develop other algorithms, including but not limited to algorithms used in commercial products.
• After publication of the AI challenge analysis by the organizers, participants are permitted to publish their methods and findings with acknowledgement of the SNMMI AI Taskforce and a reference to the AI challenge publication by the AI challenge organizers. Participants will be informed by the AI challenge organizers when individual participant publication will be permitted.