First Results and Analysis:

Prasanna Porwal, Samiksha Pachade, Manesh Kokare, Girish Deshmukh, Jaemin Son, Woong Bae, Lihong Liu, et al. "IDRiD: Diabetic Retinopathy–Segmentation and Grading Challenge." Medical image analysis 59 (2020): 101561. DOI: https://doi.org/10.1016/j.media.2019.101561

Onsite Challenge Results:

                                                 Subchallenge - 2
Rank DR and DME Grading
  Team NameDR (Ac)DME (Ac) DR+DME (Ac)
1 LzyUNCC0.7476
0.8058
0.6311
2 VRT0.5922
0.8155
0.5534
3 Mammoth0.5437
0.8350
0.5146
4 HarangiM10.5534
0.7476
0.4757
4 AVSASVA0.5534
0.8058
0.4757
5 HarangiM20.4757
0.7282
0.4078
Ac - Accuracy

Note regarding DR+DME (Ac): Challenge is to detect DR and DME both simultaneously. e.g. For An Image -> DR (Grade = 3) and DME(Grade = 2)

If participating solution satisfies both conditions then it is counted as one, else zero. Similarly, for all images, it is computed to get the total no of true (one) instances. The final count (total no of true instances) is divided by a total number of images to get the average accuracy.

                                                    Subchallenge - 3
Rank Optic Disc Detection
  Team Name ED
1 DeepDR 21.072
2 VRT 33.538
3 ZJU-Bll-SGEX 33.875
4 SDNU 36.22
 
Rank Fovea Detection
  Team Name ED
1 DeepDR 64.492
2 VRT 68.466
3 SDNU 85.40
4 ZJU-Bll-SGEX 570.133
 
Rank Optic Disc Segmentation
  Team Name JD
1 ZJU-Bll-SGEX 0.9338
2 VRT 0.9305
3 IITkgpKLIV 0.8572
4 SDNU 0.7892
ED - Euclidean Distance
JD - Jaccard index

Online Challenge Results:

Subchallenge - 1
Sr. No.
Contact Person
Team Member(s) Affiliation(s)
Team Name
MA Score
RANK
HE
Score
RANK
SE
Score
RANK
EX
Score
RANK
1
Jaemin Son
VUNO Inc., Seoul, Republic of Korea
VRT
0.4951
2
0.6804
1
0.6995
1
0.7127
11
2
Liu Lihong
Ping An Technology (Shenzhen) CO.,Ltd, China
PATech
0.474
3
0.649
2
-
 
0.885
1
3
Fengyan Wang
iFLYTEK Research, China.
iFLYTEK-MIG
0.5017
1
0.5588
3
0.6588
3
0.8741
2
4
Yunzhi Wang
University of Oklahoma, US 
SOONER
0.4003
5
0.5395
4
0.5369
7
0.739
10
5
Yoon Ho Choi
Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, Republic of Korea.
SAIHST
-
 
-
 
-
 
0.8582
3
6
Zhongyu Li
University of North Carolina at Charlotte
lzyuncc_fusion
-
 
-
 
0.6259
4
0.8202
4
7
Xiaodan Sui
Shandong Normal University, China
Southern Medical University, China
The University of North Carolina at Chapel Hill, USA
SDNU
0.4111
4
0.4572
7
0.5374
6
0.5018
17
8
Hyunjun Eun
KAIST, Republic of Korea. 
CIL
0.392
6
0.4886
5
0.5024
8
0.7554
8
9
Manu Agarwal 
Indian Institute of Technology Roorkee, India
MedLabs
0.3397
8
0.3705
8
0.2637
10
0.7863
5
10
Zhengyi Li
Peking University, China 
AIMIA
0.3792
7
0.3283
10
0.2733
9
0.7662
6
11
Zhongyu Li
University of North Carolina at Charlotte
lzyuncc
-
 
-
 
0.6607
2
0.7615
7
12
Deepak Anand
Indian Institute of Technology Bombay, India
MEDAL_IITB
0.2704
12
0.3705
9
-
 
0.7471
9
13
Qiong Bai
 
Edoctor
-
 
-
 
-
 
0.7106
12
14
Tengfei Li
The University of Texas MD Anderson Cancer Center, USA         Rice University, Houston, TX, USA                                              Southern Medical University, Guangzhou, China                            The University of North Carolina, Chapel Hill, NC, USA
BIGS2
0.2912
11
0.1408
12
0.0479
13
0.6759
13
15
Chih-Hsuan Liu
National Tsing Hua University, Taiwan
NTHU-CVLab
0.2958
10
0.2915
11
0.2554
11
0.5965
14
16
Oindrila Saha
Indian Institute of Technology Kharagpur, India 
IITKgpKLIV
0.0059
14
0.0829
13
0.1823
12
0.5498
15
17
Xingzheng Lyu
Zhejiang University, China
National University of Singapore, Singapore
Bioinformatics Institute, A*STAR, Singapore
Beijing Shanggong Medical Technology Co., Ltd., China 
ZJU-BII-SGEX
0.2601
13
0.477
6
0.5467
5
0.5094
16
18
Xuechen Li
Shenzhen University, China 
SZU
-
 
-
 
-
 
0.3718
18
19
Ling Dai
Shanghai Jiao Tong Univ. , China 
deepdr
-
 
-
 
-
 
0.2052
19
20
Mhd Hasan Sarhan
Technical University of Munich, Germany
FAT1
0.3356
9
-
 
-
 
-
 
21
Junyan Wu
Cleerly, US
Virginia Tech, US 
University at Buffalo, US
 
*
 
*
 
*
 
*
 
22
Sagar B Hathwar
PES University, India 
Error 404
-
 
-
 
-
 
!
 

 

*Data inconsistent with the format mentioned on the website

! Test results not submitted.

Note: Ranking of the teams for Subchallenge-1 is done based on the single score computed by finding the area under the Positive Predictive Value (Precision) and Sensitivity (Recall) curve using the test data of apparent retinopathy.

Subchallenge - 2
Sr. No. Contact Person Team Member(s) Affiliation(s) Team Name Accuracy
1
Junyan Wu
Cleerly, US
Virginia Tech, US 
University at Buffalo, US
Mammoth
0.9322
2 Xiaodan Sui Shandong Normal University, China
Southern Medical University, China
The University of North Carolina at Chapel Hill, USA 
SDNU 0.8789
3 Balazs Harangi University of Debrecen, Hungary HarangiM1 0.8741
4 Balazs Harangi University of Debrecen, Hungary HarangiM2 0.8692
5 Varghese Alex Individual Researcher, India  AVSASVA 0.8426
6 Jaemin Son VUNO Inc., Seoul, Republic of Korea VRT 0.7554
7 Zhongyu Li University of North Carolina at Charlotte lzyuncc 0.5327
8 Siddhesh Thakur Shri Guru Gobind Singhji Institute of  Engineering & Technology, India Py 0.5206
9 Xuechen Li Shenzhen University, China  SZU 0.4964
10 K V Sai Sundar Sri Sathya Sai Institute of Higher Learning, India SS 0.4843
11 Xingzheng Lyu Zhejiang University, China
Bioinformatics Institute, A*STAR, Singapore                                                National University of Singapore, Singapore
Beijing Shanggong Medical Technology Co., Ltd., China
ZJU-BII-SGEX 0.3995
12 Ling Dai Shanghai Jiao Tong Univ. , China  deepdr 0.3148
13 Chih-Hsuan Liu National Tsing Hua University, Taiwan  NTHU-CVLab 0.1719

 

Subchallenge - 3
Sr. No.
Contact Person
 Team Member(s) Affiliation(s)
Team Name
Optic Disc Detection  (ED)
Fovea Detection (ED)
Optic Disc   Segmentation (J)
1 Xingzheng Lyu Zhejiang University, China
National University of Singapore, Singapore
Bioinformatics Institute, A*STAR, Singapore
Beijing Shanggong Medical Technology Co., Ltd., China 
ZJU-BII-SGEX 25.6157 (1) 45.8959 0.9826 (1)
2 Jaemin Son VUNO Inc., Seoul, Republic of Korea VRT 26.6377 (2) 39.2085 (3) 0.0853
3 Manu Agarwal Indian Institute of Technology Roorkee, India MedLabs 48.433 33.253 (1) 0.4904
4 Xiaodan Sui Shandong Normal University, China
Southern Medical University, China
The University of North Carolina at Chapel Hill, USA 
SDNU 26.9181 (3) 72.6426 0.8656
5 Ling Dai Shanghai Jiao Tong Univ. , China  deepdr 29.0037 39.1054 (2) -
6 Oindrila Saha Indian Institute of Technology Kharagpur, India IITKgpKLIV 65.9374 - 0.9640 (2)
7 Chaitanya Kaul University of York, United Kingdom AI@UOY - - 0.9605 (3)
8 Tânia Melo FEUP & INESC TEC, Porto, Portugal  CBER 30.8007 53.5337 0.8910
9 Alakh Desai CVIT, IIIT Hyderabad, India  Alchemist 30.3629 63.1369 0.8021
10 Zhengyi Li Peking University, China  AIMIA 43.8468 67.43 0.9522
11 Xuechen Li Shenzhen University, China  SZU 32.62 52.0077 *
12 Yunzhi Wang University of Oklahoma, US SOONER 37.92 97.0545 0.8170
13 Deepak Anand Indian Institute of Technology Bombay, India Medal IITB 48.433 - 0.9128
14 S M Jaisakthi VIT University, Vellore, India                                        SSN College of Engineering, Chennai, India SSNMLRG - RUN 2 54.2101 - 0.6651
15 Qunwei Hu University of Science and Technology of China, China One 59.7293 69.0946 0.8408
16 Sagar B Hathwar PES University, India  Error 404 61.847 114.5051 0.6916
17
Santhosh Kumar Sukumar
 HTIC, Indian Institute of Technology Madras, India 
HTIC_IIT_Madras
(DL_Method)
61.9589
-
*
18 S M Jaisakthi VIT University, Vellore, India                                        SSN College of Engineering, Chennai, India SSNMLRG - RUN 1 115.2691 - 0.6803
19 Kartik Teotia MIT Manipal, India MIT Manipal 154.4787 -  
20 Sidhartha Dey Manipal Institute of Technology                                    Rice University, USA Affine Snakes - - 0.8286
21
Santhosh Kumar Sukumar
HTIC, Indian Institute of Technology Madras, India
HTIC_IIT_Madras
(Classical_Method)
161.9568
 
0.6122
22 Fatemeh Zabihollahy  Carleton University, Canada Carleton University - Medical Image Analysis - - 0.4594

*Data inconsistent with the format mentioned on the website

ED - Euclidean Distance

Note: Ranking of the teams for Optic Disc Segmentation is done based only on Jaccard Index (J) as the corresponding dice indices or sensitivity are not differing the rankings.