Rice Science ›› 2022, Vol. 29 ›› Issue (5): 489-498.DOI: 10.1016/j.rsci.2022.02.003
• Experimental Technique • Previous Articles
Pandia Rajan Jeyaraj(), Siva Prakash Asokan, Edward Rajan Samuel Nadar
Received:
2021-09-18
Accepted:
2022-02-15
Online:
2022-09-28
Published:
2022-07-14
Contact:
Pandia Rajan Jeyaraj
Pandia Rajan Jeyaraj, Siva Prakash Asokan, Edward Rajan Samuel Nadar. Computer-Assisted Real-Time Rice Variety Learning Using Deep Learning Network[J]. Rice Science, 2022, 29(5): 489-498.
Add to citation manager EndNote|Ris|BibTeX
Research problem | Method used | Number of data | Research outcome | Reference |
---|---|---|---|---|
Real-time visual inspection system for grading fruits | Deep learning models including ResNet and DenseNet | 360 datasets were used to grade self-collected datasets of 150 apples and 210 bananas | Average accuracies were 99.2% for apples and 98.6% for bananas | Ismail and Malik, |
Fish classification technique using deep learning networks with naive Bayesian type fusion | CNN-AlexNet was trained by using the transfer learning approach | 15 million labelled high- resolution images categorized into 22 000 classes | Classification accuracy of 98.64% for ‘Fish-Pak’ image dataset | Abinaya et al, |
To identify and count wheat mites from digital images | Three-step deep learning method by Region Proposal Network | 256-d vector was generated from presented digital images | Accuracies of 94.6% and 96.4% for two different variants in architecture | Chen et al, |
To provide a solution for food material recognition employing deep transfer learning | An embedded partial-and- imbalanced domain adaptation technique in the deep learning model | Office-31 and Meal-300 with 31 classes of each | Average accuracy of 97.71% for two step process and 96.28% for shared network | Xiao et al, |
To detect and identify bruised apples by fusing 3D deep features | An apple grading system was designed by fusing multi- dimensional features | ImageNet dataset of 1.28 million images over 1 000 generic classes of apples | Identification rate of 97.67% for bruised apple | Hu et al, |
To aerial visual perception for yellow rust disease monitoring in winter wheat | An automated rust disease monitoring framework was proposed by U-Net deep learning network | Five GeoTIFF images of the covered area with cropped region of interest | An average accuracy of 91.3%, recall of 92.6% with 0.92 F1-score | Su et al, |
To classify fruit automatically I, supermarket employing deep learning model | Fine-tuned visual geometry group-16 pre-trained deep convolution neural network | Two color image datasets of different fruit images | Classification accuracies of 99.49% and 99.75% for datasets 1 and 2, respectively | Shamim Hossain et al, |
Table 1. Summary of literature reviews similar to rice defect classification.
Research problem | Method used | Number of data | Research outcome | Reference |
---|---|---|---|---|
Real-time visual inspection system for grading fruits | Deep learning models including ResNet and DenseNet | 360 datasets were used to grade self-collected datasets of 150 apples and 210 bananas | Average accuracies were 99.2% for apples and 98.6% for bananas | Ismail and Malik, |
Fish classification technique using deep learning networks with naive Bayesian type fusion | CNN-AlexNet was trained by using the transfer learning approach | 15 million labelled high- resolution images categorized into 22 000 classes | Classification accuracy of 98.64% for ‘Fish-Pak’ image dataset | Abinaya et al, |
To identify and count wheat mites from digital images | Three-step deep learning method by Region Proposal Network | 256-d vector was generated from presented digital images | Accuracies of 94.6% and 96.4% for two different variants in architecture | Chen et al, |
To provide a solution for food material recognition employing deep transfer learning | An embedded partial-and- imbalanced domain adaptation technique in the deep learning model | Office-31 and Meal-300 with 31 classes of each | Average accuracy of 97.71% for two step process and 96.28% for shared network | Xiao et al, |
To detect and identify bruised apples by fusing 3D deep features | An apple grading system was designed by fusing multi- dimensional features | ImageNet dataset of 1.28 million images over 1 000 generic classes of apples | Identification rate of 97.67% for bruised apple | Hu et al, |
To aerial visual perception for yellow rust disease monitoring in winter wheat | An automated rust disease monitoring framework was proposed by U-Net deep learning network | Five GeoTIFF images of the covered area with cropped region of interest | An average accuracy of 91.3%, recall of 92.6% with 0.92 F1-score | Su et al, |
To classify fruit automatically I, supermarket employing deep learning model | Fine-tuned visual geometry group-16 pre-trained deep convolution neural network | Two color image datasets of different fruit images | Classification accuracies of 99.49% and 99.75% for datasets 1 and 2, respectively | Shamim Hossain et al, |
Fig. 1. Block diagram of proposed real-time rice defect classification system. AIY, Artificial intelligence yourself; CNN, Convolutional neural network.
Dataset | Orientation number | Defected rice image number | Healthy rice image number |
---|---|---|---|
Kaggle-labelled rice | 107 | 520 | 230 |
Data-world Asian variety | 219 | 425 | 190 |
CASC | 187 | 182 | 105 |
Table 2. Training image dataset in detailed description.
Dataset | Orientation number | Defected rice image number | Healthy rice image number |
---|---|---|---|
Kaggle-labelled rice | 107 | 520 | 230 |
Data-world Asian variety | 219 | 425 | 190 |
CASC | 187 | 182 | 105 |
Paddy rice defect class | No. of images selected | No. of standard images | Multiplication factor | No. of total images for augmentation | New image count |
---|---|---|---|---|---|
Broken rice | 30 | 10 | 3 | 120 | 160 |
Black spot rice | 50 | 25 | 4 | 300 | 375 |
Wrinkled rice | 25 | 15 | 2 | 80 | 120 |
Total | 105 | 50 | 500 | 655 |
Table 3. Paddy rice defect dataset image augmentation in detail.
Paddy rice defect class | No. of images selected | No. of standard images | Multiplication factor | No. of total images for augmentation | New image count |
---|---|---|---|---|---|
Broken rice | 30 | 10 | 3 | 120 | 160 |
Black spot rice | 50 | 25 | 4 | 300 | 375 |
Wrinkled rice | 25 | 15 | 2 | 80 | 120 |
Total | 105 | 50 | 500 | 655 |
Fig. 4. Concept of proposed model pre-training for feature extraction by ResNet-512 (residual learning). M(x) indicates paddy image; P(x) indicates residual; x indicates the number of input images; and X indicates the vector form of input images.
Defect category | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score | AUC |
---|---|---|---|---|---|
Standard Kaggle Vietnam rice dataset | |||||
Broken rice | 98.6 | 96.4 | 97.8 | 98.4 | 0.91 |
Wrinkled rice | 94.5 | 97.2 | 98.4 | 99.3 | 0.87 |
Black spot rice | 97.4 | 95.4 | 99.4 | 91.2 | 0.96 |
Normal rice | 98.8 | 98.4 | 97.4 | 95.4 | 0.94 |
Designed prototype system | |||||
Broken rice | 98.5 | 91.3 | 90.6 | 91.2 | 0.94 |
Wrinkled rice | 92.3 | 96.4 | 94.2 | 97.3 | 0.86 |
Black spot rice | 96.3 | 92.3 | 92.1 | 96.5 | 0.91 |
Normal rice | 95.4 | 95.1 | 96.3 | 98.4 | 0.93 |
Table 4. Experimental results from training Kaggle Vietnam rice dataset and real-time image acquired on designed prototype system.
Defect category | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score | AUC |
---|---|---|---|---|---|
Standard Kaggle Vietnam rice dataset | |||||
Broken rice | 98.6 | 96.4 | 97.8 | 98.4 | 0.91 |
Wrinkled rice | 94.5 | 97.2 | 98.4 | 99.3 | 0.87 |
Black spot rice | 97.4 | 95.4 | 99.4 | 91.2 | 0.96 |
Normal rice | 98.8 | 98.4 | 97.4 | 95.4 | 0.94 |
Designed prototype system | |||||
Broken rice | 98.5 | 91.3 | 90.6 | 91.2 | 0.94 |
Wrinkled rice | 92.3 | 96.4 | 94.2 | 97.3 | 0.86 |
Black spot rice | 96.3 | 92.3 | 92.1 | 96.5 | 0.91 |
Normal rice | 95.4 | 95.1 | 96.3 | 98.4 | 0.93 |
Classifier model | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score | Computational time (s) |
---|---|---|---|---|---|
Support vector machine | 81.5 ± 0.2 | 83.4 ± 0.5 | 86.4 ± 0.2 | 80.2 | 896 |
GoogleNet | 89.4 ± 0.2 | 87.3 ± 0.1 | 91.3 ± 0.1 | 87.7 | 245 |
ResNet-152 | 84.5 ± 0.3 | 90.4 ± 0.1 | 90.1 ± 0.3 | 90.1 | 586 |
Proposed network | 96.4 ± 0.1 | 94.3 ± 0.1 | 97.4 ± 0.1 | 97.2 | 102 |
Table 5. Performance comparison of proposed network with other classifier networks for rice defect Kaggle dataset.
Classifier model | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score | Computational time (s) |
---|---|---|---|---|---|
Support vector machine | 81.5 ± 0.2 | 83.4 ± 0.5 | 86.4 ± 0.2 | 80.2 | 896 |
GoogleNet | 89.4 ± 0.2 | 87.3 ± 0.1 | 91.3 ± 0.1 | 87.7 | 245 |
ResNet-152 | 84.5 ± 0.3 | 90.4 ± 0.1 | 90.1 ± 0.3 | 90.1 | 586 |
Proposed network | 96.4 ± 0.1 | 94.3 ± 0.1 | 97.4 ± 0.1 | 97.2 | 102 |
Fig. 8. Results of proposed model. A, AUC bar plot comparison between various classifiers and proposed classifier for Kaggle training dataset. B, AUC bar plot comparison between various classifiers and proposed classifier for real-time testing. C, Training accuracy comparison for proposed classifier and conventional classifiers. D, Training loss comparison between proposed classifier and conventional classifiers. AUC, Area under curve; SVM, Support vector machine.
Classifier model | Defect category | TPR | FPR | TNR | FNR | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score |
---|---|---|---|---|---|---|---|---|---|
Support vector machine | Broken rice | 84 | 4 | 361 | 21 | 84.3 | 79.3 | 81.5 | 88.3 |
Black spot rice | 12 | 12 | 210 | 11 | 91.6 | 77.3 | 84.3 | 89.6 | |
Wrinkled rice | 5 | 8 | 180 | 6 | 79.8 | 81.5 | 83.2 | 90.1 | |
Normal rice | 65 | 5 | 155 | 15 | 84.3 | 85.3 | 81.9 | 87.3 | |
GoogleNet | Broken rice | 82 | 3 | 161 | 14 | 91.3 | 81.3 | 90.2 | 91.3 |
Black spot rice | 46 | 2 | 184 | 8 | 89.1 | 70.9 | 91.5 | 92.5 | |
Wrinkled rice | 13 | 10 | 146 | 13 | 78.1 | 79.2 | 90.6 | 90.5 | |
Normal rice | 45 | 6 | 138 | 10 | 77.3 | 76.5 | 86.7 | 89.3 | |
ResNet-152 | Broken rice | 116 | 7 | 141 | 5 | 84.6 | 93.4 | 83.9 | 81.3 |
Black spot rice | 421 | 10 | 172 | 15 | 87.3 | 91.3 | 91.5 | 91.1 | |
Wrinkled rice | 38 | 4 | 187 | 18 | 91.2 | 90.4 | 90.5 | 89.3 | |
Normal rice | 84 | 12 | 196 | 12 | 82.1 | 81.3 | 89.3 | 88.4 | |
Proposed network | Broken rice | 120 | 1 | 194 | 1 | 94.5 | 93.5 | 96.5 | 98.9 |
Black spot rice | 141 | 2 | 187 | 0 | 96.3 | 96.5 | 94.3 | 97.5 | |
Wrinkled rice | 181 | 0 | 191 | 0 | 97.5 | 95.4 | 97.5 | 99.4 | |
Normal rice | 95 | 1 | 184 | 3 | 98.8 | 97.7 | 98.9 | 98.6 |
Table 6. Comparison of performance indices for real-time rice sample and other classifiers with augmentation.
Classifier model | Defect category | TPR | FPR | TNR | FNR | Accuracy (%) | Sensitivity (%) | Specificity (%) | F1-score |
---|---|---|---|---|---|---|---|---|---|
Support vector machine | Broken rice | 84 | 4 | 361 | 21 | 84.3 | 79.3 | 81.5 | 88.3 |
Black spot rice | 12 | 12 | 210 | 11 | 91.6 | 77.3 | 84.3 | 89.6 | |
Wrinkled rice | 5 | 8 | 180 | 6 | 79.8 | 81.5 | 83.2 | 90.1 | |
Normal rice | 65 | 5 | 155 | 15 | 84.3 | 85.3 | 81.9 | 87.3 | |
GoogleNet | Broken rice | 82 | 3 | 161 | 14 | 91.3 | 81.3 | 90.2 | 91.3 |
Black spot rice | 46 | 2 | 184 | 8 | 89.1 | 70.9 | 91.5 | 92.5 | |
Wrinkled rice | 13 | 10 | 146 | 13 | 78.1 | 79.2 | 90.6 | 90.5 | |
Normal rice | 45 | 6 | 138 | 10 | 77.3 | 76.5 | 86.7 | 89.3 | |
ResNet-152 | Broken rice | 116 | 7 | 141 | 5 | 84.6 | 93.4 | 83.9 | 81.3 |
Black spot rice | 421 | 10 | 172 | 15 | 87.3 | 91.3 | 91.5 | 91.1 | |
Wrinkled rice | 38 | 4 | 187 | 18 | 91.2 | 90.4 | 90.5 | 89.3 | |
Normal rice | 84 | 12 | 196 | 12 | 82.1 | 81.3 | 89.3 | 88.4 | |
Proposed network | Broken rice | 120 | 1 | 194 | 1 | 94.5 | 93.5 | 96.5 | 98.9 |
Black spot rice | 141 | 2 | 187 | 0 | 96.3 | 96.5 | 94.3 | 97.5 | |
Wrinkled rice | 181 | 0 | 191 | 0 | 97.5 | 95.4 | 97.5 | 99.4 | |
Normal rice | 95 | 1 | 184 | 3 | 98.8 | 97.7 | 98.9 | 98.6 |
[1] | Abinaya N S, Susan D, Rakesh Kumar S. 2021. Naive Bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries. Ecol Inform, 61: 101248. |
[2] | Alfred R, Obit J H, Chin C P Y, Haviluddin H, Lim Y. 2021. Towards paddy rice smart farming: A review on big data, machine learning, and rice production tasks. IEEE Access, 9: 50358-50380. |
[3] | Cao J J, Sun T, Zhang W R, Zhong M, Huang B, Zhou G M, Chai X J. 2021. An automated zizania quality grading method based on deep classification model. Comput Electron Agric, 183: 106004. |
[4] | Chen P, Li W L, Yao S J, Ma C, Zhang J, Wang B, Zheng C H, Xie C J, Liang D. 2021. Recognition and counting of wheat mites in wheat fields by a three-step deep learning method. Neurocomputing, 437: 21-30. |
[5] | Guerrero J M, Ruz J J, Pajares G. 2017. Crop rows and weeds detection in maize fields applying a computer vision system based on geometry. Comput Electron Agric, 142: 461-472. |
[6] | Hu Z L, Tang J S, Zhang P, Jiang J F. 2020. Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems. Mech Syst Signal Process, 145: 106922. |
[7] | Ismail N, Malik O A. 2022. Real-time visual inspection system for grading fruits using computer vision and deep learning techniques. Inf Process Agric, 9(1): 24-37. |
[8] | Jeyaraj P R, Nadar E R S. 2019. Deep Boltzmann machine algorithm for accurate medical image analysis for classification of cancerous region. Cogn Comput Syst, 1(3): 85-90. |
[9] | Jeyaraj P R, Nadar E R S. 2020a. High-performance dynamic magnetic resonance image reconstruction and synthesis employing deep feature learning convolutional networks. Int J Imaging Syst Technol, 30(2): 380-390. |
[10] | Jeyaraj P R, Nadar E R S. 2020b. Dynamic image reconstruction and synthesis framework using deep learning algorithm. IET Image Process, 14(7): 1219-1226. |
[11] | Jo H W, Lee S, Park E, Lim C H, Song C, Lee H, Ko Y, Cha S, Yoon H, Lee W K. 2020. Deep learning applications on multitemporal SAR (Sentinel-1) image classification using confined labeled data: The case of detecting rice paddy in South Korea. IEEE Trans Geosci Remote Sens, 58(11): 7589-7601. |
[12] |
Kaur H, Sawhney B K, Jawandha S K. 2018. Evaluation of plum fruit maturity by image processing techniques. J Food Sci Technol, 55(8): 3008-3015.
PMID |
[13] | Koklu M, Cinar I, Taspinar Y S. 2021. Classification of rice varieties with deep learning methods. Comput Electron Agric, 187: 106285. |
[14] | Kucuk Ç, Taşkın G, Erten E. 2016. Paddy-rice phenology classification based on machine-learning methods using multitemporal co-polar X-band SAR images. IEEE J Sel Top Appl Earth Obs Remote Sens, 9(6): 2509-2519. |
[15] | Kuo T Y, Chung C L, Chen S Y, Lin H G, Kuo Y F. 2016. Identifying rice grains using image analysis and sparse-representation-based classification. Comput Electron Agric, 127: 716-725. |
[16] | Marimuthu S, Roomi S M M. 2017. Particle swarm optimized fuzzy model for the classification of banana ripeness. IEEE Sens J, 17(15): 4903-4915. |
[17] | Mittal S, Dutta M K, Issac A. 2019. Non-destructive image processing based system for assessment of rice quality and defects for classification according to inferred commercial value. Measurement, 148: 106969. |
[18] | Nandi C S, Tudu B P, Koley C. 2016. A machine vision technique for grading of harvested mangoes based on maturity and quality. IEEE Sens J, 16(16): 6387-6396. |
[19] | Ren S Q, He K M, Girshick R, Sun J. 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 39(6): 1137-1149. |
[20] | Shamim Hossain M, Al-Hammadi M, Muhammad G. 2019. Automatic fruit classification using deep learning for industrial applications. IEEE Trans Ind Inform, 15(2): 1027-1034. |
[21] | Singh K R, Chaudhury S. 2016. Efficient technique for rice grain classification using back-propagation neural network and wavelet decomposition. IET Comput Vis, 10(8): 780-787. |
[22] | Su J Y, Yi D W, Su B F, Mi Z W, Liu C J, Hu X P, Xu X M, Guo L, Chen W H. 2021. Aerial visual perception in smart farming: Field study of wheat yellow rust monitoring. IEEE Trans Ind Inform, 17(3): 2242-2249. |
[23] | Sun B Y, Yuan N Z, Zhao Z. 2020. A hybrid demosaicking algorithm for area scan industrial camera based on fuzzy edge strength and residual interpolation. IEEE Trans Ind Inform, 16(6): 4038-4048. |
[24] | Sun J, Lu X Z, Mao H P, Jin X M, Wu X H. 2017. A method for rapid identification of rice origin by hyperspectral imaging technology. J Food Process Eng, 40(1): e12297. |
[25] | Wu L H, Liu Z C, Bera T, Ding H J, Langley D A, Jenkins-Barnes A, Furlanello C, Maggio V, Tong W D, Xu J. 2019. A deep learning model to recognize food contaminating beetle species based on elytra fragments. Comput Electron Agric, 166: 105002. |
[26] | Xiao G Y, Wu Q, Chen H, Da D, Guo J Z, Gong Z G. 2020. A deep transfer learning solution for food material recognition using electronic scales. IEEE Trans Ind Inform, 16(4): 2290-2300. |
[27] | Xu X L, Li W S, Duan Q L. 2021. Transfer learning and SE-ResNet152 networks-based for small-scale unbalanced fish species identification. Comput Electron Agric, 180: 105878. |
[28] | Yang H J, Pan B, Li N, Wang W, Zhang J, Zhang X L. 2021. A systematic method for spatio-temporal phenology estimation of paddy rice using time series Sentinel-1 images. Remote Sens Environ, 259: 112394. |
[29] | Yang S, Zhu Q B, Huang M, Qin J W. 2017. Hyperspectral image- based variety discrimination of maize seeds by using a multi- model strategy coupled with unsupervised joint skewness-based wavelength selection algorithm. Food Anal Methods, 10(2): 424-433. |
[30] | Zareiforoush H, Minaei S, Alizadeh M R, Banakar A, Samani B H. 2016. Design, development and performance evaluation of an automatic control system for rice whitening machine based on computer vision and fuzzy logic. Comput Electron Agric, 124: 14-22. |
[31] | Zeng W H, Li M. 2020. Crop leaf disease recognition based on Self-Attention convolutional neural network. Comput Electron Agric, 172: 105341. |
[32] | Zhang M, Zhang H Q, Li X Y, Liu Y, Cai Y T, Lin H. 2020. Classification of paddy rice using a stacked generalization approach and the spectral mixture method based on MODIS time series. IEEE J Sel Top Appl Earth Obs Remote Sens, 13: 2264-2275. |
[1] | Stephan Nascente Adriano, Fernando Stone Luis. Cover Crops as Affecting Soil Chemical and Physical Properties and Development of Upland Rice and Soybean Cultivated in Rotation [J]. Rice Science, 2018, 25(6): 340-349. |
[2] | Bernaola Lina, Cange Grace, O. Way Michael, Gore Jeffrey, Hardke Jarrod, Stout Michael. Natural Colonization of Rice by Arbuscular Mycorrhizal Fungi in Different Production Areas [J]. Rice Science, 2018, 25(3): 169-174. |
[3] | Lasalita Zapico Florence, Hazel Aguilar Catherine, Abistano Angelie, Carino Turner Josephine, Jacinto Reyes Lolymar. Biocultural Diversity of Sarangani Province, Philippines: An Ethno-Ecological Analysis [J]. Rice Science, 2015, 22(3): 138-146. |
[4] | OLADELE Oladimeji Idowu, Toshiyuki WAKATSUKI. Social Factors Affecting Wetlands Utilization for Agriculture in Nigeria: A case study of sawah rice production [J]. RICE SCIENCE, 2008, 15(2): 150-152 . |
[5] | Hiroshi IKEHASHI. The Origin of Flooded Rice Cultivation [J]. RICE SCIENCE, 2007, 14(3): 161-171 . |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||