Intelligent Survey Method for Tiny Rice Pests and Their Natural Predators in Paddy Fields Using Augmented Reality (AR) Glasses

doi:10.1016/j.rsci.2025.08.005

Abstract

Abstract:

Rice crops are frequently threatened by pests such as rice planthoppers (Nilaparvata lugens, Sogatella furcifera, and Laodelphax striatellus) and leafhoppers (Cicadellidae), which cause significant yield losses. Accurate identification of both pest developmental stages and their natural predators is crucial for effective pest control and maintaining ecological balance. However, conventional field surveys are often subjective, inefficient, and lack traceability. To overcome these limitations, this study proposed RiceInsectID, a two-stage cascaded detection method designed to identify and count tiny rice pests and their natural predators from white flat plate images captured by head-worn AR glasses. The method recognizes 25 insect classes, including 17 instars of rice planthoppers, 2 instars of leafhoppers, 4 spider species (Araneae), as well as Miridae and rove beetles (Staphylinidae Latreille). At the first coarse-grained detection stage, 16 visually similar classes are consolidated into 6 broader categories and detected using an enhanced YOLOv6 model. To improve small object detection and address class imbalance, the full-region overlapping sliding slices and target pasting (FOSTP) algorithm was applied, increasing the mean average precision at a 50% IoU threshold (mAP₅₀) by 35.46% over the baseline YOLOv6. Feature extraction and fusion were further improved by incorporating an efficient channel attention path aggregation feature pyramid network (ECA-PAFPN) and adaptive structure feature fusion (ASFF) modules, while the balanced classification mosaic (BCM) enhanced detection of minority classes. With test-time augmentation (TTA), mAP₅₀ improved by an additional 2.06%, reaching 84.71%. At the second fine-grained classification stage, each of the six broad classes from the first stage is further classified using individual ResNet50 models. Online data augmentation and transfer learning were employed to significantly enhance generalization. Compared with the baseline YOLOv6, the two-stage cascaded method improved recall by 4.06%, precision by 3.79%, and the F1-score by 3.92%. Overall, RiceInsectID achieved 82.85% recall, 80.62% precision, and an F1-score of 81.72%, demonstrating an efficient and practical solution for monitoring tiny rice pests and their natural predators in paddy fields. This study provides valuable insights for ecosystem monitoring and supporting sustainable pest management in rice agriculture.

Key words: tiny rice pest, natural predator, AR glasses, intelligent survey, object detection, fine-grained recognition

Hong Chen, Luo Ju, Feng Zelin, Ling Heping, Li Lingyi, Wu Jian, Yao Qing, Liu Shuhua. Intelligent Survey Method for Tiny Rice Pests and Their Natural Predators in Paddy Fields Using Augmented Reality (AR) Glasses[J]. Rice Science, 2025, 32(6): 868-884.

Figures/Tables 18

References 52

[1]	Bochkovskiy A, Wang C Y, Liao H Y M. 2020. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934.
[2]	Cai Z W, Vasconcelos N. 2018. Cascade R-CNN: Delving into high quality object detection. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018. Salt Lake City, UT, USA: IEEE: 6154-6162.
[3]	Chen H R, Wen C J, Zhang L, et al. 2025. Pest-PVT: A model for multi-class and dense pest detection and counting in field-scale environments. Comput Electron Agric, 230: 109864.
[4]	Dai X Y, Chen Y P, Xiao B, et al. 2021. Dynamic head: Unifying object detection heads with attentions. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 19-25, 2021. Nashville, TN, USA: IEEE: 7369-7378.
[5]	Ding J H, Hu C L, Fu Q, et al. 2012. A Colour Atlas of Commonly Encountered Delphacids in China Rice Regions. Hangzhou, China: Zhejiang Science and Technology Publishing House. (in Chinese)
[6]	Dong S F, Teng Y, Jiao L, et al. 2024. ESA-Net: An efficient scale-aware network for small crop pest detection. Expert Syst Appl, 236: 121308.
[7]	Du C Q, Wang Y L, Song S J, et al. 2024. Probabilistic contrastive learning for long-tailed visual recognition. IEEE Trans Pattern Anal Mach Intell, 46(9): 5890-5904.
[8]	Fujita D, Kohli A, Horgan F G. 2013. Rice resistance to planthoppers and leafhoppers. Crit Rev Plant Sci, 32(3): 162-191.
[9]	General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of China. 2009. Rules of investigation and forecast for the rice planthopper: GB/T 15794-2009. 2009-12-30 [2025-04-15]. (in Chinese)
[10]	General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of China. 2011. Rules for monitoring and forecast of the cotton aphid: GB/T 15799-2011. 2011-12-30 [2025-04-15]. (in Chinese)
[11]	Grijalva I, Clark N, Hamilton E, et al. 2024. Comprehensive wheat coccinellid detection dataset: Essential resource for digital entomology. Data Brief, 55: 110585.
[12]	Hajjar M J, Ahmed N, Alhudaib K A, et al. 2023. Integrated insect pest management techniques for rice. Sustainability, 15(5): 4499.
[13]	He K M, Zhang X Y, Ren S Q, et al. 2016. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA: IEEE: 770-778.
[14]	He X Q, Qiao Y H, Sigsgaard L, et al. 2020. The spider diversity and plant hopper control potential in the long-term organic paddy fields in sub-tropical area, China. Agric Ecosyst Environ, 295: 106921.
[15]	Horgan F G. 2024. Virulence adaptation by rice planthoppers and leafhoppers to resistance genes and loci: A review. Insects, 15(9): 652.
[16]	Howard A, Sandler M, Chu G, et al. 2019. Searching for MobileNetV3. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019. Seoul, Korea (South): IEEE: 1314-1324.
[17]	Hu G W, Zhu M H. 1981. Observation of short-winged male of the white-backed planthopper. Chin J Appl Entomol, 4: 154-156. (in Chinese with English abstract)
[18]	Hu J, Shen L, Sun G. 2018. Squeeze-and-excitation networks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018. Salt Lake City, UT, USA: IEEE: 7132-7141.
[19]	Khairunniza-Bejo S, Ibrahim M F, Hanafi M, et al. 2024. Automatic paddy planthopper detection and counting using faster R-CNN. Agriculture, 14(9): 1567.
[20]	Li C Y, Li L L, Jiang H L, et al. 2022. YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv: 2209.02976.
[21]	Liu K, Fu Z H, Jin S, et al. 2025. ESOD: Efficient small object detection on high-resolution images. IEEE Trans Image Process, 34: 183-195.
[22]	Liu S X, Liu S T, Qu H X, et al. 2023. Multi-scale rice planthopper image recognition and classification based on lightweight MS-YOLO v7. Trans Chin Soc Agric Mach, 54(S1): 212-221. (in Chinese with English abstract)
[23]	Liu T, Zhao H Y. 2010. Recognition methods of natural enemies of aphis Gossypii Glover based on extraction of local features. Comput Eng Des, 31(16): 3712-3714. (in Chinese with English abstract)
[24]	Liu Z, Lin Y T, Cao Y, et al. 2021. Swin Transformer: Hierarchical vision transformer using shifted windows. In: 2021 IEEE International Conference on Computer Vision (ICCV). October 11-17, 2021. Montreal, QC, Canada: IEEE: 10012-10022.
[25]	Liu Z, Mao H Z, Wu C Y, et al. 2022. A ConvNet for the 2020s. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 19-24, 2022. New Orleans, LA, USA: IEEE: 11976-11986.
[26]	Lyu C Q, Zhang W W, Huang H A, et al. 2022. RTMDet: An empirical study of designing real-time object detectors. arXiv preprint arXiv: 2212.07784.
[27]	Neubeck A, van Gool L. 2006. Efficient non-maximum suppression. In: 2006 IEEE 18th International Conference on Pattern Recognition (ICPR). August 20-24, 2006, Hong Kong, China: IEEE: 850-855.
[28]	Nie L L, Li B G, Jiao F, et al. 2024. EVIT-YOLOv8: Construction and research on African Swine Fever facial expression recognition. Comput Electron Agric, 227: 109575.
[29]	Qiao Y L, Guo Y Y, He D J. 2023. Cattle body detection based on YOLOv5-ASFF for precision livestock farming. Comput Electron Agric, 204: 107579.
[30]	Ren S Q, He K M, Girshick R, et al. 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 39(6): 1137-1149.
[31]	Ruder S. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv: 1609.04747.
[32]	Shanmugam D, Blalock D, Balakrishnan G, et al. 2021. Better aggregation in test-time augmentation. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). October 11-17, 2021. Montreal, QC, Canada: IEEE: 1214-1223.
[33]	Sheng H Y, Yao Q, Luo J, et al. 2024. Automatic detection and counting of planthoppers on white flat plate images captured by AR glasses for planthopper field survey. Comput Electron Agric, 218: 108639.
[34]	Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. In: 2015 International Conference on Learning Representations (ICLR). May 7-9, 2015, San Diego, CA, USA: ICLR: 1-14.
[35]	Sorgog K, Tanaka K, Baba Y G. 2023. Macro-scale perspectives in conservation biological control: Latitudinal differences in the effects of organic farming on insect pests and natural enemies in rice paddy ecosystems. Agric Ecosyst Environ, 357: 108689.
[36]	Sun G J, Liu S H, Luo H L, et al. 2022. Intelligent monitoring system of migratory pests based on searchlight trap and machine vision. Front Plant Sci, 13: 897739.
[37]	Tan M, Le Q. 2019. EfficientNet: Rethinking model scaling for convolutional neural networks. In: 2019 International Conference on Machine Learning (ICML) June 10-15, 2019, Long Beach, CA, USA: PMLR: 6105-6114.
[38]	Vijayakumar A, Vairavasundaram S. 2024. YOLO-based object detection models: A review and its applications. Multimed Tools Appl, 83: 83535-83574.
[39]	Wang C X, Grijalva I, Caragea D, et al. 2023. Detecting common coccinellids found in sorghum using deep learning models. Sci Rep, 13(1): 9748.
[40]	Wang D R, Tan J S, Wang H, et al. 2025. SDS-YOLO: An improved vibratory position detection algorithm based on YOLOv11. Measurement, 244: 116518.
[41]	Wang Q L, Wu B G, Zhu P F, et al. 2020. ECA-Net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 14-19, 2020. Seattle, WA, USA: IEEE: 11534-11542.
[42]	Wei M, Zhan W. 2024. YOLO_MRC: A fast and lightweight model for real-time detection and individual counting of Tephritidae pests. Ecol Inform, 79: 102445.
[43]	Wong S C, Gatt A, Stamatescu V, et al. 2016. Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA). November 30-December 2, 2016. Gold Coast, QLD, Australia: IEEE: 1-6.
[44]	Xiang Q C, Huang X N, Huang Z X, et al. 2023. Yolo-pest: An insect pest object detection algorithm via CAC3 module. Sensors, 23(6): 3221.
[45]	Xuan S Y, Zhang S L. 2024. Decoupled contrastive learning for long-tailed recognition. Proc AAAI Conf Artif Intell, 38(6): 6396-6403.
[46]	Yao Q, Xian D X, Liu Q J, et al 2014. Automated counting of rice planthoppers in paddy fields based on image processing. J Integr Agric, 13(8): 1736-1745.
[47]	Yao Q, Wu S Z, Kuai N Y, et al. 2021. Automatic detection of rice planthoppers through light-trap insect images using improved CornerNet. Trans Chin Soc Agric Eng, 37(7): 183-189. (in Chinese with English abstract)
[48]	Yu P S, Guo L J, Yao Q, et al. 2019. A survey method based on mobile terminal for rice planthoppers in paddy fields. Acta Entomol Sin, 62(5): 615-623. (in Chinese with English abstract)
[49]	Yu W H, Luo M, Zhou P, et al. 2022. MetaFormer is actually what you need for vision. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 19-24, 2022. New Orleans, LA, USA: IEEE: 10819-10829.
[50]	Zhang X H, Li Z Y, Ren L L, et al. 2024. Detection and recognition of the invasive species, Hylurgus ligniperda, in traps, based on a cascaded convolution neural network. Pest Manag Sci, 80(9): 4223-4230.
[51]	Zhang Y T, Huang D Q, Wang D W, et al. 2023. Review on research and application of deep learning-based target detection algorithms. Comput Eng Appl, 59(18): 1-13. (in Chinese with English abstract)
[52]	Zhang Z L, Zhan W, Sun K L, et al. 2024. RPH-Counter: Field detection and counting of rice planthoppers using a fully convolutional network with object-level supervision. Comput Electron Agric, 225: 109242.

Species	Morphological type	Labeled name	Number of training sets	Number of testing sets
Nilaparvata lugens	Macropterous female	M-BPH-female	2 115	413
	Macropterous male	M-BPH-male	704	217
	Brachypterous female	B-BPH-female	1 934	454
	Brachypterous male	B-BPH-male	421	112
	Senior-instar nymph	S-BPH-nymph	6 814	1 831
	Third-instar nymph	BPH 3-nymph	3 195	1 113
Sogatella furcifera	Macropterous female	M-WBPH-female	3 404	784
	Macropterous male	M-WBPH-male	2 216	548
	Brachypterous adult	B-WBPH-adult	961	247
	Senior-instar nymph	S-WBPH-nymph	7 105	1 561
	Third-instar nymph	WBPH 3-nymph	9 785	2 250
Laodelphax striatellus	Macropterous female	M-SBPH-female	507	131
	Macropterous male	M-SBPH-male	527	133
	Brachypterous adult	B-SBPH-adult	564	134
	Senior-instar nymph	S-SBPH-nymph	3 006	761
	Third-instar nymph	SBPH 3-nymph	833	204
Early-instar nymph of rice planthopper	Early-instar nymph	RPH 1‒2 nymph	102 410	23 458
Cicadellidae	Leafhopper nymph	RLH-nymph	508	111
Cicadellidae	Leafhopper adult	RLH-adult	11 352	2 740
Cyrtorhinus lividipennis Reuter	Miridae	Miridae	8 266	2 178
Araneae	Wolf spider	Spider-Langzhu	4 094	1 061
	Micryphantid spider	Spier-Weizhu	3 716	960
	Tetragnathidae	Spider-Xiaoxiao	572	164
	Others	Spider-Others	11 590	2 840
Staphylinidae Latreille	Rove beetle	Rove-beetle	318	85

Species	Morphological type	Labeled name	Number of training sets	Number of testing sets
Nilaparvata lugens	Macropterous female	M-BPH-female	2 115	413
	Macropterous male	M-BPH-male	704	217
	Brachypterous female	B-BPH-female	1 934	454
	Brachypterous male	B-BPH-male	421	112
	Senior-instar nymph	S-BPH-nymph	6 814	1 831
	Third-instar nymph	BPH 3-nymph	3 195	1 113
Sogatella furcifera	Macropterous female	M-WBPH-female	3 404	784
	Macropterous male	M-WBPH-male	2 216	548
	Brachypterous adult	B-WBPH-adult	961	247
	Senior-instar nymph	S-WBPH-nymph	7 105	1 561
	Third-instar nymph	WBPH 3-nymph	9 785	2 250
Laodelphax striatellus	Macropterous female	M-SBPH-female	507	131
	Macropterous male	M-SBPH-male	527	133
	Brachypterous adult	B-SBPH-adult	564	134
	Senior-instar nymph	S-SBPH-nymph	3 006	761
	Third-instar nymph	SBPH 3-nymph	833	204
Early-instar nymph of rice planthopper	Early-instar nymph	RPH 1‒2 nymph	102 410	23 458
Cicadellidae	Leafhopper nymph	RLH-nymph	508	111
Cicadellidae	Leafhopper adult	RLH-adult	11 352	2 740
Cyrtorhinus lividipennis Reuter	Miridae	Miridae	8 266	2 178
Araneae	Wolf spider	Spider-Langzhu	4 094	1 061
	Micryphantid spider	Spier-Weizhu	3 716	960
	Tetragnathidae	Spider-Xiaoxiao	572	164
	Others	Spider-Others	11 590	2 840
Staphylinidae Latreille	Rove beetle	Rove-beetle	318	85

Class	Recall₅₀(%)	Precision₅₀(%)	AP50 (%)
M-BPH-female	96.4	71.9	92.8
M-BPH-male	54.7	92.6	52.4
B-BPH-female	88.9	79.9	87.2
B-BPH-male	84.8	81.1	80.4
S-BPH-nymph	91.1	73.6	85.7
BPH 3-nymph	53.8	72.2	46.0
M-WBPH-female	94.6	73.9	90.1
M-WBPH-male	72.7	81.5	66.8
B-WBPH-adult	78.8	69.7	72.5
S-WBPH-nymph	93.1	72.7	88.8
WBPH 3-nymph	47.8	85.7	43.9
M-SBPH-female	61.5	73.4	51.3
M-SBPH-male	72.6	87.7	70.2
B-SBPH-adult	77.7	72.7	68.7
S-SBPH-nymph	76.0	62.0	70.2
SBPH 3-nymph	41.5	52.0	26.2
RPH 1‒2 nymph	91.2	80.7	86.8
RLH-nymph	90.5	85.9	87.8
RLH-adult	85.6	73.8	80.2
Miridae	93.0	83.7	91.5
Spider-Langzhu	93.6	91.2	92.2
Spier-Weizhu	82.5	73.3	74.8
Spider-Xiaoxiao	86.4	73.9	81.1
Spider-Others	78.3	77.3	68.7
Rove beetle	82.6	78.3	79.9

Class	Recall₅₀(%)	Precision₅₀(%)	AP50 (%)
M-BPH-female	96.4	71.9	92.8
M-BPH-male	54.7	92.6	52.4
B-BPH-female	88.9	79.9	87.2
B-BPH-male	84.8	81.1	80.4
S-BPH-nymph	91.1	73.6	85.7
BPH 3-nymph	53.8	72.2	46.0
M-WBPH-female	94.6	73.9	90.1
M-WBPH-male	72.7	81.5	66.8
B-WBPH-adult	78.8	69.7	72.5
S-WBPH-nymph	93.1	72.7	88.8
WBPH 3-nymph	47.8	85.7	43.9
M-SBPH-female	61.5	73.4	51.3
M-SBPH-male	72.6	87.7	70.2
B-SBPH-adult	77.7	72.7	68.7
S-SBPH-nymph	76.0	62.0	70.2
SBPH 3-nymph	41.5	52.0	26.2
RPH 1‒2 nymph	91.2	80.7	86.8
RLH-nymph	90.5	85.9	87.8
RLH-adult	85.6	73.8	80.2
Miridae	93.0	83.7	91.5
Spider-Langzhu	93.6	91.2	92.2
Spier-Weizhu	82.5	73.3	74.8
Spider-Xiaoxiao	86.4	73.9	81.1
Spider-Others	78.3	77.3	68.7
Rove beetle	82.6	78.3	79.9

Model	Recall	Precision	F1-score	Accuracy
VGG16	79.90	82.88	81.36	92.76
Mobile-v3-large	72.62	76.24	74.39	82.42
ResNet18	82.61	84.58	83.58	93.57
ResNet34	82.69	84.75	83.71	93.74
ResNet50	82.95	85.46	84.19	93.95
ResNet101	82.67	85.32	83.97	93.79
Efficient_v1-b0	73.70	81.87	77.57	91.82
Efficient_v2-b0	76.31	68.20	72.03	88.96
Swin-Transformer	69.46	56.25	62.16	82.71
Poolformer-S12	76.93	84.69	80.62	92.80
Convnext-base	62.73	50.22	55.78	82.84