A 3D–2D Dual-Modal Collaborative Framework Based on UAV Oblique Photogrammetry for Automated Measurement of Canopy Volume and Porosity in Banana Plantations

Abstract

Banana plant canopies exhibit pronounced three-dimensional heterogeneity due to their large, sparse, and overlapping leaves. Under dense planting and severe occlusion, conventional single-modality approaches commonly suffer from insufficient information and limited recognition accuracy, highlighting the necessity of cross-modal complementarity and collaborative modeling. This study proposes a 3D-2D dual-modal collaborative framework based on low-cost UAV oblique imagery to enable end-to-end estimation of canopy volume and porosity in banana plantations. The framework integrates a task-oriented YOLO-SPES model to improve the detection of irregular and overlapping canopies and combines it with the SoftGroup instance segmentation model for three-dimensional structural extraction. At the data level, a cross-modal coordinate interaction strategy (PCI-LLCM) is introduced to achieve precise alignment between point clouds and orthomosaics. At the structural level, a consistent indexing scheme between 3D instances and 2D detection boxes is established, upon which a 3D-2D collaborative modeling algorithm (SGP-YS DMCA) and a multi-scale volume differencing algorithm (MSVDA) are developed for canopy volume and porosity estimation. At the decision level, an adaptive canopy volume completion module (CVC-AOML) leverages 2D detection information to correct and supplement errors and omissions in 3D segmentation, thereby ensuring the accuracy and completeness of large-scale automated measurements. In addition, the coupling performance of multiple geometric algorithms and collaborative models is systematically evaluated. Experimental results demonstrate that the proposed dual-modal collaborative framework achieves a coefficient of determination (R$^{2}$) of 0.885 for canopy volume estimation, representing an average improvement of 0.15 over single-modality baseline methods, with a corresponding mean absolute percentage error (MAPE) of 6.4%. For canopy porosity estimation, an R$^{2}$of 0.65 is obtained with a MAPE of 1.05%. These results not only overcome the limitations of single-modality approaches in phenotypic analysis of complex banana canopies but also provide a low-cost and scalable solution for large-scale agricultural monitoring and precision management of tropical fruit crops.

Code and data availability

公開論文であることは確認できましたが、現在の公式API・許可済み取得経路では本文を自動取得できませんでした。

No evidence-backed public reproduction asset is currently recorded.