Binocular Multispectral Stereo Imaging Enables High-Resolution Plant Phenotyping

The system enables pixel-level alignment between shape and spectral data, allowing detailed visualization of traits like chlorophyll distribution across plant surfaces.

Traditional plant phenotyping relies on 3D imaging to measure structure and on spectral analysis to evaluate biochemical status. Existing devices often collect these datasets separately, complicating data fusion and limiting spatial-biochemical interpretation. Spectral imaging is widely used for nutrient detection, stress diagnosis, and mineral mapping, whereas 3D imaging offers precise geometric information. However, uniting both in a single platform remains technically challenging due to registration mismatch, resolution differences, and computational complexity. Based on these challenges, there is an urgent need for an integrated system that acquires structural and spectral information simultaneously for comprehensive 4D phenotyping.

A study (DOI: 10.1016/j.plaphe.2025.100105) published in Plant Phenomics on 20 September 2025 by Ziru Yu's team, Sun Yat-Sen University, offers a non-destructive and scalable solution for real-time plant phenotyping, precision agriculture and digital crop management.

In this study, a series of experimental demonstrations were carried out to validate the binocular multispectral stereo imaging (BMSI) system, beginning with precise system calibration, then spectral image segmentation, 3D reconstruction performance evaluation, and finally chlorophyll (Chl) mapping on plant surfaces. First, the camera's intrinsic and extrinsic parameters and distortion coefficients were estimated using Zhang's method with an 11 × 9 checkerboard, yielding reprojection errors of 0.067 and 0.089 pixels for the left and right imagers and 0.094 pixels for the binocular system. The compact 10-band prototype (713–920 nm) supports mobile use and completes a full 4D workflow-optimal index factor (OIF) band selection, fusion, segmentation, and reconstruction-within 7.546 s. To optimize segmentation, spectral images acquired under controlled lab conditions were analyzed across 120 band combinations, and the best trio (bands 0, 7, 9) was fused and transformed into HSV space. The resulting algorithm was compared against Otsu, K-means, graph-based, and watershed methods on two plant species, including a highly reflective sample. System performance was further assessed by reconstructing planar calibration plates positioned from 700 to 1500 mm, followed by noise removal, spectral correction, and plane fitting to evaluate standard deviation, RMSE, and residual distributions. Finally, first-order derivatives and the normalized difference red edge (NDRE) index were computed from the spectral data to derive Chl distribution on 3D plant models.

The calibrated system achieved a plane-fitting standard deviation of 0.9778 mm and MAE of 0.778 mm, with RMSE minimized (0.89 mm) in the 900–1220 mm optimal range and R² maintained at 0.8–0.9 over most distances. The proposed fusion–HSV segmentation outperformed all four baseline methods, achieving higher SM/UM/RPD scores, ~37.6% improvement in shadow separation, ~92% feature preservation, and <5.2% organ geometry error. Reconstructed 4D plant models showed dense, smooth point clouds with clear leaf edges, and NDRE–derivative analysis consistently distinguished regions with lower Chl (redshifted RE, dimmer NDRE colors) from healthier, high-Chl regions, confirming the system's robustness for detailed, spatially resolved plant phenotyping.

The BMSI platform provides a new tool for high-resolution phenotyping, enabling breeders and agronomists to assess plant morphology and biochemical traits simultaneously. Its ability to detect chlorophyll distribution and subtle spectral variations can support precision fertilization, stress detection, yield prediction, and digital farming systems. Compared to conventional hyperspectral or 3D cameras, this system eliminates cross-sensor registration, reduces data acquisition time, and maintains high spectral-spatial fidelity-making it suitable for laboratory research, greenhouse monitoring, and future field-deployable agricultural IoT applications.