Computer vision methods for the analysis of multimodal and multidimensional data for high-throughput plant phenotyping

Drought and nutrient stresses substantially impact crop productivity, frequently resulting in lower yields and financial losses to farmers. Early detection and tracking of these stresses are significant for improving agricultural yield and achieving the world food challenge goals. Advanced image technologies such as the use of conventional cameras and imaging spectroscopy combined with computer vision methodolgoies are being leveraged for non-invasive plant phenotyping to improve crop productivity, resilience, and sustainability. This thesis seeks to combine these technologies with machine learning algorithms to develop automated phenotyping pipelines to phenotype plants at different growth stages, particularly; for nutrient and drought stress identification and quantification. With this proposal, the onset of plant nutrient and drought stresses could be detected early and at different growth stages of the plants. Three experiments (two in the glasshouse and one in the field) were conducted, and images were acquired using digital RGB and a hyperspectral camera. The first experiment (nutrient stress) was performed in the glasshouse, the second (nutrient stress) on the field and the third (drought stress) in the glasshouse. The glasshouse nutrient experiment was performed on quinoa and cowpea plants made of four treatments: high nitrogen high phosphorus (HNHP), high nitrogen low phosphorus (HNLP), low nitrogen low phosphorus (LNHP) and low nitrogen low phosphorus (LNLP). The field nutrient experiment was performed on wheat plants made of 12 Olsen phosphorus varitions (approximately 3, 6, 9, 12, 15, 18, 21, 25, 30, 40, 50 and 60 ppm). The third experiment is a wheat drought analysis under variable nitrogen at selected plant growth stages. The treatments include: well-watered high- nitrogen (WWHN), well-watered low-nitrogen (WWLN), drought-stress high-nitrogen (DSHN) and drought-stress low-nitrogen (DSLN). Several image processing and machine learning techniques were employed to pre-process, post-process, and analyze plant- specific traits for tracking plant drought and nutrient stresses. Specifically, using digital imaging, a new segmentation algorithm invariant to illumination and complex background scenes was proposed to segment field and glasshouse-based images. Statistical and machine learning methods were employed to identify phenotypic traits sensitive to nutrient (nitrogen and phosphorus) deficiencies. Additionally, plant colour, morphology, and texture features were critically analyzed to assess their response to different stresses in plants. Using a hyperspectral imaging, a hybrid deep learning model was proposed to detect and track plant nitrogen and phosphorus deficiencies. In this case , the spatial and spectral characteristics of plants were analyzed, and deep learning algorithms were combined to understand their response to nutrient and drought stress in plants. Finally, using the spectral characteristics of plants, different conventional machine learning algorithms including Random Forest, Partial Least Square Regression and Support Vector Machines were developed to model the trends and patterns of plants for drought stress detection. The research results show a link between colour and nutrient stress, while texture and colour features were highly responsive to drought stress. The short-wave infrared region of the electromagnetic spectrum was highly responsive to plant phosphorus deficiency, while the blue, red, and near-infrared regions were highly connected to plant nitrogen deficiency. Furthermore, combining a proposed vegetation indices (VIs) from the VNIR regions of the spectrum with already known VIs resulted in the easy identification of plant drought stress compared to using only the known or proposed indices.