Improved depth recovery in consumer depth cameras via disparity space fusion within cross-spectral stereo

We address the issue of improving depth coverage in consumer depth cameras based on the combined use of cross-spectral stereo and near infrared structured light sensing. Specifically we show that fusion of disparity over these modalities prior to subsequent optimization, within the disparity space image, facilitates the recovery of scene depth information in regions where structured light sensing alone fails. This joint approach, leveraging disparity information from both structured light and cross-spectral stereo, facilitates the recovery of global scene depth comprising both texture-less object depth, where stereo sensing commonly fails, and highly reflective object depth, where structured light active sensing commonly fails. The proposed solution is illustrated using dense gradient feature matching and is shown to outperform prior approaches that use late-stage fused cross-spectral stereo depth as a facet of improved sensing for consumer depth cameras.