Learning spatio-temporal representations with a dual-stream 3-D residual network for nondriving activity recognition

Simple item page
dc.contributor.authorYang, Lichao
dc.contributor.authorShan, Xiaocai
dc.contributor.authorLv, Chen
dc.contributor.authorBrighton, James
dc.contributor.authorZhao, Yifan
dc.date.accessioned2021-08-16T15:47:23Z
dc.date.available2021-08-16T15:47:23Z
dc.date.issued2021-07-28
dc.description.abstractAccurate recognition of non-driving activity (NDA) is important for the design of intelligent Human Machine Interface to achieve a smooth and safe control transition in the conditionally automated driving vehicle. However, some characteristics of such activities like limited-extent movement and similar background pose a challenge to the existing 3D convolutional neural network (CNN) based action recognition methods. In this paper, we propose a dual-stream 3D residual network, named D3D ResNet, to enhance the learning of spatio-temporal representation and improve the activity recognition performance. Specifically, a parallel 2-stream structure is introduced to focus on the learning of short-time spatial representation and small-region temporal representation. A 2-feed driver behaviour monitoring framework is further build to classify 4 types of NDAs and 2 types of driving behaviour based on the drivers head and hand movement. A novel NDA dataset has been constructed for the evaluation, where the proposed D3D ResNet achieves 83.35% average accuracy, at least 5% above three selected state-of-the-art methods. Furthermore, this study investigates the spatio-temporal features learned in the hidden layer through the saliency map, which explains the superiority of the proposed model on the selected NDAs.en_UK
dc.identifier.citationYang L, Shan X, Lv C, et al., (2022) Learning spatio-temporal representations with a dual-stream 3-D residual network for nondriving activity recognition. IEEE Transactions on Industrial Electronics, Volume 69, Number 7, July 2022, pp. 7405-7414en_UK
dc.identifier.issn0278-0046
dc.identifier.urihttps://doi.org/10.1109/TIE.2021.3099254
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/16995
dc.language.isoenen_UK
dc.publisherIEEEen_UK
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.subjectaction recognitionen_UK
dc.subjectnon-driving related tasken_UK
dc.subjectautomated drivingen_UK
dc.titleLearning spatio-temporal representations with a dual-stream 3-D residual network for nondriving activity recognitionen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
non-driving_activity_recognition-2021.pdf
Size:
2.41 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)