A refined non-driving activity classification using a two-stream convolutional neural network

Simple item page
dc.contributor.authorYang, Lichao
dc.contributor.authorYang, Tingyu
dc.contributor.authorLiu, Haochen
dc.contributor.authorShan, Xiaocai
dc.contributor.authorBrighton, James
dc.contributor.authorSkrypchuk, Lee
dc.contributor.authorMouzakitis, Alexandros
dc.contributor.authorZhao, Yifan
dc.date.accessioned2020-07-23T11:02:28Z
dc.date.available2020-07-23T11:02:28Z
dc.date.issued2020-06-29
dc.description.abstractIt is of great importance to monitor the driver’s status to achieve an intelligent and safe take-over transition in the level 3 automated driving vehicle. We present a camera-based system to recognise the non-driving activities (NDAs) which may lead to different cognitive capabilities for take-over based on a fusion of spatial and temporal information. The region of interest (ROI) is automatically selected based on the extracted masks of the driver and the object/device interacting with. Then, the RGB image of the ROI (the spatial stream) and its associated current and historical optical flow frames (the temporal stream) are fed into a two-stream convolutional neural network (CNN) for the classification of NDAs. Such an approach is able to identify not only the object/device but also the interaction mode between the object and the driver, which enables a refined NDA classification. In this paper, we evaluated the performance of classifying 10 NDAs with two types of devices (tablet and phone) and 5 types of tasks (emailing, reading, watching videos, web-browsing and gaming) for 10 participants. Results show that the proposed system improves the averaged classification accuracy from 61.0% when using a single spatial stream to 90.5%en_UK
dc.identifier.citationYang L, Yang T, Liu H, et al., (2021) A refined non-driving activity classification using a two-stream convolutional neural network. IEEE Sensors Journal, Volume 21, Number 14, July 2021, pp. 15574-15583en_UK
dc.identifier.issn1530-437X
dc.identifier.urihttps://doi.org/10.1109/JSEN.2020.3005810
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/15586
dc.language.isoenen_UK
dc.publisherIEEEen_UK
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.subject2-stream CNNen_UK
dc.subjectoptical flowen_UK
dc.subjectLevel 3 automationen_UK
dc.subjectNDA classificationen_UK
dc.titleA refined non-driving activity classification using a two-stream convolutional neural networken_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Refined_non-driving_activity_classification_using_a_two-stream_convolutional-2020.pdf
Size:
3.18 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)