Keypoints-based heterogeneous graph convolutional networks for construction

Simple item page
dc.contributor.authorWang, Shuozhi
dc.contributor.authorYang, Lichao
dc.contributor.authorZhang, Zichao
dc.contributor.authorZhao, Yifan
dc.date.accessioned2023-10-02T12:13:28Z
dc.date.available2023-10-02T12:13:28Z
dc.date.issued2023-09-22
dc.description.abstractArtificial intelligence algorithms employed for classifying excavator-related activities predominantly rely on sensors embedded within individual machinery or computer vision (CV) techniques encompassing a large scene. The existing CV-based methods are often difficult to tackle an image including multiple excavators and other cooperating machinery. This study presents a novel framework tailored to the classification of excavator activities, accounting for both the excavator itself and the dumpers collaborating with the excavator during operations. Distinct from most existing related studies, this method centres on the transformed heterogeneous graph data constructed using the keypoints of all cooperating machinery extracted from an image. The resulting model leverages the relationships between the mechanical components of an excavator in varying activation states and the associations between the excavator and the collaborating machinery. The framework commences with a novel definition of keypoints representing different machinery relevant to the targetted activities. A customised Machinery Keypoint R-CNN method is then developed to extract these keypoints, forming the basis of graph notes. By considering the type, attribute and edge of nodes, a Heterogeneous Graph Convolutional Network is finally utilised for activity recognition. The results suggest that the proposed framework can effectively predict earthwork activities (with an accuracy of up to 97.5%) when the image encompasses multiple excavators and cooperating machinery. This solution holds promising potential for the automated measurement and management of earthwork productivity within the construction industry. Code and data are available at: https://github.com/gillesflash/Keypoints-Based-Heterogeneous-Graph-Convolutional-Networks.git.en_UK
dc.description.sponsorshipRoyal Academy of Engineering Industrial Fellowship: IF2223B-110en_UK
dc.identifier.citationWang S, Yang L, Zhang Z, Zhao Y. (2024) Keypoints-based heterogeneous graph convolutional networks for construction, Expert Systems with Applications, Volume 327, Part C, March 2024, Article Number 121525en_UK
dc.identifier.eissn1873-6793
dc.identifier.issn0957-4174
dc.identifier.urihttps://doi.org/10.1016/j.eswa.2023.121525
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/20310
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.rightsAttribution 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/*
dc.subjectActivity classificationen_UK
dc.subjectComputer visionen_UK
dc.subjectGraph convolutional networksen_UK
dc.subjectKeypoint extractionen_UK
dc.titleKeypoints-based heterogeneous graph convolutional networks for constructionen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Heterogeneous_graph_convolutional_networks-2023.pdf
Size:
4.16 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)