Constraints on optimising encoder-only transformers for modelling sign language with human pose estimation keypoint data

Simple item page
dc.contributor.authorWoods, Luke T.
dc.contributor.authorRana, Zeeshan A.
dc.date.accessioned2023-11-03T10:17:21Z
dc.date.available2023-11-03T10:17:21Z
dc.date.issued2023-11-02
dc.description.abstractSupervised deep learning models can be optimised by applying regularisation techniques to reduce overfitting, which can prove difficult when fine tuning the associated hyperparameters. Not all hyperparameters are equal, and understanding the effect each hyperparameter and regularisation technique has on the performance of a given model is of paramount importance in research. We present the first comprehensive, large-scale ablation study for an encoder-only transformer to model sign language using the improved Word-level American Sign Language dataset (WLASL-alt) and human pose estimation keypoint data, with a view to put constraints on the potential to optimise the task. We measure the impact a range of model parameter regularisation and data augmentation techniques have on sign classification accuracy. We demonstrate that within the quoted uncertainties, other than ℓ2 parameter regularisation, none of the regularisation techniques we employ have an appreciable positive impact on performance, which we find to be in contradiction to results reported by other similar, albeit smaller scale, studies. We also demonstrate that the model architecture is bounded by the small dataset size for this task over finding an appropriate set of model parameter regularisation and common or basic dataset augmentation techniques. Furthermore, using the base model configuration, we report a new maximum top-1 classification accuracy of 84% on 100 signs, thereby improving on the previous benchmark result for this model architecture and dataset.en_UK
dc.identifier.citationWoods LT, Rana ZA. (2023) Constraints on optimising encoder-only transformers for modelling sign language with human pose estimation keypoint data. Journal of Imaging, Volume 9, Issue 11, November 2023, Article number 238en_UK
dc.identifier.issn2313-433X
dc.identifier.urihttps://doi.org/10.3390/jimaging9110238
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/20501
dc.language.isoenen_UK
dc.publisherMDPIen_UK
dc.rightsAttribution 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/*
dc.subjectsign language recognitionen_UK
dc.subjecthuman pose estimationen_UK
dc.subjectclassificationen_UK
dc.subjectcomputer visionen_UK
dc.subjectdeep learningen_UK
dc.subjectmachine learningen_UK
dc.subjectsupervised learningen_UK
dc.subjectregularisationen_UK
dc.subjectdata augmentationen_UK
dc.titleConstraints on optimising encoder-only transformers for modelling sign language with human pose estimation keypoint dataen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Constraints_on_optimising_encoder-only_transformers-2023.pdf
Size:
658.25 KB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)