Eliciting expert knowledge to inform training design

Abstract

To determine the elicitation methodologies best placed to uncover and capture the expert operator’s reflective cognitive judgements in complex and dynamic military operating environments (e.g., explosive ordinance disposal) in order to develop the specification for a reflective eXplainable Artificial Intelligence (XAI) agent to support the training of domain novices. Approach: A bounded literature review of the latest developments in expert knowledge elicitation was undertaken to determine the ’art-of-the-possible’ in respects to uncovering an expert’s cognitive judgements in complex and dynamic environments. Candidate methodologies were systematically and critically reviewed in order to identify the most promising methodologies for uncovering expert situational awareness and metacognitive evaluations in pursuit of actionable threat mitigation strategies in high-risk contexts. Research outputs are synthesized into an interview protocol for eliciting and understanding the in-situ actions and decisions of experts in high-risk, complex operating environments. Practical implications: Trainees entering high-risk operating environments can benefit from exposure to expert reflective strategies whilst learning the trade. Typical operator training focuses on technical aspects of threat mitigation but often overlooks reflective self-evaluation. The present study represents an initial step towards determining the feasibility of designing a reflective XAI agent to augment the performance of trainees entering high-risk operations. Outputs of the expert knowledge elicitation protocol documented here shall be used to refine a theoretical framework of expert operator judgement, in order to determine decision support strategies of benefit to domain novices.