Bayesian model for strategic level risk assessment in continuing airthworthiness of air transport

Abstract

Continuing airworthiness (CAW) of aircraft is an essential pre-requisite for the safe operation of air transport. Human errors that occur in CAW organizations and processes could undermine the airworthiness and constitute a risk to flight safety. This thesis reports on a generic Bayesian model that has been designed to assess and quantify this risk. The model removes the vagueness inherent in the subjective methods of assessment of risk and its qualitative expression. Instead, relying on a transparent, structured mathematical process based on Bayes’ Theorem of conditional probabilities, the model yields a quantitative risk output expressed as a probability of error coupled with a probability of consequence based on data. The Bayesian model has 184 nodes and 1138 parameters that define causal factors for error against which data is collected as either beliefs or evidence, the latter returning more reliable results. Beliefs could be gradually replaced with evidence as they become available, improving fidelity. The generic model can be modified by adding or truncating parameters to suit conditions applicable to specific organizations or similar groups. The model was validated using field data from a cargo operator using large western jet freighters, covering 34,338 sectors of which 193 carried human error. Separate tests were performed simulating the operator’s belief that it was operating to global standards. The output for belief was consistent with global and UK flat rate safety levels, achievable if the operator flew 3M and 6M sectors respectively according to their belief. However, the output from evidence returned a risk level more severe than the belief, partly driven by the allowance for unknowns built into the computing technique and part by the relatively small number of sectors considered. In “what-if” prediction mode the model calculates the change in risk level due to new errors, and through sensitivity analysis it can identify and rank performance indicators. In CAW organizations subjected to Risk Based Oversight (RBO) concept and ICAO mandate on Safety Management System (SMS), the model can set risk threshold levels for individual organizations, to measure variations, and by continuous updating, to monitor safety performance at strategic level. Sharing data and with agreed performance levels, the Regulator and operators should be able negotiate an oversight plan. Using the model pro-actively, the organization could exercise a degree of self-regulation, thereby accruing cost benefits through reduced Regulator oversights.