MATLAB automatic differentiation using source transformation

Abstract

This thesis presents our work on compiler techniques to implement Algo- rithmic Di erentiation (AD) using source transformation in MATLAB. AD is concerned with the accurate and e cient computation of derivatives of complicated mathematical functions represented by computer programs. Source transformation techniques for AD, whilst complicated to imple- ment, are known to yield derivative code with better run-time e ciency than methods using overloading support of the underlying language. We present results from MSAD that con rm the increase in e ciency using source trans- formed code for MATLAB AD. Most importantly, we demonstrate the use of a unique compiler code specialisation method to implement AD. We also assert the need for compiler optimisations in MATLAB, especially in the con- text of AD, and showcase MSAD as an extensible infrastructure to implement new optimisations and algorithms for AD or other applications. Where other e orts on MATLAB AD are implemented using operator overloading or a mix of overloading and source transformation, MSAD (Springer LNCS, Vol. 3994, 2006) was the rst to generate di erentiated MATLAB code using source transformation alone. MSAD is also the only e ort to implement source transformed AD by resolving overloaded MAT- LAB code. The existing MAD package (ACM TOMS, 32, No.2, 2006) pro- vides a highly e cient overloaded implementation of MATLAB AD. MSAD uses compiler code specialisation techniques to specialise and inline fmad and derivvec overloaded operations of the MAD package in order to generate MATLAB AD code. The operator overloading overheads inherent in MAD are eliminated while preserving the derivvec class's optimised derivative combination operations. As a compiler framework for MATLAB, MSAD demonstrates a novel use of an existing e ective compiler algorithm (Sparse Conditional Constant Propagation) to infer properties of MATLAB variables such as type, rank, shape, sparsity and value by propagating a composite lattice of all the prop- erties together.