A complete reinforcement learning based framework for reconfigurable manufacturing system scheduling

Since the last decade of the 20th century, a new kind of manufacturing system paradigm known as the reconfigurable manufacturing system (RMS) has been emerging. The purpose of an RMS is to offer a balanced solution that can swiftly respond to volatile global markets with fluctuating product demand. It achieves this by combining the high throughput of conventional dedicated manufacturing lines (DMLs) with the flexibility of flexible manufacturing systems (FMSs). To instigate the market uncertainty, RMS possesses six core characteristics, namely modularity, integrability, convertibility, scalability, diagnosability, and customisation. These core characteristics are becoming more available with the development of Industry 4.0 technologies. Simulation on digital twins is one of the compelling approach that help RMSs check their status in real time. However, the extended data flow challenges the traditional rule-based scheduling policies and urges a flexible approach to replace ill-suited approaches, then further reveals the potential of an RMS. Reinforcement learning (RL) is a promising decision-making approach which had already led to breakthroughs in a lot of research aspects including game playing, robotics, finance, and autonomous driving. With a monolithic parametric reward function, RL agents addressed a wide range of complex tasks by integrating information from real manufacturing processes. Simulating complex and changeable production systems, like RMS, is an area where RL principles may be applied. An end-to-end deep reinforcement learning framework was developed in this research. The resulting policy is trained to generate a sequence of consecutive actions that can be used as an RMS schedule to manage a fluctuating market simulator in real-time.