Speaker
Description
Decision or controlled processes are of particular interest for maintenance problems, especially for determining optimal control actions to maximize the reward or minimize the cost of an operating system experiencing failures and when its temporal behavior is described by a Markov process or a semi-Markov process. This presentation includes two parts. In the first part, we introduce the fundamentals of Markov decision processes, followed by a constrained Markov decision process with absorbing states. The objective is to find a policy that maximizes the expected average reward over an infinite horizon before system failure, subject to a constraint on the asymptotic average failure rate. We propose a linear programming algorithm to obtain a stationary optimal policy, belonging to the class of history-dependent policies. Finally, we apply this method to the optimization of the maintenance policy of an aeronautical system, where the system failure rate must remain below a critical value provided by the Perron-Frobenius eigenvalue of the transition matrix.
Given the complexity of this controlled process problem, and to simplify the presentation and treatment of this model, in the second part, we present controlled semi-Markovian random evolutions in discrete time and prove some limit theorems, in particular averaging and diffusion approximation.
References
1.Boussemart M., Limnios N. (2023). Constrained Markov Decision Process for the Industry, In Ch. 16, Vol. 1 - Computational Methods and Mathematical Modeling in Cyber physics and Engineering Applications, Koroliouk D., Lyashko S. I., Limnios N. (Eds), ISTE, J. Wiley, London & N.J.
2.Limnios N., Swishchuk A. (2023). Discrete-Time Semi-Markov Random Evolutions and Their Applications, Birkhäuser/Springer.