Markov Chains and Decision Processes for Engineers and Managers

This book presents an introduction to finite Markov chains and Markov decision processes, with applications in engineering and management. It introduces discrete-time, finite-state Markov chains, and Markov decision processes. The text describes both algorithms and applications, enabling students to understand the logical basis for the algorithms and be able to apply them. The applications address problems in government, business, and nonprofit sectors. The author uses Markov models to approximate the random behavior of complex systems in diverse areas, such as management, production, science, health services, finance, and marketing.