This graduate textbook provides an alternative to discrete event simulation. It describes how to formulate discrete event systems, how to convert them into Markov chains, and how to calculate their transient and equilibrium probabilities. The most appropriate methods for finding these probabilities are described in some detail, and templates for efficient algorithms are provided. These algorithms can be executed on any laptop, even in cases where the Markov chain has hundreds of thousands of states. This book features the probabilistic interpretation of Gaussian elimination, a concept that unifies many of the topics covered, such as embedded Markov chains and matrix analytic methods.