The goal of prognostic decision making (PDM) is to utilize information on anticipated system health changes in selecting future actions. One of the key challenges in PDM is finding a sufficiently expressive yet compact mathematical representation of the system for use with decision optimization algorithms. In this paper we describe a general modeling approach for a class of PDM problems with non-linear system degradation processes and uncertainties in state estimation, action effects, and future operating conditions. The approach is based on continuous Partially Observable Markov Decision Processes (POMDPs) used in conjunction with 'black box' system simulations. The proposed modeling framework can be cast into simpler representations, depending on which sources of uncertainty are being included. The approach is illustrated with a mission planning case study for an unmanned aerial vehicle (UAV). In the case study a PDM system is tasked with optimizing the vehicle route after an in-flight component fault is detected. A stochastic algorithm (based on particle filtering) is used for decision optimization, with a second, deterministic algorithm providing a performance evaluation baseline. Both algorithms utilize a UAV physics simulator for generating predictions of future vehicle states. Performance benchmarking is done on a set of mission scenarios of increasing complexity.