Advanced Dynamic Multi-Dimensional Optimisation and the Probability Approach

BEA510 Advanced Dynamic Multi-Dimensional Optimisation and the Probability Approach

  • Topics



    Review of continuous Time Models

    Dynamic Programming Equations

    Construction of the Approximating Markov Chain

    Computational Methods for Controlled Markov Chains

    Approximations in policy space

    Approximation in value space

    Combined approximations in policy and value space

    Accelerated Jacobi and Gauss-Seidel Methods

    Calculus of Variations: Infinite Horizon Problem Schemes

    Viscosity Solution Approach


  • Learning outcome

    Learning outcome

    This course is concerned with numerical methods for stochastic control and optimal stochastic control problems in continuous time and higher dimension (several control and state variables). The random process models of controlled or uncontrolled stochastic systems are either diffusions or jump diffusions. This area of research is very active and finds applications in all fields of science. The class of methods treated is referred to generically as the Markov Chain approximation method. It is a powerful and widely usable set of ideas for numerical and other approximation problems for either controlled or uncontrolled stochastic processes with important applications to deterministic problems as well.


    The basic idea is to approximate the original controlled process by an appropriate controlled Markov Chain on a finite state space. One also needs to approximate the original cost function by one which is appropriate for the approximating chain. These approximation must be chosen such that a good approximated solution can be obtain with a reasonable amount of  computation. An important criterion to be satisfied by the approximating process is often labeled as "local consistency".


    A main objective for the course focus on different ways to establish a proper discrete form of the associated variational formulation (e.g. the Hamilton-Jacobi-Bellman equation).


  • Teaching


    Normally offered in fall semester.
    The course is not offered on a regular basis - contact the course responsible.

  • Required prerequisites

    Required prerequisites


  • Requirements for course approval

    Requirements for course approval


  • Assessment


    Project and oral exam.


  • Grading Scale

    Grading Scale

    Grading scale: A-F.


  • Semester



  • Literature


    - Numerical Methods for Stochastic Control Problems in Continuous Time By Harold J. Kushner and Paul Dupois

    - Springer 2nd ed. (2001) in the series Applications of Mathematics: Stochastic Modelling and Applied Probability




ECTS Credits
Teaching language

Course responsible

Leif Kristoffer Sandal, Department of Business and Management Science