Winter 2010

Instructors:

- Steven Lalley
- Mihai Anitescu
- Mihai's 310 Web Site

This course will consist of two five-week modules devoted to (1) stochastic simulation and (2) continuous optimization. Students will be expected to complete several substantial programming projects during the course. Following is a brief synopsis:

- importance sampling and sequential importance sampling
- MCMC (Markov chain Monte Carlo)
- perfect sampling
- particle filtering
- simulated annealing

Connections with some important problems of combinatorial optimization, including max-flow, graph-partitioning, and matching algorithms, will also be discussed. The utility of MCMC methods will be illustrated by a number of substantial examples, including

- simulation of self-avoiding random walks
- enumeration of contingency tables with fixed margins
- simulation of Ising models and spin glasses
- cypher-breaking

- Fundamentals of Unconstrained Optimization.
- Line Search Methods
- Trust-Region Methods.
- Conjugate Gradient Methods for Unconstrained Optimization.
- Quasi-Newton Methods (if time permits).
- Calculating Derivatives, Automatic Differentiation.
- Nonlinear Equations (and, if time permits, Nonlinear Least-Square Problems).
- Theory of Constrained Optimization.
- Fundamental of Algorithms for Nonlinear Constrained Optimization.
- Interior Point Methods for Nonlinear Programming.

- Monte Carlo Strategies in Scientific Computing by Jun Liu
- Algorithms by Cormen, Leiserson, Rivest, and Stein (Excellent Reference.)

- Radford Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods
- Chen et. al., Sequential Monte Carlo Methods for Statistical Analysis of Tables
- Madras & Sokal, The pivot algorithm: A highly efficient Monte Carlo method for the self-avoiding walk
- Alan Sokal, Monte Carlo Methods in Statistical Mechanics
- J. Propp and D. Wilson, Coupling from the past: a user's guide

- Assignment 1. Due: Thursday January 14
- Assignment 2. Due: Tuesday January 26 corrected January 17
- Assignment 3. Due: Tuesday February 2
- Assignment 4. Due: Thursday February 11

Coded Text for Assignment 3.

TSP Itinerary for Assignment 4.