Statistics 310: Optimization and Simulation
Winter 2010

Instructors:

• Steven Lalley
• Mihai Anitescu
• Mihai's 310 Web Site

1. Stochastic Simulation:

• importance sampling and sequential importance sampling
• MCMC (Markov chain Monte Carlo)
• perfect sampling
• particle filtering
• simulated annealing

Connections with some important problems of combinatorial optimization, including max-flow, graph-partitioning, and matching algorithms, will also be discussed. The utility of MCMC methods will be illustrated by a number of substantial examples, including

• simulation of self-avoiding random walks
• enumeration of contingency tables with fixed margins
• simulation of Ising models and spin glasses
• cypher-breaking

2. Continuous Optimization:

• Fundamentals of Unconstrained Optimization.
• Line Search Methods
• Trust-Region Methods.
• Conjugate Gradient Methods for Unconstrained Optimization.
• Quasi-Newton Methods (if time permits).
• Calculating Derivatives, Automatic Differentiation.
• Nonlinear Equations (and, if time permits, Nonlinear Least-Square Problems).
• Theory of Constrained Optimization.
• Fundamental of Algorithms for Nonlinear Constrained Optimization.
• Interior Point Methods for Nonlinear Programming.

References

A. Stochastic Simulation

---

Books:

• Monte Carlo Strategies in Scientific Computing by Jun Liu
• Algorithms by Cormen, Leiserson, Rivest, and Stein
(Excellent Reference.)

Articles:

• Radford Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods
• Chen et. al., Sequential Monte Carlo Methods for Statistical Analysis of Tables
• Madras & Sokal, The pivot algorithm: A highly efficient Monte Carlo method for the self-avoiding walk
• Alan Sokal, Monte Carlo Methods in Statistical Mechanics
• J. Propp and D. Wilson, Coupling from the past: a user's guide

B. Continuous Optimization

• Numerical Optimization (Springer Series in Operations Research and Financial Engineering), by Jorge Nocedal and Stephen Wright.

Assignments (Stochastic Simulation):

• Assignment 1. Due: Thursday January 14
• Assignment 2. Due: Tuesday January 26 corrected January 17
• Assignment 3. Due: Tuesday February 2
• Assignment 4. Due: Thursday February 11

Darwin Finch Data

• finches.pdf
• finchData.txt