A Standard Testbed for Simulation Optimization Algorithms

Abstract

Much effort in simulation optimization has gone into showing algorithm properties that aren't practically very useful, such as performance in the limit or on easy problem cases. A suite of standard problems and comparison measures has been proposed in an attempt to encourage simulation optimization researchers to pay more attention to finite-time properties and performance on real-world problems. We are evaluating the usefulness of these problems and comparisons at distinguishing between simulation optimization algorithms and at pointing out those with the most desirable properties. To date, each of two algorithms has been implemented for each of two proposed benchmark problems. At least one of the problems evaluated, the Rosenbrock problem, seems to be worth including in a test suite.

Resources

• The paper we're working from:

  [pdf]

  R. Pasupathy, S. Henderson. "A Testbed of Simulation-Optimization Problems." Proceedings, 2006 Winter Simulation Conference.

• My final report from the fall '06 semester:

  [pdf]

  E. Herbst, S. Henderson. "Evaluating a Standard Test Suite for Simulation Optimization Algorithms."

• I'm happy to answer questions via e-mail.