Hypothesis
A simulation calibrated on historical data ranks candidate team configurations in the same order as their observed outcomes more often than chance and more often than simple heuristics.
Proposed methodology
- Build a scenario simulator with explicit assumptions (the Lab is a heuristic prototype).
- Calibrate on historical data; hold out later projects.
- Compare predicted rankings of configurations with observed outcomes.
Required data
- Historical team configurations and outcomes.
- Constraint definitions: budget, deadline, review capacity.
Evaluation metrics
- Rank correlation between simulated and observed outcomes.
- Regret of the recommended configuration.
Current status
Conceptual. No data has been collected and no experiment has been registered for this track.
Known uncertainties
- Few organizations try several configurations on comparable work, so counterfactuals are scarce.
- Optimization may exploit simulator errors.
Source references
- Law, A. M. (2015). Simulation Modeling and Analysis (5th ed.). McGraw-Hill.