Hypothesis
Models using historical execution features produce better-calibrated duration ranges than expert estimates and naive reference-class baselines, measured prospectively.
Proposed methodology
- Register predictions in a Prediction Ledger before outcomes are known.
- Compare against reference-class and expert baselines.
- Score with proper scoring rules; publish per model version.
Required data
- Completed project histories with planned vs. actual dates and costs.
- Feature windows frozen at prediction time.
Evaluation metrics
- Interval coverage (does the 80% range contain the outcome 80% of the time?).
- Brier score and CRPS.
- Improvement over baselines.
Current status
Planned. No data has been collected and no experiment has been registered for this track.
Known uncertainties
- Projects change scope; outcomes are hard to define.
- Too few comparable projects may exist in any one organization.
Source references
- Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review 78(1).
- Gneiting, T. & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. JASA 102(477).