Hypothesis
Teams whose approval rates track agent reliability (calibrated trust) achieve lower escaped-defect rates at equal throughput than teams that over- or under-approve.
Proposed methodology
- Define calibrated trust as agreement between approval decisions and later-verified correctness.
- Observe approval practice and autonomy settings across teams.
- Analyse effects on throughput and escaped defects, with consent and without individual profiling.
Required data
- Approval decisions and their later-verified correctness, aggregated per team.
- Autonomy and escalation settings.
- Reporting format preferences, declared by teams.
Evaluation metrics
- Calibration of trust (approval vs. correctness).
- Escaped-defect rate.
- Throughput at equal quality.
Current status
Conceptual. No data has been collected and no experiment has been registered for this track.
Known uncertainties
- Measuring trust without profiling individuals limits resolution.
- Practices may change because they are measured.
Source references
- Lee, J. D. & See, K. A. (2004). Trust in automation: designing for appropriate reliance. Human Factors 46(1).
- Parasuraman, R. & Riley, V. (1997). Humans and automation: use, misuse, disuse, abuse. Human Factors 39(2).