1. An AI team is evaluating whether to use self-consistency prompting for a high-stakes medical triage classification task. They plan to run the same prompt 15 times and take the majority vote. Which statement BEST describes a key limitation of this approach they should consider?
- A. Self-consistency only works with temperature set to 0.0, making it identical to a single deterministic run
- B. Majority voting across runs can amplify a systematic bias or consistent error in the model's reasoning, not just random errors, potentially giving false confidence in a wrong answer✓ Correct
- C. Self-consistency is only applicable to open-ended generation tasks, not classification tasks
- D. Running the prompt 15 times is computationally equivalent to fine-tuning the model, making it impractical
Explanation
Self-consistency improves robustness against random variance, but if the model has a systematic bias or a consistent reasoning flaw (e.g., it always misclassifies a specific symptom pattern), majority voting will confidently return the wrong answer repeatedly. This is especially dangerous in high-stakes medical scenarios. Option A is incorrect; self-consistency requires varied outputs and thus uses temperature > 0; at temperature 0.0 all runs are identical and majority voting adds no value. Option C is incorrect; self-consistency is commonly applied to classification and question-answering tasks, not just open-ended generation. Option D is incorrect; running inference 15 times is simply 15x the inference cost, which is expensive but entirely different from the compute-intensive process of fine-tuning.