You're probably familiar with the statistical concept of type 1 and 2 errors. One way to look at it is that one addresses the case where you think you're right, but you're wrong; the when you think you're wrong, but you're actually right.
A thought occurred to me last night that the classic Clint Eastwood movie Dirty Harry shows this in two parts (warning, graphic content):
:
In one case, the criminal was thinking that Harry had fired only five bullets, but he was wrong, he had fired six, and his gun was empty. In the other case he thought he had fired six bullets, but he was wrong, too, as Harry had one bullet left. But, which was the worse of the two errors?
In evaluating Type I and II errors, it's helpful to investigate the impact of both errors, to determine which, if we had to, we'd prefer to make.
I'm finalizing a study on attribution, where, among other things, I've found cases where holdings-based attribution can cause effects to show the wrong sign. For example, we might see a positive allocation effect, when it should be negative, or a negative allocation effect when it should be positive. Both are errors, but they mean different things.
If we mistakenly report allocation that actually hurt performance as being positive (i.e., contributing to performance), then we're misrepresenting our skill to the client, telling them that we did something right, when we didn't.
If we report allocation as a negative, meaning it hurt performance, when in reality it was positive, contributing to the outcome, then we're hurting the manager. One might even suspect that a manager could be terminated for failing when they actually succeeded.
Again, both are errors, but which is worse? I guess it's a classic case of, it depends.
Subscribe to:
Post Comments (Atom)
.png)
Your question is (as usual) both interesting and useful. I have to say that your conclusion is also spot on - which error is worse does indeed depend on the circumstance. I have heard of two illustrations that provide differing views on which error is more serious, thereby supporting your idea that the seriousness of the type of error does depend on the circumstance. It's often said that a type 1 error is a "false positive" and this is most often used in the context of medical tests. So, if the test says you have a serious condition and you don't, you get unnecessary treatments, which may be unpleasant and costly, but are not typically life-threatening. On the other hand, a false negative (type 2 error) would certainly be life-threatening in this context, since you would continue to ignore treatment for a condition that could be cured, but which could also end your life prematurely if not treated. Using this example, I would say that a type 2 error is the more serious.
ReplyDeleteThen again, let's take an example that I heard Dan diBartolomeo use at a conference. The premise is that "I can fly." Of course the "null hypothesis" is that I cannot fly, so let's assume that the null hypothesis is true. If I commit a type 1 error and incorrectly reject the idea that you cannot fly, I encourage you to jump off a building and you then fall to your untimely death. On the other hand, if you truly have the ability to fly (so that the null hypothesis is false) and I fail to reject the idea, then I convince you that you cannot fly, and you spend your time walking, waiting for buses and trains or stuck in traffic jams - all of which are inconvenient but not life threatening (unless you live in New York City or Los Angeles.) So in this instance, the Type 1 error is the more serious kind.
Steve, thanks for your comment (sorry for the delay in posting). In general it probably always "depends" as to which error is worse, but I think it worthwhile to consider this matter before making decisions. I have found it a worthwhile exercise, even when the matter may not seem that important.
ReplyDelete