Tuesday, March 08, 2011

the problem with saying a % prediction is wrong

wendell steavenson in the NYer:
Even after a month of demonstrations in Tunisia had brought about the downfall of President Zine El Abidine Ben Ali, on January 14th, some White House officials, along with American and Israeli intelligence experts, put the likelihood of a copycat revolution in Egypt at no more than twenty per cent
he then goes on to note that the prediction was wrong.

but was it? saying something is 25% likely to happen is not the same as saying it won't happen. predicting that something is unlikely is still predicting that it is a possibility. the experts did not say that there was a zero percent chance of a copycat revolution. so wouldn't that make them right?

unless the prediction is 0% or 100% you can't ever say a prediction about an either/or event are ever right either. say the model had predicted that egypt had a 75% chance of having a copycat revolution, and then it happened. what about that 25% of doubt? the model didn't say it definitely would happen but then it definitely did happen. any time anyone makes a prediction that isn't 0% or 100% isn't it per se wrong? no matter what happens, it will never be completely right. hasn't the predictor admitted that from the outset by giving a percentage chance rather than a certainty?

things happen, or they don't. by giving a 25% chance that there would be a copycat revolution the model was saying 1 time out of 4 it will happen. but then history unfolds and we only get to see one of those times, the one that actually happened. we don't know what happened in those three other cases, because those cases didn't happen. or rather, we do know what happened in those three other cases, they didn't.

the percentage only makes sense when the event hasn't happened yet. once it has happened, there is no percentage chance, just what occurred. we only use percentages because we can't completely predict the future. so why get upset when it turns out that we didn't completely predict the future?