As I make progress with an upcoming article I'm writing on standard deviation, I will occasionally share some of the information I discover. For the data I'm using the monthly returns for the S&P 500 for the 36-year period ending December 2008. I am first calculating the results for the 36-month period ending this date, to conform with what is proposed for GIPS 2010. I will also evaluate the longer period and probably a few other 36-month periods throughout this time frame.
I haven't tested completely for normality, yet, but did discover something of interest. For the 36-year period I determined that at the 97.5% confidence level there should be 11 returns outside the range of plus-or-minus 1.96 standard deviations: there are 25, more than twice the expected number. Most (15) are at the low end. Other authors (e.g., Anson, Mark (2002), The Handbook of Alternative Assets. (Wiley)) have found similar results.
One reasonable question that might arise is "does it matter?" Well, if your analysis is based on an assumption which is flawed, one would suspect that your conclusions might also be flawed, yes? There have been studies that have shown that in some circumstances the absence of a normal distribution isn't a problem. But I like to hearken back to a standard line that IBM used to use when someone would do something other than what was specifically called for: "unpredictable results may occur." Bottom line: we just don't know. In some cases there may not be a problem, but in other cases there will. Over the 36-year period I'm reviewing there are more results outside the boundary just at the low end than are predicted for the entire distribution, so wouldn't we expect our assumptions to lead to "unpredictable results," which likewise might call into question any of our conclusions?
Standard deviation remains the most criticized risk measure, and probably with much justification. We like things that are easily understood: standard deviation fits this bill. However, we also like things that work properly...unfortunately, it's unclear that we can say this holds.