Topology makes it way into performance and risk

Yesterday, The Spaulding Group held a luncheon in NYC, where Jed Schneider, CIPM, FRM reviewed some of the findings from our recent Performance Attribution survey. As with all of these research projects, some very interesting insights can be drawn. And since this is the fourth time we've surveyed asset managers on this topic, we can compare results from one period to another, to identify trends, changes, etc. Our cosponsors for this survey were:

• BiSam
• DST Global
• FirstRate
• Morningstar
• Statpro
• VPD

As with all of our surveys, we invited our cosponsors to come to the luncheon and make brief presentations, and four did. One, Steve Shefras from BiSam, mentioned how it is often said that risk and return are two sides of the same coin. He went on to say that he believes they're on the same side. I immediately thought of a totally different anaology: a Möbious strip!

You may be familiar with them. They come from the mathematical field topology, that deals with "mapping." A Möbious strip is a one-sided object. You can build your own by taking a long strip of paper, twisting it, and then connecting the ends. If you traverse either "side," you'll cover both "sides,"  meaning there is only one side.

And so, rather than saying that risk and return are on the same side of a coin (which encourages one to ask, "what's on the other side?"), we could say that they're both on a Möbious strip, and therefore on the same (and only) side.

Performance & ethics

Calculating rates of return really isn't that difficult. And while we may debate whether returns should be calculated using money-weighting or time-weighting, such issues pale in comparison with the broader aspect of ethics.

I must confess that I initially thought that the Certificate in Investment Performance Measurement (CIPM) Program's emphasis on ethics seemed a bit excessive. And perhaps the exam questions could be geared more to the issues and situations that performance analysts and performance heads are more likely to encounter. But the issue of ethics shouldn't be ignored. We seem to be reminded of this on a fairly regular basis.

Yesterday's Wall Street Journal has an article on Raj Rajaratnam and his Galleon hedge fund. Recall that Mr. Rajaratnam has been accused of trading on inside information. His prowess at building relationships, which allegedly resulted in him gaining access to confidential information, allowed him to build a successful business and amass a fortune of more than a billion dollars.

Galleon's returns were apparently quite impressive. And one might not find any fault with the accuracy of the valuations or the return methodology employed. But what is the value of the returns if, as has been suggested, they reflect the results of illegal activity?

What is the responsibility of the performance analyst or manager, should they have reason to believe that the returns reflect illicit activity? Perhaps questions like these should be added to the CIPM program. What would YOU do, if you had reason to believe that the firm's results were arrived at through illegal means? Something to ponder, yes? 

p.s.,We are investigating another fraud case which might involve a firm that claimed compliance with the Global Investment Performance Standards (GIPS(R)). We will provide details once we've done further vetting of the information we've seen so far.

How big should our performance measurement team be?

One question that occasionally surfaces is how big a performance department should be. There is no simple answer and it's difficult to decide by simply doing a comparison with other firms. Some key points:

• all firms are different. Okay, maybe a bit of hyperbole, but there are a variety of models that can be applied, which make comparisons difficult.
• assets under management may be a good gauge to say how much you can afford but not necessarily how big you should be.

The criteria that should be considered when deciding staffing include:

• number of accounts: the more you have, the larger your staff has to be
• reliance on spreadsheets: the greater the reliance, the greater the amount of manual effort and therefore the greater need for staff. Packaged software typically has many tools that aid the team, thus reducing the manual effort.
  
• activities of the group: performance measurement can address returns (portfolio and  subportfolio), risk, attribution (equity, fixed income, balanced, etc.), and the Global Investment Performance Standards (GIPS(R)). If the firm is only engaged in returns, their staffing needs won't be too great; but, if they take on more tasks, more staff may be needed
• reporting: if the group handles both internal and client reporting, staff is needed; also, if there are custom reports or if the frequency varies from daily to monthly to quarterly, more staff may be needed
  
• reliance on and support from back office operations: if the operations staff is clued into the needs of performance, they may be helpful in addressing data issues; but, if they are clueless (okay, a bit strong, but you get the point) about their needs, they may not be as helpful as one might hope, meaning more work falls to the performance measurement team.
• client demands: if the firm caters to a lot of individual needs, including custom reporting or fielding inquiries, the workload increases, meaning the need for staff.
  

There are no doubt other criteria to consider, and your thoughts are invited.

The answer to the amount of staff one needs is hardly a simple one. Hopefully this at least gives you some ideas.

What's the standard way to handle cash flows?

Here, I'm addressing cash flows from a return calculation perspective.

First, recall that in two weeks we'll have a requirement for GIPS(R) compliant firms to revalue portfolios for large cash flows, where the firm decides what "large" represents. But what happens when the flows aren't large?

I'd guess that most firms treat all flows as start-of-day events. There are, of course, some firms that treat them as end-of-day events. My recommendation: treat inflows as start-of-day and out-flows as end-of-day. I'd like to provide you with a clear explanation and rationale for this, but I'm unable to do this. Anecdotal evidence suggests that this is the best approach. Of late we've seen many firms and vendors begin to adopt this practice. That doesn't make it "right," but it at least adds support for my position.

Regardless of what approach you take, ensure that you're consistent in its application.

p.s., I just realized I didn't answer my own question! There is NO standard on how cash flows should be handled. GIPS doesn't address this, either. The only "standard" would be to be consistent...i.e., don't use cash flow timing to "game" the process to generate higher returns.

Dealing with breaks

A client recently asked about dealing with "breaks" in performance. First, what IS a break? We'd define it as a temporary loss of discretion over a client's assets, during which time no trading can be done. Breaks can be caused by changes in custodians as well as for other reasons. Another term for a "break" is a "gap." I opined on this topic a few years ago in our newsletter, regarding the calculation of returns (that is, can you link across gaps for returns). The client's question had to do with GIPS(R).

If you search the GIPS Q&A database you'll only find one item dealing with this topic, and it doesn't really address temporary breaks. I recall discussing this a few years back with a group and we couldn't arrive at any clear consensus. Some thought ANY break meant that performance stops, while others felt that there should be an assessment as to whether or not there would likely have been any trading during the break: if not, then what's the harm in linking across it?

I tend to be in the latter group's camp: that is, when one has a break, they should determine the likelihood of trading occurring. If, for example, the manager is very much a buy-and-hold manager, who trades infrequently, then a gap of even a few weeks might not cause a problem. However, if the manager trades almost daily, then even a short break would be problematic.

Ideally, the manager has other similar accounts that they can compare the account-with-the-break to, to determine if there truly was an absence of trading. This is where the verifier can come in...to provide an additional degree of analysis.

It would be nice to see something formal regarding this topic, but for now there is little guidance. Hopefully mine won't conflict with anything that comes in an official capacity.

Calculating returns ... the wrong way

We got a call recently from a firm that wants us to review their method to derive returns. They take their beginning market value plus cash flows to determine an average capital base; they then determine their account's appreciation for the period and calculate a simple average to derive the return. Extremely intuitive, yes?

But sadly wrong, too. (Hopefully you agree).

This isn't the first time we've encountered firms who employ a proprietary approach to derive returns. Not long ago during a conference lunch I was sitting next to an attendee who told me of the approach they had developed. Again, quite intuitive ... one that surely no one would find objectionable. Unfortunately, it, too, was invalid.

Long ago, before the publishing of performance books and various standards, individuals were often compelled to figure out a return methodology on their own. Fortunately, this is no longer the case. So, if you have such a formula in your shop, perhaps you should have it checked out.

Waltzing through the blogosphere

I guess it's not surprising that as a blogger, I occasionally wonder around looking at other blogs ... I regularly visit about a dozen and am always looking for new ones to add to my list. Today I've visited several new sites and have picked up a few ideas.

As far as I know it, my blog remains unique in its focus, addressing topics such as the GIPS(R) standards, rates of return, performance attribution, and risk on a regular basis.

To date we've attracted some 500 visitors, which I guess is pretty good as a start. Our newsletter has several thousand subscribers, so perhaps we have a bit to go to get to that level. Visitors have come from every continent and many different countries. And while only a few have signed up as "friends" so far, we know that many circle back regularly. The idea of being a "friend" is that you are notified when a new post is added.

Photo by chrismar

I'm pleased that we've had a few comments regarding posts, as this suggests that some agree while others not with what has been offered.

I've now been blogging for six months and am open to your ideas, thoughts, suggestions. Feel free to contact me directly (DSpaulding@SpauldingGrp.com) or by posting a comment. Thanks!

Rates of return ... come again?

I stumbled upon a website today that provided the following brief explanation about returns:

"To evaluate the performance of a portfolio manager, you measure average portfolio returns. A rate of return (ROR) is a percentage that reflects the appreciation or depreciation in the value of a portfolio or asset"

We measure "average" returns? I don't think so. Average returns have been shown to have zero value. A classic example: Year 1 +100 return, Year 2 - 50% return, average= (100 - 50) / 2 = 25 percent. Now, let's use some dollars: start with $100; at the end of year 1 you're at $200, then at the end of year 2 you're at $100, meaning zero percent return.

In addition, while there are times when a return will reflect the appreciation or depreciation, once we introduce cash flows, forget about it! Recall that time-weighting can yield funny situations, like having a positive return but losing money.

I guess the lesson is: be careful about what you read on the Internet ... may not always be correct.

More precise returns ... why did it take SO long???

I'm in Toronto for a couple days teaching a performance class for a client. And, as often happens, a thought occurred that seemed interesting.

Peter Dietz introduced the "original Dietz formula" in 1966, which treats cash flows as occurring mid-period; this was adjusted several years later to day-weight the flows (what has become known as the "Modified Dietz Formula"). Why didn't he introduce this earlier?

Also, in 1968 the Bank Administration Institute pointed out that the Exact Method (where we revalue portfolios whenever flows occur) is the best approach to time-weighting. And so why didn't they encourage firms to use it sooner?

In both cases the answer is the same: technology. In 1966 and 1968 we didn't have personal computers, spreadsheets, or even calculators. No doubt many folks calculated returns by hand (math by hand? perish the thought!). To do a mid-point method is pretty simple by hand but to add day-weighting could complicate the process quite a bit.

And to use the Exact method would have required access to daily prices: in the mid '60s? Right! Forget about it!

Makes sense, yes?

Announcing our question & answer protocol

Through this blog I recently received a question that wasn't related to a specific post. I opted not to respond because (a) I didn't know who it was from (it was sent anonymously) and (b) it didn't fit what it was tied to. I will be happy to respond to questions relating to a blog piece, whether they're sent anonymously or not.

And, I will be happy (usually) to respond to questions to non-blog-initiated topics, provided the sender identifies themselves. Feel free to ask questions regarding GIPS(R), attribution, risk measurement, returns, etc. You can send these directly to my e-mail address (DSpaulding@SpauldingGrp.com). If we feel the question should be responded to in the blog or our newsletter, we will do this, and the questioner will be sent a response directly, too.

Hope this sounds reasonable. As always, your thoughts are invited.

Measuring

Methodology, like sex, is better demonstrated than discussed
- E.E. Leamer

I'm reading Measurement, Design and Analysis by Pedhazur & Schmelkin for a course I'm taking and am finding it quite interesting. While we often address topics such as how to measure returns or risk, there is an entire discipline that addresses the broader subject of "measurement."

One thing the authors address is the average, something many of us calculate regularly. When it comes to performance, the reality is that in many cases, NO ONE gets the average return! Think about your GIPS composites: you show an asset-weighted return (which is an average), but do any of the accounts in the composite achieve it? In my experience as a verifier, it's not uncommon to find that no one does. This is one reason we require a measure of dispersion, to show the breadth of returns around this average.

There is such an allure, an almost magical quality,

in specialized terminologies, in formulas and fancy analyses
- Pedhazur & Schmelken

The authors suggest "that the choice of an analytic approach is by no means a routine matter," and I suggest that many of us wrestle with this on a regular basis. We are frequently told by firms that they have to report time-weighted returns, and sometimes I ask "why?" The typical response: "because GIPS requires it." BUT, in many of these cases GIPS doesn't apply, and yet they believe they're bound to the tradition of using time-weighting, even when it doesn't make sense! To paraphrase these authors, "returns and risk measures are generally presented with little or no attention to substantive context, the characteristics of the manner in which they are applied, or the properties of the measures used." To extend my paraphrasing a bit further, "knowledge of the methods and analytic approaches employed is essential for critical evaluation of a performance or risk report." But how extensive IS this knowledge?

The authors cite D.A. Freedman's exhortation to "start a new trend." Well, Steve Campisi, Stefan Illmer, and a few others have joined me in an effort to do just that, regarding at least the way we measure returns, and more likely much more. As I continue to make my way through this 800 page book, don't be surprised if you see further references to it in the future.

p.s., if you're wondering how the lead quote fits in, I can't recall ... I'm sure I had a good reason for using it, other than it sounded neat. Plus, it gave me the opportunity to show off one of my favorite clip art pieces!

Trust, but verify

John Simpson and I are conducting a GIPS(R) verification and I'm reminded of Ronald Reagan's advice to trust, but verify.

On multiple occasions we've encountered situations where the returns are incorrect, as produced by the software. Sometimes it's because the formula is simply wrong. An example of this surfaced a few years ago during another verification where I discovered that the composite return methodology the client's vendor was using was invalid. Another time a major vendor's calculation was in error: I told the vendor they had a "bug," but they said that no, the software was doing what they wanted. And they were correct! Unfortunately, what they wanted the software to do was wrong. Another vendor had a known "bug" that caused errors whenever certain corporate actions occurred; for some reason they allowed this problem to exist for several years before addressing it.

Sometimes a verifier will offer their clients formulas that turn out to be wrong. This happened recently with a formula to do carve-outs. The client had used this formula for several years, only to learn (from us) that it wasn't correct. We have also encountered rate of return formulas, though appearing to be quite right, are in reality, quite wrong.

It's unfortunate that this occurs more often than it should. Clients pay their software vendors and verifiers lots of money, and expect accurate and correct information in return.

What happens if you use the wrong stats?

Continuing our discussion of Michael Lewis' Moneyball, I think there's a HUGE parallel between baseball statistics and what we do in investment performance measurement. Both deal with measuring performance: the performance of baseball players / the performance of money managers.

Lewis points out that for the first 150 years of baseball, the wrong statistics were used to evaluate the performance of players. As a result, the wrong ones were often rewarded and chosen, resulting in teams not doing as well as they had expected, given the "talent" they selected.

While investment performance's history is much shorter than baseball's (roughly 40 years), we have had the same challenges because we quickly adopted certain measures (e.g., standard deviation for risk, time-weighting for performance) which arguably are WRONG much of the time! And so, what's the consequence? Misleading information, misinterpretation of results, mis-allocation of resources.

Not everyone in baseball has "signed on" to the new statistics, although those who haven't will continue to suffer. The same can be said in our industry, where most firms have yet to see the wisdom of alternative measures. We should be glad that it hasn't taken us 150 years to figure out the error of our ways.