Employing medical terms to performance

I was recently introduced to a new word, which if truth be known, I wish I hadn't. But given that it's an interesting word, I can see some applicability in our world of performance measurement.

Idiopathic means "without a known cause." My favorite source for definitions defines it as

adjective Pathology .

of unknown cause, as a disease.

 

And so, there's a clear link to medicine. But "unknown causes" aren't limited to this field, are they? And we have a whole segment of our industry devoted to identify causes; we call it "performance attribution." 

 

Imagine what it would be like without these tools to properly evaluate and assess what causes the returns and excess returns our portfolios achieve. Without attribution, our results are idiopathic, since we cannot identify what caused them. Granted, one might guess, but without proper analysis the accuracy of the assessment is weak, at best.

 

Much of life is idiopathic, is it not? But in those areas, such as performance measurement, where we can perform the analysis, we should, as the value from these discoveries is great. 

Should the GIPS EC learn from the SEC?

Yesterday’s WSJ  carried an interesting op-ed piece (“SEC Smackdown”) regarding the U.S. Securities & Exchange Commission’s attempt to alter the way committees are voted on, an idea that was called “unutterably mindless” by the D.C. (District of Columbia) Circuit Court of Appeals. The panel sided unanimously with plaintiffs (who challenged the new rules) because “the SEC didn’t ‘determine the likely economic consequences’ of the rule and its effect on ‘efficiency, competition and capital formation.’” The SEC is required to do this by law, and the piece mentioned that this was the “fourth time in recent years that the court rejected SEC rules on similar grounds.” And so, what does this have to do with the Global Investment Performance Standards (GIPS(R)) and the Executive Committee (EC)?, you might ask.

Consider the current proposed changes to the Performance Examination Guidance Statement, which is open for comments until August 31. Included are suggestions that verifiers should not rely upon external records (e.g., custodial statements) provided by their client, but rather go directly to “independent external parties” for these records. We discussed this in a prior post, our July newsletter, and yesterday’s monthly webinar. I am opposed to these suggested changes because of the added cost and complexity they would bring.

Leading up to the release of GIPS 2010, I chaired a meeting where we discussed some of the changes. During the session I voiced strong opposition to the then proposed (since adopted) recommendation that compliant firms provide their existing clients with copies of the composite presentation(s) they’re in on an annual basis. My objections were based on the anticipated excessive complexity and added cost that would result, since roughly half of a firm’s clients have returns below the composite’s average, who therefore might be inclined to require their manager to explain why. One individual who attended, who isn’t affiliated with any asset manager, responded “well, isn’t that just too bad.” I’ll admit that such a response was far from expected and wasn’t well received by many of the others in the room.

I believe that just as with the SEC, the GIPS Executive Committee should determine the likely economic consequences of changes to the standards, as well as their effect on the efficiency of compliance. In my years of involvement with the standards I am unaware of anyone voicing these concerns (as I’ve shown above) as loudly as I have, and perhaps my concerns are thought of by others as being a nuisance, or perhaps “just too bad,” but the standards were never intended to be overly burdensome on a compliant firm’s budget.

Perhaps the GIPS EC could learn from the recent Court of Appeals ruling and consider taking these points into consideration with future changes. Perhaps make them a formal part of their own rules for introducing change. It would be nice to see some sensitivity in this regard.

By all means, if you feel differently (or, perhaps agree), please chime in; thanks!

A graphical history of verifications

In preparing for today's webinar on GIPS(R) (Global Investment Performance Standards) Performance Examinations, I constructed, what I believe to be, the first graphical depiction of the confusing history of verification:

A brief explanation is in order:

First, we can consider Level I to be analogous to today's "verifications," while Level II is akin to today's "performance examinations." Now to the figure:

• In the beginning, under the "old" AIMR-PPS(R), we began with the requirement that firms MUST undergo a firm-wide Level I verification before commencing with a (composite level) Level II verification.
• Things got a big awkward when the large accounting firms declared that they would not do a Level I, but would do a Level II. And so, a "Modified Level II" verification was added, which was at the composite level, but incorporated the elements of Level I.
• The 1997 edition of GIPS changed the original Level II to its Modified form, but dropped the qualifier.
• Moving onto GIPS, the draft did not reference performance examinations, but only firm-wide verification.
• However, when the first edition was presented in 1999, performance examinations appeared.

And so, briefly, this is what has occurred. Oh, and it's not too late to sign up for today's webinar, which will be at 11:00 AM! Contact Patrick Fowler (732-873-5700) for details. We already have quite a crowd showing up!

Error correction, made perhaps TOO easy!

As of January 1, 2010, any firm claiming compliance with the GIPS(R) standards (Global Investment Performance Standards) must have a documented error correction policy. The policy that the firm comes up with should cover all aspects of their compliant presentations, both numeric (covered in section I.5 of the standards; e.g., returns) and non-numeric (i.e., disclosures, covered in section I.4; e.g., composite descriptions). Many firms fail to address the latter, and often only address returns when it comes to numeric errors, even though all numeric values (required and recommended (that appear); e.g., assets under management) need to be noted, too.

As for the non-numeric disclosures, what needs to be said? Well, some firms have taken the easy route and simply stated that any errors in this category will be treated as "non material." I have to object to this because materiality should include anything, found to be in error and then corrected, that might cause the recipient of the previous non-corrected version to possibly draw different conclusions. And surely there are required disclosures that fall into this category. For example, the composite description. Granted, a spelling error is probably non-material, but if the wording has to be changed, might that not constitute a material change? It would be very difficult to craft rules that could somehow distinguish between material and non-, when it comes to disclosures, (i.e., to bifurcate your rules into "material" and "non-material" for each disclosure) other than for spelling errors.

And so, I think firms have two choices:

1. Review every required (and recommended, if they appear in the presentation) disclosure to identify which ones would fall under the "material" category, and list these (you could state that the balance are non-material), or
2. Indicate that any error, other than spelling to disclosures will be deemed material.

Is the second option onerous? Well, it could be, so #1 would be perhaps better. But if your diligent in ensuring that your wording is correct, you'll never encounter an error, and so won't have to worry about it.

To me, these are the only choices available to you. But, if you have other ideas, please let me know!

As the saying goes, when it rains, it pours

The next few months will see me doing a fair amount of public speaking.

Next Monday I will host our firm's monthly webinar, when I'll discuss at length the subject of GIPS(R) (Global Investment Performance Standards) Performance Examinations.

Later that week (Thursday, July 28 @ 2:00 PM EST, to be exact), I will conduct a webinar for the Advent Users' Group titled "Rethinking Performance Measurement."

I will deliver a luncheon speech before the NYSSA (New York Society of Security Analysts) titled  "Performance measurement and alternative investments: where things stay the same and where they diverge" on Tuesday, September 27.

The NAMIC (National Association of Mutual Insurance Companies) invited me to deliver a talk at their Investment Workshop in November, but I had to ask John Simpson to take this instead, as I will be a few thousand miles away delivering the keynote speech at a Risk Management Symposium in Kuala Lumpur. That same week I will deliver a talk before a Directors' Convention (also in Kuala Lumpur).

I very much enjoy public speaking and try to fit events in as best I can. Earlier this year I delivered talks in Beijing and Singapore, and of course at PMAR and PMAR Europe, but had little time to take on much more because our GIPS and non-GIPS verification practice has grown quite a bit, as have our consulting engagements, which often leaves little time for these programs. Fortunately, there has been some flexibility in my schedule to allow me to do these upcoming sessions.

Because John Simpson and Jed Schneider are also accomplished public speakers, we are usually able to accommodate groups who host events. Anyone interested in having us speak should contact Christopher Spaulding or Patrick Fowler.

Expanding ones vocabulary

I don't know about you, but often when I learn a new word I want to take ownership of it, by using it in sentences and making it part of my lexicon. To do this I often have to search for opportunities, which perhaps aren't as elegant as they might otherwise be.

In submitting my comments regarding the proposed changes to the Performance Examination Guidance Statement, I couldn't recall one particular word that would have worked, but then saw it in a front page article in yesterday's WSJ. The subheading: "Authors Who Crave Verisimilitude Learn Secrets of Bodice Ripping"; you can probably guess which word I am referring to.

Well, I came to know the word quite some time ago, but hadn't really taken ownership of it, but my comment letter provided me the opportunity, but I failed to take advantage of it. And "how exactly" you might ask? Well, the GIPS(R) (Global Investment Performance Standards) Executive Committee has suggested that perhaps verifiers should seek out documentation for the client being examined from independent parties, such as custodians and brokers. In my response I suggest that it's possible that to save the verifier the hassle, they could perhaps ask the client to contact these parties and ask them to provide the materials. However, who's to say that the client, who might be committing fraud, might not "go the extra mile" and fabricate what appears to be legitimate materials from these parties? And so, the verisimilitude they present might fool the verifier, right? This, to me, suggests that one cannot possibly rely upon the money manager as doing this on behalf of the verifier, but rather that the verifier must do all the "leg work" themselves.

And so, while I missed out on the opportunity to employ said word in my letter (and for that matter, our July newsletter which is about to be published), I got to here AND hopefully provided you with some insightful information about this document; more to follow!

p.s., if "verisimilitude" is a new or perhaps forgotten word for you, it means "the appearance or semblance of truth; likelihood; probability; something, as an assertion, having merely the appearance of truth."

LeRoy Neiman's influence on performance measurement

More than 10 years ago we held a Performance Measurement Forum meeting at the W Hotel in Los Angeles. Our meeting room abutted the swimming pool, and I happened to notice famed artist LeRoy Neiman sitting in the pool area. I mentioned this in the meeting, and later that afternoon our then editor of The Journal of Performance Measurement(r) saw Mr. Neiman in the lobby. She took the opportunity to approach him and ask "Are you Leonard Nimoy?" As you might imagine, we had a good laugh over this, and she was a good sport about her error.

In today's WSJ there's an article that informs us of how the Mr. Neiman is seeking to inspire a new generation of artists. I was particularly struck by his suggestion that "all young artists need to be amused in a classroom to maintain interest."

I have felt this way about performance measurement. Thus my attempt (though sometimes lost on the students) to interject humor whenever possible; I do the same with my public speaking. I learned the value of humor in presentations close to 40 years ago, when, as a newly minted army second lieutenant, I attended a class on a rather boring subject during Field Artillery Officer's Basic at Ft. Sill, Oklahoma. The instructor would interject a joke roughly every 40 to 60 minutes. Not only did this make us laugh, it also raised our attention levels.

Granted, not everyone is good at telling jokes or using humor, but it's a skill that can be learned. I'm pleased that many speakers we invite to PMAR, such as Steve Campisi and Carl Bacon, strive to liven up their presentations with a bit of humor. Shocking as it may sound, to some performance measurement, risk, benchmarks, and the like can get a bit dry once in a while, and there's nothing like a good joke to help keep people awake.

p.s., LeRoy Neiman's image above comes directly from the WSJ article.

Tests: too hard, too easy, and the world of cheating

This past week saw a few articles appear dealing with test taking. The WSJ reported on teachers who cheat for their students, so that the students' scores on standardized tests are higher, presumably so that the teachers can qualify for monetary benefits (or perhaps to avoid being terminated). The article cited one student who refused to take the test, but somehow managed to pass! Sadly, even in the Garden State (New Jersey), home of some of the most corrupt politicians in the States, apparently is also home of some corrupt teachers, as we are seeing evidence of cheating, revealed in last Saturday's Home News Tribune.

At the other end of the spectrum we learned of EU Bank exams that are apparently so easy that too many banks refused to fail them, as reported in this past weekend's Wall Street Journal.

An exam that's near and dear to many of us in GIPS(R)-land (Global Investment Performance Standards) is the Performance Examination, which can be done alongside, or after a firm undergoes, a verification. The GIPS Executive Committee has presented us with a proposed revised version of the Examination Guidance that suggests that verifiers perhaps should get the required account details (holdings and transactions) from an independent external party (e.g., custodians or brokers), offering three levels for your consideration:

• “preferable” to obtain from independent external parties
• “must make every reasonable effort” to obtain from independent external parties
• “must” obtain from independent external parties.

I consider these rather extreme measures, that appear to be aimed at the likes of the currently incarcerated Bernard Madoff; must we all suffer as a result of his misbehavior?

In this month's soon-to-be-published newsletter, I go into some detail on the exposure draft (the draft is available for public comment until August 31). In addition, I will discuss performance examinations in general, as well as the exposure draft, at this month's webinar (Monday, July 25, @ 11:00 AM, EST). If you're interested in participating in the webinar, please contact Patrick Fowler. And please review the draft so that you can comment; this document affects both verifiers and asset managers.

Providing performance results in a "non-GIPS" world

We have been in discussions with a firm that doesn't claim compliance with GIPS(R) (Global Investment Performance Standards), but still wants their returns "verified." They wanted to know what they need and what their options are.

The introduction of GIPS doesn't mean that firms can't show prospects performance results; granted, GIPS is considered "best practice" within our industry, but there can be a variety of reasons why a firm isn't able to comply. And so, what can they do? Here are three approaches:

1. Hypothetical or model results. With this option, the firm doesn't use real portfolios but a model portfolio to represent their strategy. This would be the least attractive option, but is permitted. Adequate disclosures are needed. (Steve Stone and I wrote an article on this topic for the CFA Institute, which you might find of value).
2. A representative portfolio. This is probably more attractive than a model, but less attractive overall, because it suggests "cherry picking," and you'd also have to deal with cases where an account may terminate. However, it's an option nonetheless. It would require appropriate disclosures as to what the returns represent.
3. A composite of all accounts in the strategy. This could be just like what GIPS requires (e.g., asset-weighted returns) or not completely (e.g., to use equal-weighted returns). This is preferable to the other two options shown (but obviously not as preferable as GIPS compliance), and will require your rules for composite construction to be identified, and preferably written down. Some form of disclosures would be needed, but perhaps not as many as the other options.

We conduct "non-GIPS" verifications for several firms, and so this option (to get your returns verified by an independent, third party) exists in this arena, too. We obviously encourage all managers to comply, but recognize this isn't always possible. This doesn't mean you can't have your numbers reviewed.

Oh, and when it comes to firms that fall under their country's regulator(s) (e.g., the Securities and Exchange Commission (SEC)): you will want to ensure that your materials conform with their rules, just as GIPS requires.

What happens when benchmarks change?

I received some interesting questions this week from someone regarding changes to a client's benchmark, which I thought would be worth sharing here.

I am gathering views from respected performance measurement professionals on an issue that has arisen with regard to benchmark performance. Within the scenarieo whereby at some point within the measurement period of a client’s portfolio there has been a change of benchmark (for example due to a change in investment strategy) would you agree with the following statements:

1. A client would still be interested in the performance of their portfolio over the full period of of measurement of their portfolio, spanning the two benchmarks.

This can absolutely be done. And, expected. Why would a change in the client's benchmark result in a cessation of full-period reporting?

2.  The client would expect the benchmark returns to be linked together accordingly in their client report.
 
This makes sense and appropriate. The investor surely wants to see the management vis-a-vis the benchmark, even if and when it changes.And we would expect the linking to reflect the multitude of  benchmarks which may have been used over the period.

3.  It is industry standard functionality within performance measurement systems for benchmark returns, spanning a change in benchmark, to be linked when this information is retrieved from the system?

Yes. This is a requirement in GIPS(R) (Global Investment Performance Standards).

Benchmarks often change. To avoid problems we would expect the historical benchmark(s) to remain, for the periods they aligned with.

Yes, we're now tweeting...

Not me, but we (our firm, The Spaulding Group, that is), are tweeting away. We have decided to create a Twitter account, meaning that we will be tweeting news and informational items we feel have interest to our friends and colleagues. Dave Mory, our sales associate and resident social media overseer, has set this up for us.

But, we have to be careful; we don't want to be captured by Fox News tweeting some dumb things, such as:

• Jessica Biel on the Days of the Week: “I work out every day -- Monday to Saturday.”
• Bow Wow on the Meaning of Life: “Im getting an xbox controller tatted on my arm. they say tats post to be meaningful, thats apart of my life video gaming.”
• Paris Hilton's Geography Lesson: “No, no, I didn't go to England; I went to London.”
• Solange Knowles Knows Her Body: “My thumb itches and my armpits smell like spongebob fever.”
• Lady Gaga on Love and Construction Materials: “Love is like a brick. You can build a house, or you can sink a dead body.”
• Sean 'P Diddy' Combs' Multiple Personalities: “I Am Destiny To Do Something GREAT Today!!!”
• Mary J. Blige Addresses Her Detractors: “Why is that people always try to understand estimate my intelligents?! They should never do that!”
• Ashton Kutcher on World Affairs: “Watching these dictated countries implode is just crushing.”
• Alicia Machado on Asia Tensions (Tweeting about the tense situation between North and South Korea): “Tonight I want to ask you to join me in a prayer for peace, that these attacks between the Chinas do not make our situation worse.”

 Please sign up so you can follow us! Thanks!

Is compliance enough?

A firm we consulted to a few years ago responded to one of our marketing pieces about verification, wondering if they should make the investment to undergo this exercise. They have claimed compliance with the Global Investment Performance Standards (GIPS(R)) for some time, but have noticed that they aren't getting the business that they thought they'd get, especially their above average returns.

To put it simply, YES! Verification is not an expense but an investment, that serves multiple purposes.

Claiming compliance is critically important today, especially when marketing in the institutional space. However, these institutions want some assurance that your claim has a degree of validity, and verification does just that. Granted, contrary to common belief, verification doesn't verify compliance. Rather, as GIPS 2010 points out, "Verification assesses whether (1) the firm has complied with all the composite construction requirements of the GIPS standards on a firm-wide basis and (2) the firm's policies and procedures are designed to calculate and present performance in compliance with the GIPS standards." But for many this is seen as being tantamount to affirming, though not in so many words, that the firm's compliance is correct.

And given that both our research and experience have shown that most firms who have not undergone a verification but claim compliance are falling short of the requirements (i.e., not really complying), the claim without the verification doesn't carry a lot of weight.

Another advantage for SEC registered firms: while the SEC offers "free verifications," you don't want to fail them (i.e., you don't want them to be the ones to tell you that you're not actually in compliance). Therefore, isn't it better to make the investment to avoid having these problems?

How frequently should you have a verification done? We recommend annually; we don't in any way support quarterly. Not because we don't like to visit our clients, but because there is no benefit for this level of frequency, other than to keep the verifier's employees busy throughout the year. We believe that quarterly visits are also disruptive to the client. Granted, some of the firms that do these verifications don't actually visit their clients, but we do, and none of ours has the need for us to see them that often (unless, of course, we're engaged in some other work for them).

Our research has also shown that most firms that claim compliance do have verifications done. And so, firms like the one cited above are in the minority. Yes, it's nice to be able to "tick off" the box in the RFP (Request for Proposal) that asks "Do you claim compliance with the GIPS standards?" But if you can't also tick off the "Have you had a GIPS verification done?," then the likelihood of being hired, or even considered, is significantly reduced.

And so to these firms I have to ask: you made the investment to achieve compliance; isn't it worth the investment to undergo a verification, too? We (and most of the industry) would say "yes."

p.s., a point of clarification is in order. The SEC doesn't actually offer "free verifications." They do freely come in once in a while to check registered investment advisors out, and in the course of their work may conduct tests to validate a firm's compliance, but aren't in the business of conducting verifications. Please don't call them and say "Dave Spaulding said you offer free GIPS verifications and we'd like you to come in and conduct one for us." I have used this line about "free verifications" in a number of talks I've given, thinking that surely everyone understood it was meant as a joke, but apparently some in attendance believed me.

p.p.s., while the suggestion that verification become mandatory has died away, the new requirement (as part of the 2010 edition) to declare whether or not the firm has undergone verification should encourage more firms who haven't made the investment yet to do so. That's at least the expectation.

Tracking your performance

This weekend's WSJ has once again provided us with a topic to address, specifically from Jason Zweig's column, titled "Why We Can't Tell if the Market is Half Empty or Half Full." He points out that "For most investors, getting a clear picture of where your portfolio stands isn't easy." The key words here are "your portfolio."

He later quotes Tom Chubb, a retired marketing executive who said "What I want to be able to do is to look at those different locations where my nest egg is stored and try to determine whether my financial advisers are doing a good job." Here we want to evaluate the financial advisers.

Jason identifies a few sites that might help the investor, such as Moringstar's, which has a "performance tab" where an investor can compmare their stock or ETF (exchange-traded fund) against the S&P 500, with dividends included. But would we really want to look at single investments? This has some value, no doubt, but the portfolio view is probably what most want to see. And if these investments are purchased and sold at different times, or in pieces, such a review won't really do us much good.

Apparently Google Finance is considering a "total-return chart," but they don't have any plans to introduce one. Yahoo apparently has one "in the pipeline."

What is it the investors are interested in? We've been through this a few times, yes? It depends on the perspective from which the question is posed:

• How am I doing? We want a money-weighted result.
• How is my portfolio manager or financial advisor doing? We're talking time-weighting.

Modified Dietz isn't a difficult return to calculate, and I'll be happy to provide anyone who'd like it a basic spreadsheet that can be used to derive these returns. They can then be chain-linked to develop an approximation to the true, time-weighted return, which will no doubt suffice for most investors who want to gauge the performance of those they rely on for their investments. For their own performance (i.e., money-weighting), they can rely upon Excel's built-in IRR functionality. We can provide guidance on how to do this, too. It's not really that hard. 

Oh, and for Mr. Chubb and others like him, we are finding more and more broker/dealers (several of whom are our clients) providing their clients with rates of return. Granted, they're often only from one perspective, but this solves at least one of their needs. And many can consolidate the holdings from other managers, so that the investor sees a single report that covers all their investments.

It really shouldn't be that hard, should it? We don't think so.

Operations & Compensation Survey

The Advent Users Group is hosting a compensation survey, which is open to all. We invite you to join in.

Our firm regularly does surveys ourselves, so we know how important it is to encourage individuals to participate. And we're happy to support the AUG in their effort to gather this information.

Examination Guidance ... it's time to be heard!

There is little doubt in my mind that if Bernie Madoff were to be given a copy of the proposed changes to the Guidance Statement on Performance Examinations he would pat himself on the back, and be quite proud that his nefarious actions have extended so far.

At the beginning of the document we find a few specific questions that the GIPS(R) (Global Investment Performance Standards) Executive Committee (EC) would like comments on, including:

Regarding performance examination procedures" we're asked if "you believe it also needs to be indicated that:

a. it is  preferable that verifiers obtain appropriate documentation directly from independent third parties;
b. verifiers must make every reasonable effort to obtain appropriate documentation directly from independent external parties; or
c. verifiers must obtain appropriate documentation directly from independent external parties?"

Also, regarding the existence and ownership of client assets we're asked if "it needs to be indicated that:

a. it is preferable that verifiers obtain appropriate documentation directly from independent external parties; or
b. verifiers must obtain appropriate documentation directly from independent external parties?"

Within the document itself we find on page 6: "Beginning- and end-of-performance measurement period portfolio positions are supported by sufficient documentation such as custody statements and custody reconciliations. The verifier must make every reasonable effort to obtain these documents directly from independent external parties (e.g., custodian, broker)."

The references to "independent external parties" trouble me, as it could possibly require the verifier to reach out to brokers and/or custodians to obtain documents, which in itself can be a challenge, and would likely add further complication and cost to the process. While I can understand why the EC would "float" such ideas, I would hope that they do not make it into the final document.

We're aware of legal cases involving firms that had committed fraud and that had both claimed compliance with the Standards and had undergone verification. Questions arose whether or not it's the job of the verifier to "detect fraud," and while it's fairly clear that it isn't, some of us feel that there are cases when verifiers should be alert to possible problems. These changes would, in essence, put the onus on verifiers to do just that: detect fraud. Is this really the verifier's role? I think not.

Our firm is on record as not supporting examinations as we don't believe they add to the compliance process. We do conduct them for a few of our clients, but most recognize that they're an expense that usually cannot be justified.

Please take the time to read the draft guidance and send your comments in (you have until August 31, 2011). You don't have to read the entire document; by simply referring to the first few pages (that ask specific questions) you'll understand the key items that the EC wishes to hear your comments on.

Keys to success for a software search

Although I'm technically on vacation this week, I came into the office to wrap up a report for a client I visited last month. I did a review of their entire performance operation, which resulted in a recommendation that they move forward with a software search. Since they wanted some help in getting pointed in the right direction, I included a schematic we developed some time ago, which identifies what see as the "keys to success" for such an undertaking:

As you can see, we believe there are several “keys to success” for software search projects.

• Performance measurement & GIPS expertise: it is no doubt obvious that an in-depth knowledge of performance, the GIPS standards, and attribution is paramount to having a successful search.
• Software development and design experience: We believe that a solid technical background is also paramount as it assures you that the analyst is intimately familiar with the other side of the equation.
• Exceptional analytical skills: software searches are very much an analytical activity; therefore, it’s critically important that the individual who heads up the effort be skilled at analysis, which includes the ability to know what questions to ask, detail the requirements, evaluate how the competing products meet these needs (micro level), and compare vendors along broader perspectives (macro level).
• Familiarity with the vendors: having an awareness of the “major players” is important to assure that the most appropriate vendors are considered. Ideally you will want to narrow the field down to the vendors you think are most likely to meet your needs.
• Recognized expertise: ideally the analyst should be recognized for their knowledge so as to establish credibility with the vendors being considered, though this isn’t a requirement.
• Proven success: it’s also ideal to have past success with these projects. A software search and selection is a huge investment in resources, both time and money, so success is critical, as you don’t want to repeat the process soon after a selection is made.
• Independence & objectivity: if the analyst is biased, it is likely they will favor one vendor over another, even though there may be a better solution available for the client.

We use the term "analyst" to represent the team leader, his or her designee, a group of individuals chosen for the task, and/or a consultant. I'll touch further on this topic shortly.

A brain teaser's solution

Recall that on Tuesday I posted what I described as a "brain teaser," involving the allocation of flight costs across three clients in two cities. The total cost is $710.10. Let's begin with some suggested solutions.

Allocate evenly: one-third to each

This is clearly the easiest solution to apply, but is it the most equitable? Let's say that the normal flight to and from Boston is $300, while it's usually $900 to Chicago. If we divvy up the $710.10 evenly, everyone's cost is below what it normally would be, but the Chicago client saves a much larger amount, from a proportionality perspective: is this fair? I don't think so.

Allocate based on the number of days

We're at Clients A and C one day each, and Client B for two days, thus Client B gets to pay half the fee, while the other two pay one-quarter each: rather simple to apply. However, what does it matter how long we're on the ground at each city? Let's say that I'm flying to Boston, and the normal cost is $300, and I'm there for four days; I then fly to London, and the normal cost there is $700, and I'm there just one day. And so, the Boston client pays four-fifths of the cost, even though a normal cost there is much lower; again, the other client benefits greatly while the other seems to be penalized.

Split the leg costs up

Here, the specific response was as follows:

• I would charge the Boston client the cost of the ticket from Newark to Boston
• I would charge the Chicago client the cost of the ticket from Chicago to Newark
• I would split the cost of the ticket from Boston to Chicago between the two clients

This requires finding out the one way costs from Newark to Boston, Boston to Chicago, and then Chicago to Newark. The charging of these fees seems fairly equitable, yes? And splitting the costs between the two legs also seems fine. I would say that this is an acceptable approach, though not what I do.

Allocate based on the normal relative round trip costs 

We find out what the normal round trip cost is to and from each city. We then add these together, and determine a ratio of what they would normally pay relative to one another. We apply this ratio to the actual costs. The following table shows the details:

I think this provides an equitable allocation of the costs, but agree that the third method, too, seems worthy of consideration.

Happy 4th

We wish all of our U.S.A. readers a Happy 4th of July!