“Poor feedback about skill makes it virtually impossible for most fund managers to improve … This is largely behind the generally disappointing results from the overall active equity management industry.”
From the opening chapter of Skill Versus Luck
Relative return, tracking error, information ratio, attribution, upside/downside capture and batting average … these are just a few of the analytical tools still commonly used by asset allocators seeking to identify skilled investment managers. Such performance metrics have become required elements in investment company marketing literature. And as Michael Ervolini writes in his new book, “Mountains of research, more than a handful of Nobel prizes, and scores of analytic techniques lie before the altar of skill identification.”
Causes and Effects
There is just one rather significant problem. These metrics don’t really work as intended. That is, they don’t elucidate the presence or absence of investment skill. Conventional portfolio analytics help in assessing past sources and patterns of performance, but they fail to connect different types of skill with results. Or, as Mr. Ervolini notes, “Skill is not found in outcomes but in decisions. There is a cause-and-effect component to skill assessment absent from conventional analytics.”
Michael Ervolini is the founder of Cabot Investment Technology—which was sold to FactSet in 2021—and the author of several books published by MIT Press on the science of measuring fund manager skill. FactSet-Cabot has analyzed more than 600 funds and published numerous articles based on their research results. I had the good fortune to meet Mike while working with a client who retained Cabot to analyze and help improve her company’s investment decision process.
Skill-based analysis clearly links decisions to outcomes. For example, here is an analysis of a fund manager’s skill in buying, selling and sizing based on 14 years of fund history:
This chart is part of a case study in Skill Versus Luck. It is interesting to note that all three skills—buying, selling and sizing—are positive, meaning that they add to excess returns. This is relatively uncommon according to research referenced in the book.1
Key Questions About Skill
Skill Versus Luck shows how decision-based analytics can help funds2 and asset allocators answer a number of vitally important questions about the presence or absence of skill:
- Does the manager exhibit buy skill consistently and, if so, how consistent is buy skill by year, by sector and by financial attribute?
- Were younger winners sold too soon? Were older winners sold too late?
- How effectively are new buys brought up to a full weight? Does the fund commit too much capital before investments start to hit their stride or chase them after they become tired?
- Do adds and trims generate excess returns?3
- Is the fund manager perhaps “pulling out the flowers and watering the weeds” by selling winners while clinging to losers?4
Skill Versus Luck details a number of techniques for answering such questions, including hypothetical or counterfactual portfolios as a baseline to evaluate fund manager actions. “Specific counterfactuals,” notes Mr. Ervolini, “directly support fund diligence by assessing the management of substantial losers, the timeliness of building winners to full size, and the timing of selling winners and losers.” Traditional analytics use fund return series and holding histories as their data inputs. By contrast, decision-based analytics assess the impact of specific decisions that change fund composition and, thereby, returns.
A Few Bullet Points?
These new decision-based analytics can enhance the relevance and impact of investment marketing by supporting narratives with rigorously computed results. According to Skill Versus Luck, “the prevailing means of describing a buy process involves a few bullets points on a slide. This is usually accompanied by several anecdotes and a statement of guiding principles. This information may or may not reflect what happens when new stocks are purchased. Even when it is fairly accurate, which is more unlikely than not, there is no analytic verification to support what is being presented.”
Yep, that in my experience is about the size of it. Only potential investors often don’t even get the benefit of a few anecdotes to clarify the bullet points.
Why It Matters
The stakes are high for getting skill assessment right. An accurate understanding of the skills contributing to returns, for example, Mr. Ervolini writes, “can mitigate overresponding to short-term results … And having the internal fortitude to stay with an allocation that is experiencing a temporary headwind can pay off handsomely.”
Better analytics based on real results produced by real decisions can increase the distributions available from a sovereign wealth fund, enable a pension fund to better meet the needs of beneficiaries, expand the ability of endowments and foundations to achieve their missions and assist all investors in meeting their financial goals.
So yes, a description of the alpha-generating skills driving performance certainly merits more than “a few bullet points”!
A Conversation with the Author
Mike was kind enough to answer a few of my questions about the evolution and impact of his work:
Tell me a success story about improved alpha generation based on these new analytics.
An equity fund located in the UK was experiencing a good bit of volatility in their excess returns. They strongly believed that they were good at buying stocks but could not sort out why their results weren’t stronger. While collaborating with Cabot, they learned that they were in fact very good at buying stocks—their buy skill (contribution to excess return) was over 3% annually. Their selling, however, had a serious flaw not easily observed using conventional analytics: They were selling their winners far too quickly. This fund did a great job of sourcing stocks but did not allow strong performers to stay in the portfolio long enough to harvest their full alpha potential. This quick selling of winners cost the fund more than 2% annually. By implementing a modest process enhancement—scrutinizing potential sales at the Investment Committee level—the team began to hold winners longer and pick up portfolio results. Investment Committee decisions meant more eyes and ears and more opportunities to challenge every potential sale—as opposed to sell decisions driven solely by the analyst and portfolio manager.
Your book makes it clear that you see conventional analytics and the newer, decision-based analytics as a valuable complement to one another. Can you give an example?
Sure. Here is a classic example of how conventional performance metrics focus on past results—without full understanding of the skills driving those results. Let’s say a manager has upside capture of greater than 100% and downside capture of less than 100%. That’s valuable information. But the upside/downside capture numbers alone don’t yield insight into the underlying skills—buying, selling and sizing—driving those results. It is very common for one skill (e.g., buying) to drive upside capture while a different skill (e.g., selling or sizing) is responsible for downside capture. Knowing which skill is responsible and assessing its persistence adds immense insight into the overall desirability of a fund. The conventional metric (upside/downside capture) offers insight into past results while the new analytics provide the why behind those results. And the why provides a clear foundation for improvement.
How has the audience for your work perhaps evolved over time?
Initially, we were focused exclusively on helping equity teams make better decisions. This includes supporting research analysts, better integration of quant scores and other research into daily decision-making, and helping fund managers improve overall alpha generation. Since then we’ve expanded by using the same analytics to assist capital owners and allocators in better understanding fund manager skill. The more investors know about fund managers, the stronger their conviction in the allocations they make. And this is great for the entire industry.
What’s next? What do you think the science of investment skill assessment will look like 10 to 20 years from now?
The big unknown today is just how AI will be integrated into active management. A recent study by finance professor Lauren Cohen of Harvard (Mimicking Finance, 2026) used Large Language Models to mimic the decisions of equity managers based on 33 years of Morningstar fund data. The study found that the LLMs were good at selecting predictable, average performers but generally missed the unpredictable, strong alpha generators. This fits with my guess about the future which is that no matter the analytics available there will always be an alpha contribution from human judgment.
For more information about decision-based analytics, visit www.skillversusluck.com.
- Analyses of over $4 trillion equity assets involving hundreds of funds conducted by Cabot Investment Technology Inc. and FactSet Research Systems Inc. show that less than 15% of funds reflect positive values across buying, selling and sizing skills. ↩︎
- The terms “fund” and “manager,” explains Mr. Ervolini, are used interchangeably throughout the book given the number of funds increasingly managed by more than one person and “the continued evolution toward human/machine collaborations now managing money.” ↩︎
- Adds and trims have been found to be unproductive within the majority of approximately 2,000 professionally managed funds analyzed by Inalytics Ltd., Alpha Theory LLC and Cabot Investment Technology, Inc., a subsidiary of FactSet Research Systems Inc. ↩︎
- This metaphor about flowers and weeds originated in the Peter Lynch classic, One Up On Wall Street. ↩︎