She is retired now but for years she taught critical thinking in the classroom and at the dinner table to my brother, sister and me. Most importantly, she taught us to ask questions. I am thankful that she did. Good questions, it turns out, are a core component of good research. Good research is at the core of a successful investment strategy.
In the investment world, what often passes for a well-researched idea and sold to the public is often either created to support a marketing idea or is simply conjecture or opinion. Pundits begin with a point that they want to prove and then set to figuring out how to prove it. Instead, the ideal research process starts with a question about a phenomenon and is conducted with no agenda other than to explore the phenomenon, and then report the results of the exploration.
In turn, a rigorous data analysis should be free of weaknesses such as:
- Suspect data that is too short-term, too small of a sampling to be significant, or otherwise corrupted.
- “Survivorship Bias”, in which the returns of investments that didn’t survive the time frame in question (usually because they performed poorly) are omitted from the results.
- Comparing apples to oranges, such as using the wrong benchmark to evaluate results.
- Insufficient or inaccurate use of advanced statistics like multi-factor regression which helps identify the important factors from a noisy data stream.
Next, the results need to be repeatable by other researchers across multiple, comparable environments. And then last but hardly least, the researcher must publish their detailed results and methodology, ideally in an appropriate journal, so that similarly credentialed peers can review and critique their work and either agree that their results are sound or rebut them with counterpoints.
A good research process therefore starts with a few good questions rather than with the answer the researcher would like to prove. It then rigorously and openly tests the results and then reports them for peers to review. Make sure that your investment portfolio has these concepts at its foundation.
We cover these ideas and more in our video series and white paper on Evidence-Based Investing. If you have a question about Evidence-Based Investing, feel free to contact me. I’ll be glad to answer your questions.