Mary Rose, one of my best friends for decades, was a “psychic” who developed her own system of reading Tarot cards — using the entire deck for a reading. She lived in Las Vegas, charged $1 a minute with a 15 minute minimum, often gave 20+ readings a day, and had clients who came to her DAILY! She had volunteered for the police for a few years, finding missing persons and identifying perpetrators. Despite her almost perfect record, she quit because seeing crime scenes was too traumatic for her, especially when children were involved.
Her as-a-friend readings for me were 90+% accurate, and she was a valuable asset when I was leading organizations. Understanding the political climate and hidden agendas and knowing in advance how budgets would “come down” was priceless, I marveled at her gift of ”forecasting” – which I got as a bonus of our friendship.
So the book on the art and science of prediction by researcher Philip Tetlock interested me. It states, “forecasting is a skill that can be cultivated. And this book will show you how”.
REFLECT:
What do you believe about intuition? forecasting?
Do you believe the future is knowable?
After twenty years of researching forecasters / the accuracy of forecasts, Tetlock states most forecasts / predictions are no better than random guessing. The few that beat the odds of random guessing were for short- term predictions. The accuracy declined with each year out. Making the distinction between predicting nature’s patterns – like the sunrise and sunset – and the economy, Tetlock states that predictability depends on what we are trying to predict, how far into the future, and under what circumstances. Most often, predictions are vague hunches.
The IARPA (Intelligence Advanced Research Projects Activity), a US intelligence community agency, created a forecasting tournament in which five scientific teams would compete to generate accurate forecasts on tough questions like the likelihood of an attack in the Middle East and the future state of the economy in Europe. The teams were required to submit forecasts every day at 9AM EST from September 2011 to June 2015. Over this time, over 500 questions were addressed by the forecasting teams.
On of those teams was The Good Judgment Project (GJP), founded by Tetlock and his partner, Barbara Mellers. Their project invited volunteers to sign up and predict the future. Over 20,000 “ordinary citizens” volunteered. (I was asked by declined due to time requirements.) Tracking their performance, Tetlock was able to identify and separate out the “superforecasters”. He assembled them into a team which competed in the IARPA tournament. The first year, the GJP team beat the official control group by 60%. Year two it was 78%.
The Washington Post reported the GJP forecasters performed 30% better than the intelligence community analysts who were full time employees, had access to secret data, and could read intercepts. (Worldview could factor in here – see the blog on Worldview: 9/11). At that point, IARPA dropped the other teams and used the GJP team.
What was the GJP “superforecasters” secret? Many had advanced degrees In math and science and were news junkies. All were open-minded, careful, curious and self critical. They scored in the top 30% of the population for intelligence and knowledge, though far short of the 1% genius category. The dominant factor was a commitment to self improvement!
One superforecaster is Doug Lorch, a retired IBM computer programmer living the good life in Santa Barbara with his artist wife. Doug has no expertise in international affairs, but he volunteered for the GJP. For an hour each day, he opened his laptop, read the news, and tried to forecast. In one year, he made about 1,000 forecasts with about 68% accuracy. In the second year, Doug became the best forecaster of 2,800 GJP volunteers. Questions included: Will Serbia be granted official European Union candidacy by December 31, 201ll? Will the London Gold Market Fixing price of gold (USD ounces) excel $12,000 by September 30, 2011?
Bill Flack, a retired Department of Agriculture worker from Nebraska, scored equally impressively. He uses a Fermi approach. For instance, to answer the question, “How many piano tuners are there in Chicago?” four basic foundation questions must be answered:
1. How many pianos are in Chicago?
How many people are in Chicago?
What percentage of people own a piano? Guesstimate.
How many institutions (schools, etc.) own a piano? Guesstimate.
2. How often are pianos tuned?
Check data for this.
3. How long does it take to tune a piano?
Check data for this. Factor in @ 2 hrs travel time to/from.
4. How many hours a year does the average piano tuner work?
Guesstimate.
With this information, make an educated guess.
APPLY:
Try this approach for a problem you want to solve.
This is the approach used by the superforecasters and what I call “researcher mind”. It can be developed. In contrast, intuition – like my friend Mary’s – appears to be inherent. With his “math mind,” Tetlock presents intuition as a recognition of patterns based on cumulative knowledge and experience deep in memory banks (see the blog on Brain States / the Theta brain state). These memories give valid cues that unconsciously register for future use. Whether intuition generates delusion of insight depends on these valid memory cues.
I disagree. While that explains one form of insight, it does not explain Mary’s ability to forecast the future outcome of a government budget. She had zero memory bank for legislative budgeting processes, much less multi-million dollar budgets.
As an intuitive, I often have unexplainable flashes of insight. For instance, at an exposition I attended last month, I stopped at a booth to explore the product line. Learning I worked with the Theta brain state for personal transformation, one of the booth staffers regaled me with his personal journey for the past twenty years, and asked what he should do next. My instantaneous response was, “What happened when your were 3 years old?”
He responded, “My mother died.” And our conversation about his healing became one about childhood abandonment and trauma.
This hardly fits the model of cumulative memories providing insight or the lack of them creating delusion. And while it was not forecasting into the future, it did reach into the past to answer a question about future direction.
While I found the book informative and well written and the research extensive, it is limited to a mathematical approach, excluding decades of studies of “ways of knowing”.
Superforecasters can be identified by the accuracy of their predictions.They are not limited to the methods they use.
Superforecasting: The Art and Science of Prediction, Philip E. Tetlock & Dan Gardner, Crown Publishers, 2015.
