What happens when you analyze your favorite college football team like the CIA?

# What happens when you analyze college football like the CIA? Source: [https://www.cultivatelabs.com/posts/what-happens-when-you-analyze-college-football-like-the-cia](https://www.cultivatelabs.com/posts/what-happens-when-you-analyze-college-football-like-the-cia) A couple weekends ago, the Illinois football team lost to Duke at home, 31–27\. My[Hinsley](https://www.hinsley.ai/p/d2f4100f/lifecycle)model immediately became much less optimistic about Illinois making the College Football Playoff\. But it became**more**optimistic about the offensive line and our new quarterback\. That sounds contradictory, but it's exactly what I wanted to happen\. For the past 15 years, we've worked with people whose job is to make judgments about uncertain futures: intelligence and government analysts, foreign\-policy researchers, investors, and corporate strategists\. This year I decided to try an experiment\. I took the methodology we've developed for that kind of work and applied it to something considerably less consequential: assessing the fortunes of the University of Illinois football team\. To understand why, it helps to think about what an intelligence analyst actually does\. ### How intelligence analysts think During the Cuban Missile Crisis, American intelligence analysts were trying to understand what the Soviet Union was doing in Cuba\. They had a growing collection of evidence, but the difficult part was deciding what it meant\. Analysts had to consider competing explanations, identify the observations that distinguished one from another, and revise their assessments as new evidence arrived\. Eventually, U\-2 photography provided much stronger evidence that the Soviets were installing nuclear missiles\. The stakes are obviously rather different, but the analytical problem is surprisingly general\. Usually there isn't one fact that gives you the answer\. There are several possible futures, a huge amount of imperfect information, and a smaller number of things that actually help distinguish among them\. The analyst's job is to impose structure on all of this without becoming more certain than the evidence warrants\. You find versions of this problem everywhere\. A foreign\-policy analyst might be trying to understand whether a conflict will escalate\. An investment analyst might be thinking about how geopolitics, regulation, or a new technology will affect an asset over the next decade\. A government analyst might be assessing how another country will respond to a policy change\. The useful question isn't simply, "What do I think will happen?" It's: What are the plausible ways this could turn out? What would have to be true for each of them? What should I be watching? And what new evidence would cause me to change my mind? It's not broadly known, but for more than a decade, Cultivate ran a prediction market for the U\.S\. Intelligence Community, giving analysts a way to make and aggregate probabilistic forecasts about geopolitical and national\-security events\. More recently, our work has expanded beyond forecasting individual questions into the broader analytical process around them\. That's what led us to develop[Continuous Probabilistic Foresight](https://www.hinsley.ai/guides/continuous-probabilistic-foresight), or CPF, the methodology at the heart of[Hinsley](https://hinsley.ai/), our AI/human hybrid analysis platform\. CPF starts with a strategic question and maps the range of plausible outcomes as scenarios\. It decomposes the problem into the drivers and indicators that would make those scenarios more or less likely, makes assumptions explicit, and turns important uncertainties into resolvable forecasting questions\. As new evidence arrives, those forecasts and the larger assessment can change with it\. The idea isn't to build a crystal ball\. It's to maintain a structured, explicit view of an uncertain future, and to know why your view changes when the evidence does\. Which brings me back to Illinois football\. ### Building an intelligence model for Illinois football Having grown up in Champaign, I'm a lifelong Illinois fan, and college football turns out to be almost comically well suited to this kind of analysis\. A season is an uncertain future surrounded by an enormous amount of information that is constantly evolving\. We have preseason recruiting, game results, injuries, competitor performance, statistics, coaching changes, on\-field performance, preseason models, beat reporting, podcasts, and endless amounts of informed and uninformed commentary\. We know some things with reasonable confidence, have strong opinions about others, and are almost certainly wrong about a few things we currently regard as obvious\. So instead of just following the season the way I normally would, I asked Hinsley to follow Illinois the way an analyst might follow a country, company, market, or strategic issue\. I started a couple months ago with the question I think most Illinois fans are ultimately trying to answer before a season:**What is the ceiling this season for the University of Illinois football team?** From there, I let Hinsley get to work\. Its research agent began collecting information about the team: returning players, transfers, recruiting, injuries, coaching changes, position\-group strengths and weaknesses, the schedule, preseason models, and so on\. Its findings were that this outside view was fairly optimistic\. Illinois had won 19 games over the previous two seasons, and Hinsley's research suggested a 10\-win regular season and a possible College Football Playoff berth represented a plausible ceiling\. But there were obvious reasons it might not happen\. Illinois was replacing Luke Altmyer at quarterback, returning only one starter on the offensive line, and replacing a lot of defensive experience under a new coordinator\. There was also a particularly interesting warning buried in the research: Illinois had won 13 one\-score games over the previous three seasons\. Maybe Bret Bielema's teams are unusually good at winning close games\. Or maybe some of that was luck that wouldn't continue forever\. That's exactly the kind of thing I wanted this exercise to expose\. Instead of saying, "Illinois has won nine games two years in a row, so they'll probably be good again," I now had a set of assumptions hiding underneath that belief\. The next step was to ask Hinsley to turn the big question into four scenarios for how the season could end, and generate initial likelihoods for each of those scenarios based on everything it knew at that point\.  But scenarios and even their associated probabilities by themselves aren't especially useful if you can't*explain*why one is becoming more likely and another less likely\. So I built what we call a decomposition: essentially a map of the things that could meaningfully affect which scenario we ended up in\.  Mine had nine broad categories\. They included the quarterback transition, offensive\-line continuity and health, whether the defense could reload, performance against the best teams on the schedule, execution in toss\-up games, how quickly transfers gelled, whether key players stayed healthy, special teams and possession margin, and the possibility of some unexpected roster or eligibility shock\. Underneath those were much more specific things Hinsley could actually watch: Houser's completion and interception rates, sacks allowed, third\-down defense, turnover margin, one\-score results, injuries, and so on\. This was the point where it started to feel less like having an opinion about Illinois and more like having a model of Illinois\. Not a statistical model in the traditional sense, but a structured description of what would have to go right for the team to have a great season, what could prevent that from happening, and what evidence would tell me which direction we were heading\. For a handful of the most important uncertainties, I went another step and turned them into forecast questions\. Will Illinois allow 30 or fewer sacks this season? Will it finish with a turnover margin of at least \+7? Will opponents convert fewer than 40% of their third downs? Will Katin Houser complete at least 64% of his passes while keeping his interception rate below 2\.5%? Will Illinois finish in the top 12 of the final College Football Playoff rankings? [Here's an example you can follow along with](https://www.hinsley.ai/forecasting/questions/b927fe7e-12b7-4d97-a7c6-cbbffb561ee2)and see the latest results\. In truly trying to assess the future performance of the team, those questions are much more useful to me than the endless debate on my message board subscription asking whether the offensive line is "good" or whether Houser is "playing well," because eventually there will be an answer\. Hinsley's AI forecasting ensemble puts probabilities on them, and I can make forecasts myself or invite friends to do the same\. Over time, I can compare what the AI thought, what a bunch of Illinois fans thought, and what actually happened\. [You can see the entire model here:](https://www.hinsley.ai/p/d2f4100f/lifecycle)  ### Then Illinois lost to Duke A home loss like that tends to produce a fairly predictable response from fans\. The team isn't as good as we thought\. The season outlook is worse\. It can be entertaining to vent and read others doing the same, but it's not very rational\. Hinsley reacted differently because the game contained several distinct pieces of evidence\. Illinois's chances of finishing in the top 12 of the final CFP rankings dropped, from 8% before the game to 4% afterwards\. That makes sense: if you're already an outsider trying to get into the playoff, losing at home to Duke uses up a lot of your margin for error\. But some of my forecasts moved in the opposite direction\. One of the biggest preseason concerns about the team was the offensive line, where Illinois was replacing almost everyone\. I had created a forecast asking whether the team would allow 30 or fewer sacks over the season\. After two games, including Duke, Illinois hadn't allowed a single sack\. So despite losing the game \-\- and despite some injuries on the line \-\- the forecast went from 52% to 61%\. The same thing happened with Houser\. I was tracking whether he could complete at least 64% of his passes while throwing interceptions on no more than 2\.5% of his attempts\. After Duke he was completing more than 71% of his passes with one interception in 52 attempts\. His forecast improved from 37% to 42%\. Meanwhile, the forecast that Illinois would finish with at least a \+7 turnover margin fell from 29% to 22%\. Illinois had lost the turnover battle and was now sitting at even for the season\. Put those four movements next to each other and you get a much richer context of what happened and where the season could still head than "Illinois lost to Duke at home, we're f\*^&ed": CFP Top 12: 8% → 4% ↓ 30 or fewer sacks: 52% → 61% ↑ Houser efficiency: 37% → 42% ↑ \+7 turnover margin: 29% → 22% ↓ This, more than anything, is what I like about the approach\. A loss is obviously important, but it doesn't follow that everything you believed about the team should move in the same direction\. The playoff outlook got substantially worse\. The evidence about pass protection got better\. The evidence about Houser got somewhat better\. The turnover outlook got worse\. The question isn't simply whether the latest piece of news is "good" or "bad\." It's which parts of your model that new evidence should actually change\. In a very low\-stakes way, that's the same analytical habit I described earlier\. Start with several possible futures and work backward to the things that would make one more likely than another to make the important uncertainties explicit\. Then when new evidence arrives, update the beliefs that the evidence actually bears on rather than allowing one dramatic event to overwhelm the entire analysis\. And because the structure is already there, I don't have to rebuild my view of Illinois every Sunday morning\. The research keeps running, the forecasts keep updating, and the scenario probabilities change as new evidence comes in\. Each week I can see not only what changed but why it changed\. It also gives me something I've never really had as a fan: a running record of what I believed about the team and why I believed it\. That's surprisingly useful because sports fans are very good at rewriting history\. After a player breaks out, it quickly starts to feel as though everyone knew he would be good\. After an upset, all the warning signs suddenly seem obvious\. Forecasting forces you to track what you actually thought before you knew the answer\. ### What else could you do with this? I'm not much of a sports bettor, but there is an obvious application there too\. If I were betting on games or trading on Kalshi or Polymarket, I'd be less interested in whether Hinsley thought Illinois would win than in places where its probability differed meaningfully from the market's\. A disagreement gives you something to investigate: what does my analysis believe that the market apparently doesn't? I'd record those disagreements before the games and then keep score\. Over enough predictions, I'd find out whether I had discovered an actual informational advantage or merely a more elaborate way of expressing my fandom\. There are more serious sports applications as well\. If I were working for a Big Ten football program, I might have an analysis like this running for every other team in the conference\. A research agent could continuously follow each program, maintain a structured assessment of its strengths and weaknesses, track important uncertainties, and flag meaningful changes\. Coaches and analysts would still make the judgments, but they wouldn't have to spend as much time finding and organizing the information in the first place\. The same seems useful for sports journalists\. If I covered the Big Ten, why wouldn't I have one of these running for every team? Instead of trying to keep a mental model of the whole conference, I'd have an explicit one for each team that was constantly being updated\. I could still disagree with it, but at least I'd have something systematic to disagree with\. More broadly, I think this experiment illustrates something interesting about where AI is taking analysis\. A lot of sophisticated analysis has historically required either specialized technical expertise or a great deal of manual work\. I don't know how to build a serious quantitative model of a college football team, and I don't particularly want to learn\. But I do know enough about Illinois football to ask useful questions, decide what matters, challenge assumptions, and judge whether an answer makes sense\. AI changes which parts of that process I have to do myself\. It can conduct much of the research, organize the evidence, help construct the analytical framework, make forecasts, monitor indicators, and continuously update the analysis\. My job shifts toward deciding whether we're asking the right question, whether the model of the problem makes sense, and where I disagree with its judgments\. You could do the same thing with almost any team or sport\. Start with the question you actually care about\. Define the plausible futures\. Work backward to what would have to be true for each one\. Identify the uncertainties that matter enough to forecast\. Then keep updating the whole thing as reality unfolds\. ### Moneyball for people who don't know how to Moneyball And now I'm curious to try it on other teams\. If there's a team you think would make an interesting test case, send it my way and I may build one and share what it finds\. Or, if you'd rather try it yourself, sign up for[Hinsley](https://hinsley.ai/)and[get in touch with me](https://www.cultivatelabs.com/cdn-cgi/l/email-protection#dbbabfbab69bb8aeb7afb2adbaafbeb7bab9a8f5b8b4b6)\. I'm happy to walk you through how I set mine up and help you build a model for your own team\. I'm still going to read the Illinois message boards, of course\. This just gives me a slightly more scientific way to decide when they're wrong\.