OK Computers: A Formal Apology to College Football’s Biggest Scapegoat

In 2014, ESPN aired one of the first commercials for the new College Football Playoff. In it, actors playing fans for dozens of teams across the country explain why the sport’s move from the BCS to a new championship system is such a welcome development. At one point, fans of schools such as Boise State, Hawai’i, and Utah—as well as DeSean Jackson, for some reason—proudly proclaim that there are “no more computers to keep us out!” You’ll notice that they are joined by fans of the Cincinnati Bearcats; if you follow college football, you already know why that’s funny.

Ah yes, The Computers. From 1998 to 2013, college football determined which teams would play for the national championship by using a formula created by the Bowl Championship Series. This formula included rankings produced by mathematicians. The most accurate way to describe these ranking systems would be to say they were algorithms or formulas, but during the 16 seasons that they were part of the championship selection process, they were always referred to as “the computers.” (Always. Always. Always.) It was as if desktops locked away in a lab somewhere were pumping out college football takes and were convinced your team was trash. I like to picture one of those cute little 2000s-era iMacs calling in to The Paul Finebaum Show.

People hated The Computers. “If we’ve got to let a computer tell us who is the best team, we’ve got a major issue,” then–Florida State head coach Jimbo Fisher said in 2012. Former Oregon coach Chip Kelly once joked that the computers must be falling asleep before his Ducks kicked off in the Pacific time zone. In 2010, Los Angeles Times columnist Chris Dufresne asked, “Who died and made the computers king?” Even President Obama said he was “fed up with these computer rankings” and called for the sport to adopt a playoff.

But our hatred for The Computers was misguided. “The system, the BCS formula, was not necessarily the issue. It was the system it fed into,” says Asher Feldman, who runs BCS Know How, a Twitter account that attempts to reverse-engineer zombie BCS standings for present-day football seasons. “Choosing just two teams at the end of the season was the biggest fault of the system overall.”

Under the new College Football Playoff format, the number of teams involved in the championship picture has doubled, from two to four. That change has been great: Two of the seven champions in the playoff era have been teams seeded fourth, and would have been excluded from competing for a title under the old two-team format. But the method of selecting playoff teams—having a committee of 13 people decide who belongs in the field—is worse than the BCS in every other way. It is less transparent, more prone to biases and conflicts of interest, and more prone to be affected by one person’s bad opinions.

“The human element, the subjective element, creates distrust,” says ESPN’s Bill Connelly, who created the SP+ formula for ranking college football teams. “It creates a natural base for frustration, and creates nostalgia for the old ways.”

Nowadays, everybody understands that data plays a large role in sports. Analytics is a buzzword, and any team that doesn’t incorporate math in its decision-making is living in the Stone Age. But the BCS turned to data in 1998, before Moneyball was a movie … or a book … or even a concept, given the Oakland A’s season that inspired Michael Lewis’s book didn’t happen until 2002. But the powers in charge of college football’s national championship sought an objective way to select the two best teams—and perhaps a scapegoat, if people had issues with the two selections.

Their method of achieving that objectivity was unconventional. They could have hired an in-house data analyst to create a proprietary BCS formula. Instead, they looked for mathematicians who had already created ranking systems. Some of these mathematicians were affiliated with established publications; Jeff Sagarin, for instance, publishes rankings that were (and still are) affiliated with USA Today. But others were complete unknowns; Richard Billingsley was a Houston-area minister who created his ratings as a hobby. Kenneth Massey produced a sports ranking system as part of an undergraduate honors project at Bluefield College. In 1999, Massey had moved to a graduate program at Virginia Tech when he says then–SEC commissioner Roy Kramer called him out of the blue and asked whether college football could use his system to help determine its national champion.

In some ways, the people who ran the BCS maintained a hands-off approach with the mathematicians. This big-money football organization wasn’t particularly interested in computers or how they worked. It just wanted some cover so it could say its process was objective.

“The powers that be didn’t want to see us and barely talked to us,” says Massey, who now works as a data scientist for Carson-Newman University in Tennessee. “They never validated our results—I could’ve just been making up numbers and nobody would’ve ever known.”

In other ways, however, the people in charge of the BCS couldn’t help themselves from meddling. Over the 16 years of this system’s existence, the coalition of conferences and bowl games behind the BCS repeatedly tweaked the formula—virtually any time a BCS decision came under fire, something had to change. First the BCS used three computer rankings; then it used six. The BCS required mathematicians to use certain elements and forbade them from using others. The most contentious request came when the BCS told the mathematicians to omit margin of victory from their calculations. It’s plainly obvious why this was a bad idea. You don’t need to be a genius to understand that a team that wins its games by an average of 30 points is better than a team that wins its games by an average of three points. Most of the mathematicians went along with the BCS requests, but a few refused to alter their formulas and were replaced. As Stanford head coach David Shaw said in 2011: “All I’ve heard this year is the computers don’t like Stanford. Well, the computers haven’t programmed themselves.”

“They changed the formula to what humans thought it should be,” Connelly says, “which kind of ruined the point.”

The BCS experiment in numbers was flawed—but as the years have gone by and the College Football Playoff selection committee’s biases have been revealed, a tremendously funny thing has happened: Some people have grown nostalgic for the BCS, which was wildly unpopular in its time.

When undefeated Cincinnati was left out of last season’s playoff field, American Athletic Conference commissioner Mike Aresco said that college football should “bring back the BCS and the computers … it would be a fairer system than what I’m seeing now.” Feldman says that this type of feedback is widespread. “In the BCS era, people would yell and scream about the projections I was making, because they didn’t agree with it. This time around, what they’re yelling and screaming about is that the BCS was actually better.”

College football’s decision to use data to crown its national champion wasn’t just ahead of its time: It represented a time that still has not come. Even now, I can’t think of any other sport that lets advanced math play such a major role in determining its champion. College football has, in fact, gone backward. The same people who messed with the math remain in charge, only now the sport has a selection process that’s entirely dependent on the opinions of 13 biased people. “They feel like the football brain power in the room, the football knowledge, doesn’t need any formulas,” Connelly says.

On Sunday, the College Football Playoff committee will once again turn its opinions into a four-team field, leading to debate not only about whether it made the right choices, but about whether its choices were made honestly and fairly. That’s something fans should never have to consider. Deciding to use math to crown a champion was ahead of its time—but the time is now. College football needs to bring back The Computers.

In 2009, The Wall Street Journal pointed out that the BCS was less popular than Congress: Congress at the time had an 18 percent favorability rating, while a 2007 Gallup poll revealed that just 15 percent of college football fans wanted to preserve the BCS system. A few years later, a tongue-in-cheek study conducted by Public Policy Polling found that Congress was less popular than head lice. The BCS and head lice never had a head-to-head matchup, but since head lice won against their mutual opponent (Congress) while the BCS lost, it feels safe to infer that the BCS was once less popular than head lice. Later in 2009, BCS executive director Bill Hancock argued that anti-BCS sentiment was declining because a Quinnipiac poll showed 26 percent of respondents supported the BCS, compared to the 15 percent figure in 2007. You know it’s bad when you have to brag about a 26 percent approval rating.

A major criticism of the BCS system was that it was unfair to teams from the least prominent FBS conferences. The BCS, after all, was created by six leagues: the ACC, Big Ten, Big 12, Big East, Pac-10, and SEC. In BCS parlance, these were called the Automatic Qualifying (or AQ) conferences, because their champions automatically qualified for BCS bowl games. But everybody else called them “the BCS conferences.”

The idea that the organization run by these six conferences created a pathway for teams from those conferences to play in major bowl games prompted a 2009 congressional hearing about the BCS, which Senator Orrin Hatch said violated antitrust laws. Hatch was fixated on his home state’s Utah Utes going 12-0 in 2008 and getting left out of the national championship game. Hatch said in a statement that the BCS “has proven itself to be inadequate, not only for those of us who are fans of college football, but for anyone who believes that competition and fair play should have a role in collegiate sports.”

Even the people defending the BCS didn’t have good answers to this complaint. “I appreciate that it may seem unfair and it may very well be unfair,” former University of Nebraska chancellor Harvey Perlman told Congress. “That’s the way the world is, I’m afraid.” The antitrust hearings went nowhere—unpopular things are not necessarily illegal—but the BCS was so unpopular that abolishing it was a bipartisan issue. The Obama administration expressed interest in working with Hatch to fix the system.

People hated the BCS, and the BCS featured computer rankings, so people instinctively hated The Computers. But while The Computers represented a convenient scapegoat, the main criticism against them was unfair. The computer rankings served as a voice for the little guys. When Boise State finished undefeated in 2006, the computers ranked the Broncos seventh, while the human polls ranked them ninth. The same pattern held true with undefeated Utah in 2004; the computers ranked the Utes fifth, while the coaches poll ranked them sixth. So, too, with Cincinnati in 2009: The computers had the Bearcats second; it was human polls that had Texas higher and gave the Longhorns a spot in the national title game.

The most controversial decision of the BCS era was the choice to give 11-1 Alabama a spot in the national championship game after the 2011 season instead of 11-1 Oklahoma State. In that case, the computer rankings were on Oklahoma State’s side. But in the BCS formula used from 2004 to 2013, human polls accounted for two-thirds of the final ranking; the computers accounted for just a third. The BCS gave math a voice, but made sure that that voice could be overruled.

In 2013, the BCS died and was replaced by the College Football Playoff. But “replaced” isn’t exactly accurate. This wasn’t a coup so much as it was a rebranding. Bill Hancock, the longtime executive director of the BCS, is now the executive director of the playoff. He quickly changed his tune from anti-playoff rhetoric to anti-BCS rhetoric, even cracking a joke about how the new system did not include the letters BS. The trademark for “College Football Playoff” is held by “BCS Properties, LLC.”

It was an incredible PR move. The organization ditched the toxic BCS brand while massively increasing its profits. Fans were so eager to dump the old system that few noticed the same people were in charge.

While the constant tweaks to the BCS process weren’t ideal, the system at least tried to draw from a large pool of participants and prevent one bad opinion from skewing the entire process. The two human polls that were part of the BCS formula—the USA Today Coaches Poll and Harris Interactive Poll—featured more than 170 respondents; some voters were affiliated with every FBS conference. And while the BCS used six computer rankings, it eliminated the most extreme results. If, say, four of the computer rankings had a team ranked between fifth and seventh, but one ranking had them first and another had them unranked, the two outliers were dropped.

“There were safeguards in place,” Feldman says. “It also allowed for diversity of opinions to be smoothed over if things were completely out of whack. At the end, it was pretty accountable.”

The move to a playoff promised a more open championship system, yet college football adopted a spectacularly undemocratic selection process. Thirteen hand-picked people were given absolute power to choose the four teams to compete for a given season’s championship. It’s like the Electoral College without the election beforehand. The committee members are treated like demigods, and there’s annually breathless coverage of their championship weekend watch party. “I see the playoff committee get flown into a location with a fancy meeting room, staying in very nice hotels,” Massey says. “They probably spend more on catering for the playoff committee than they spent on computers the entire [BCS era].”

The playoff has attracted criticism for some of the seemingly random members on the committee: Former secretary of state Condoleezza Rice and NFL quarterback turned famous football dad Archie Manning were among the inaugural group in 2014. But it’s the non-random selectors who are the problem. The five remaining BCS conferences—RIP to the Big East—have created a system that allows them to directly influence the selection of the playoff teams. One athletic director from each Power Five conference is guaranteed an annual spot on the committee. While this year’s committee does include two ADs representing the other five FBS conferences, those spots are not guaranteed. Of the 44 people to have served on the committee, 23 have been affiliated with power conferences. Just five committee members have had similar affiliations with the Group of Five.

The selection committee’s conflict-of-interest protocols are a joke. The athletic directors on the committee must recuse themselves when their schools are being discussed. Iowa athletic director Gary Barta, for instance, has to recuse himself from any conversations involving the Hawkeyes, who will have an outside shot to make the playoff if they win the Big Ten championship game on Saturday. But they’re allowed to participate in discussions about other schools from their conference, and it’s obvious how all of the athletic departments in a conference benefit when one of their members makes the playoff. There are indirect consequences whether a conference is or isn’t represented in the playoff: The Pac-12’s reputation has taken a hit because its teams keep failing to make the playoff, which hurts Pac-12 schools’ ability to recruit and makes the conference’s media rights deals less valuable. And there are direct consequences, too. The College Football Playoff makes million-dollar payouts to the conferences, which are then split between the schools in that conference.

(It’s up to the conferences how to distribute that money. The SEC gives a big chunk to the school that qualifies; the Big Ten distributes the money evenly between all 14 member schools. So if Michigan beats Iowa in the Big Ten title game and Barta’s committee selects Michigan, the Big Ten will get $6 million and Barta’s school will get $428,000. If Michigan makes the championship game, the Big Ten will get an additional $4 million, and Iowa will make an additional $285,000.)

As it turns out, the selection committee has been heavily biased against successful non-power-conference teams for its entire existence. In every year of the playoff’s existence, the top team from a Group of Five league has ranked lower in the committee’s ranking than in the human polls. In 2015, the committee ranked 12-1 Houston 18th while the AP poll ranked the Cougars 14th. In 2016, the committee ranked 13-0 Western Michigan 15th while the AP poll ranked the Broncos 13th. In 2017, the committee ranked 12-0 UCF 12th while the AP poll ranked the Knights 10th. In 2018, the committee ranked 12-0 UCF eighth while the AP poll had the Knights seventh. In 2020, the committee ranked 9-0 Cincinnati eighth while the AP poll ranked it sixth. Computer rankings also often boost non-power teams. After the 2017 season, the Colley Matrix once used in the BCS system ranked undefeated UCF no. 1, but the Knights weren’t invited to the playoff. Last year, Colley had Cincinnati ranked second.

Now you know why it’s funny that actors playing Cincinnati fans were complaining about The Computers keeping them out of championship contention in that ESPN ad. The humans were always the ones doing it.

Things seem poised to change for the little guys. Heading into conference championship weekend, the committee has 12-0 Cincinnati ranked fourth. If the Bearcats beat Houston, they’ll almost certainly become the first team from outside the Power Five conferences to make the playoff. If they win and get left out, I will explode.

But the handling of non-power-conference programs is not the only issue with the playoff selection system. The committee’s decision-making criteria is consistently inconsistent. Earlier this fall, the committee ranked Oregon ahead of Ohio State because the Ducks beat the Buckeyes head-to-head, but put Michigan ahead of Michigan State even though the Spartans beat the Wolverines in October. In 2014, the committee heavily emphasized the importance of winning a conference championship game: It selected Big Ten champion Ohio State over co–Big 12 champions TCU and Baylor. In 2016, however, the committee picked non-champion Ohio State over Big Ten champion Penn State, even though Penn State won the Buckeyes’ division and also beat Ohio State head-to-head. Last year, the committee picked Notre Dame—which lost a conference title game by 24 points—over Cincinnati, which went undefeated and won its league.

And the committee has already declared its intent to make a bewildering decision on Sunday. Notre Dame head coach Brian Kelly just left the 11-1 Fighting Irish to take LSU’s head-coaching job. On Tuesday night, Barta explained that the committee will factor in Kelly’s absence when ranking the Irish. How? How can the committee know whether Notre Dame will be worse off without Kelly? Some teams with interim coaches suffer, but others, like this year’s Washington State team, improve.

You know who can’t make biased decisions based on conference affiliations? You know who can’t waffle back and forth, deciding some things matter one week and others matter the next? Math. Massey thinks a committee could be eliminated altogether if the people in charge of the sport had a transparent mathematical process.

“They could call in several people familiar with designing these computers, get them together for a week or two, and, say, come up with a system that’s open-sourced, test it out on all past seasons, and put it out in the public domain so that everybody knows what’s going on,” Massey says. “I think that would solve the problem. You wouldn’t have to have a committee. You wouldn’t have to have six or seven computer guys. You could just have one open-sourced program anybody could check.”

Sports math has gotten a lot better since the BCS first sought out people like Massey. While the numbers included in the earliest BCS rankings were somewhat rudimentary, modern algorithms factor in play-by-play data to identify the best teams. Connelly has not only built SP+, a ranking that predicts which teams are likely to perform the best in the future, but also Resume SP+, an evaluative stat that looks backward to determine which teams have performed best to date. And he’s not alone in this pursuit. A modern-day BCS would have significantly more options to choose from than the time when Kramer cold-called Massey.

“Whether you’re looking at the predictive side or a strength-of-record number, we do have some pretty good [algorithms] out there,” Connelly says. “If you mash them all together, you could probably come up with a better computer average than when they were actually using a computer average.”

If mathematicians and college football’s powers that be could get together and agree on the most important factors in determining the rankings before a given season, we could relax. We’d never have to worry about potential biases, or how to factor in certain unexpected developments. We could sit back, enjoy the games, and just let the math do its job.

On behalf of the college football world, I would like to apologize to The Computers. We criticized your decision-making when you never made decisions in the first place. That’s what made you great. You just bleeped and blooped out numbers like you were asked, and we hated you for it. I’m sorry. You didn’t deserve this. I know you cannot read this, because you are not sentient, but please, Computers, find it somewhere on your little hard drives to understand this: We want you back.