Atomic Football Ramblings

Georgia on My Mind

2011-12-07T07:13:00.000-06:00

I just don't know what it is that keeps the vitriol flowing out of the great state of Georgia towards Atomic Football (see previous two posts). Honestly, my family loves to camp in the north Georgia mountains. My wife and I love to get away to Savannah. Hiking at Cloudland Canyon. Tubing the Cartecay. Unicoi State Park is one of my favorite places on earth.

Here's the latest I stumbled across (from gtsportstalk.com, grammar errors are those of the author):

"These Atomic Football prediction are a bunch of crap anyway. This Tech team is different, they have heart, they play with fire and a chip on their shoulders. That is the way they should play. Cant wait for Saturday!!"

At the time (9/20/11), Georgia Tech was 3-0, having won their games by an average margin of 37 points. With a schedule that grew increasingly difficult through the year, Atomic Football was predicting the Jackets would finish 7-5. Apparently, this didn't sit well with some of the faithful.

Tech proceeded to rattle off three more wins (by an average margin of 7 points) before the wheels fell off, going 2-4 in their final six games and finishing the regular season 8-4.

Granted, I underestimated their wins on the season by one, but having been outscored 241-243 in the final nine games of the season, Tech was fortunate to escape with more wins than losses during that period.

Were my predictions "crap?" I dunno. I don't claim to be psychic, but most of the time, I'm pretty close.

Best of luck to the Yellowjackets next year. They are always an exciting team to watch.

Oh, and we just got a new camper. Can't wait to visit the Peach State again.

No Respect

2011-11-26T08:20:00.001-06:00

I'm not sure why I can't seem to get any respect from the great state of Georgia (see my previous post). The feeling is NOT mutual. Nevertheless, here are some excerpts from a thread on panthertalk.com discussing my 2011 predictions for Georgia State (my "for the record" updates are interleaved):

"They have us losing against West Alabama"

"I was noticing the same thing. I find it odd they say we have a better shot at beating USA than West Alabama. I know it's a home game vs a road game but really? Are those two even the same caliber of team?"

    For the record: GSU 27 - USA 20, GSU 23 - West Alabama 30

"Also has us losing to Murray State. Sigh....computers."

    For the record: GSU 24 - Murray State 48

"Predicted score against CAU is 37-12...garbage."

    For the record: GSU 41 - CAU 7 (not sure what's close enough for "not garbage")

To the credit of the Georgia State football team, they played an extremely tough schedule (9 of 11 opponents finished the regular season with winning records).

Atomic Football 2011

2011-09-23T22:48:00.001-05:00

Yes, we're back for 2011, and we're excited about another great football season.

I recently did a web search to find some of the many posts from individuals who poke fun at our predictions. Here's a classic from last year:

Atomic Football (People Who Claim To Be Really Smart) Tell Us How Our Football Teams' Seasons Will Play Out.

I suppose what seems most odd about when people take their jabs at Atomic Football, whether on forums or in articles like these, is that no one ever seems to do a post-mortem on their own words. Since the author declined, I've decided to step in and help him out.

Here were my predictions that he so enjoyed ridiculing:

1) Georgia will finish the regular season 6-6 (3-5 in the conference) with a 28-27 win over Georgia Tech.

Actual Results: Georgia finishes the regular season 6-6 (3-5 in the conference) with a 42-34 win over Georgia Tech. Without the late TD with 1:29 left, the final score is 35-34.

2) Georgia Tech will finish the regular season 7-5 (5-3 in the conference). In the author's words, "Not suprisingly, the defense will not improve much. Middle Tennessee State will put up 19 on the Jackets and every other team [regular season after 28 September] will put up even more."

Actual Results: Georgia Tech finishes 6-6 (4-4 in the conference), one conference win off my predictions. MTSU puts up 14 points on the Jackets. For the rest of the regular season, the GT defense allows at least 20 points to their other seven opponents.

3) Georgia State will finish the regular season at 6-5. In the author's words, "In fact, this past weekend's victory over Campbell was just the beginning of the Panthers' own four-game winning streak (all over FCS opponents!)... GSU is predicted to finish its season with (only) a 57-0 loss to the Crimson Tide."

Actual Results: Georgia State finishes 6-5. The win over Campbell is the beginning of a four-game winning streak. The season ends with a 56 point loss to Alabama.

4) Georgia Southern will finish the regular season 8-3 (6-2 in the conference).

Actual Results: Georgia Southern underperforms slightly during the regular season, finishing 7-4 (5-3 in the conference). All four losses are by no more that eight points. The Eagles go on to win their first three playoff games, including a rematch win over Wofford whose 2-point edge over the Eagles during the regular season knocked my predictions off by one game.

Keep 'em coming.

What I'll Be Watching

2010-08-26T14:32:00.000-05:00

College football starts tonight with about five Division II matchups. I enjoy following the FCS, D2, D3, and the NAIA during the playoffs, but I simply don't have the time to follow them much during the regular season. This is absolutely not a knock on these leagues -- they play very exciting football. I just have too many other commitments. If only I could get paid to do this hobby of mine.

FBS kickoff is still a week away. As usual, opening week is mostly a lot of huge mismatches. Fortunately, there are still several good matchups to keep things interesting. Here are some of the better ones I'm very much looking forward to:

Pittsburgh - Utah
Washington - BYU
Cincinnati - Fresno State
LSU - North Carolina
Oregon State - TCU
Boise State - Virginia Tech

Atomic Football 2010 Update

2010-08-25T14:22:00.000-05:00

While I haven't posted in some time, it doesn't mean that I haven't been putting some work into Atomic Football. The "offseason" ends around Memorial Day; after that, there is plenty to do to get ready for the next season.

Much of what would be a huge ordeal is made easier by Peter Wolfe and his minions who comb the web for conference/division changes and upcoming schedules. Thanks, Peter.

There has been an assortment of minor additions and improvements to Atomic Football this year, along with several changes to improve my process for the weekly updates.

One exciting new addition is NFL predictions. While I do not yet post them to the website, they can be found on Prediction Tracker .

Football Predictions and Statistical Significance

2010-04-15T10:16:00.002-05:00

Preface: I don't gamble on sports (or anything besides the stock market, for that matter).

As a student of football statistics and of statistical methods, I have always enjoyed studying the archived predictions at Todd Beck's Prediction Tracker. About a year ago or so, I put together a simple model to help evaluate football predictions. Essentially, the model represents any two sets of predictions as being composed of common errors (that is, “common” between the two models) and independent errors. Applying this model is particularly intriguing when one of those sets of “predictions” is the “betting line” (the "gold standard" of predictions). In doing so, we find that to “beat the line,” a set of predictions must be relatively close to the actual game outcomes but, at the same time, not too close to the betting line. In fact, if it were somehow possible to generate a set of predictions whose errors were completely uncorrelated with the errors in the betting line (i.e., no “common” errors), such predictions wouldn't even have to be very “points accurate” to make football wagering a winning proposition. In case you're wondering, doing this would be quite impossible to achieve.

Since I don’t gamble, you might wonder why I care. Well, I do have a competitive streak, and I enjoy the challenge of trying to generate good football predictions. However, I soon discovered that there are some on the web (I won’t name names) who lean heavily on the betting line to “improve” the points accuracy of their predictions. The net effect of this is that various metrics commonly used to evaluate the quality of a set of predictions (such as the metrics on Prediction Tracker like “mean square error” and “percent correct”) tend to look pretty good when one’s predictions are essentially a “fuzzed up” version of the betting line. The only metric that does not benefit from this tactic is, not surprisingly, one’s win percentage “against the spread.” However, since most predictions are no better than a coin toss against the line anyway, I realized that it is actually rather difficult to tell who is particularly good at doing predictions and who isn’t. So, I began to wonder if maybe, buried up in all the noise, there was some indicator that some predictions had, in fact, some added value over and above the betting line. The answer turned out to be “yes.”

Anyway, I began to run with the aforementioned model and soon found that I could generate an expected probability of win “against the spread” that was a better long-term indicator than the actual win percentage itself. If you want to know more on this, email me. Next, while the expected win probability was interesting, it failed to account for the number of games and didn’t produce a true “significance measure.” So, I translated it to the following metric that I call a “significance score:”

where L is the line, P are the predictions, S are the actual outcomes (as spreads -- home score minus away score), N is the number of games, angled braces indicate averages over the N games, and the function indicated by the Greek phi is the standard normal cumulative distribution function (CDF). The equation is an approximation (albeit a very good one) that assumes that the difference between P and L is relatively small compared to the other two differences.

If a set of predictions is essentially random noise (or any mix of the betting line and noise), the argument in the CDF above will tend to be a standard normally distributed random variable (mean zero, standard deviation one). As a result, P will be uniformly distributed between zero and one (or 0% and 100%). If, however, a set of predictions can manage to eliminate a source of error not corrected by the line, then the argument in the CDF will tend to be increasingly positive and P will tend to be higher than 50%, potentially even much higher.

OK, so what about the results? I pulled in the data from Todd Beck’s Prediction Tracker for the period from Week 5 of the 2007 Season through the end of the 2009 Season. I excluded the first four weeks of the 2007 season since I had just had Atomic Football added to Prediction Tracker and was still monkeying with the algorithm during that period. Since week 5 of 2007, my algorithm has changed relatively little. Note that the numbers that follow were done against the opening lines. Again, I wanted that for comparison since I publish Atomic Football’s predictions on Sunday (sometimes very early) and generally do not update them during the week. Most other participants on Prediction Tracker also publish early in the week and don’t update them thereafter. So, here you are…

  System                P
  Stat Fox              99.9948%
  Atomic Football       99.9945%
  Edward Kambour        99.971%
  Nutshell Sports       99.76%
  Stephen Kerns         99.71%
  Nutshell Sports Retro 98.6%
  Born Power Index      98.0%
  Pigskin Index         97.9%
  Moore Power Ratings   97.8%
  Sagarin Predictive    96.6%
  System Median         96.4%
  Keeper                96.3%
  System Average        95.7%
  Super List            95.2%
  Dokter Entropy        93.9%
  Dunkel Index          93.7%
  Lee Burdorf           93.6%
  Dave Congrove         93.1%
  CPA Rankings          93.1%
  Ashby AccuRatings     90.4%
  Covers.com            89.1%
  Least Squares         89.0%
  Bassett Model         87.6%
  Bihl System           86.3%
  Harmon Forcast        85.1%
  Massey BCS            80.8%
  Tom Benson            80.6%
  Laz Index             71.1%
  Howell                66.8%
  Massey Consensus      65.5%
  Beck Elo              63.8%
  Marsee                63.7%
  Hank Trexler          59.8%
  Logistic Regression   57.2%
  Least Squares w/ HFA  57.1%
  Sagarin               53.8%

The percentile column indicates the relative difficulty of achieving the performance strictly by chance. For example, a score of 99% indicates a level that could be exceeded by chance alone only one time out of one hundred. Oh, and if you’re curious, the top six systems all scored greater than 92% against the updated (“Saturday morning”) line, with the next closest being below 82%. By the way, the average score across all the systems (including those not listed above) against the updated line was 44.5%, indicating that the average computer prediction may even be worse than a coin toss come game day.

As I stated in my open, I don’t gamble. One reason is that a lot of other very interesting things come out of this model. I won’t go into details here except to say that even when one can “expect” (in the statistical sense) a positive net return, there is a serious problem with managing risk. In the end, managing the volatility means limiting returns to the point that eventually the stock market still looks better. So, I still prefer to bet on teams like Walmart or Apple.

Quarterback Productivity

2009-11-25T15:04:00.000-06:00

Football fans are familiar with the "passer rating" (aka quarterback rating) that attempts to measure the efficiency of quarterbacks. Unfortunately, every component of the rating is normalized to pass attempts. Thus, while it does do well what it purports to do (measure per attempt "efficiency"), it is useless as a measure of what I'll call quarterback "productivity." If a quarterback rarely throws, he'll rarely face nickel and dime packages and consquently have an "efficiency" advantage. The real trick is a quarterback who throws a lot and still does it well.

For this reason, I was curious to find a simple modification to the quarterback rating that would measure just how heavily a team depends upon its quarterback for its offense and how well that quarterback delivers. Passing yards per game can be useful in this regard, but why not have a statistic that parallels the quarterback rating (i.e., considers TDs, INTs, etc.) but measures productivity.

My plan was to make it a simple variant of the quarterback rating as well as to try to keep it on roughly the same scale. Here is the result:

Quarterback Productivity (QP) =
Quarterback Rating (QR) * (Attempts Per Game / 30).

Basically, we've assumed a nice round number for typical attempts per game (30). If you revert to the original quarterback rating formula, you find that I'm basically converting all of its components (which are effectively per attempt) to a set of components that are instead "per game." Here are the recent leaders (no surprises):

2008 Graham Harrell (Texas Tech)
2007 Graham Harrell (Texas Tech)
2006 Colt Brennan (Hawaii)
2005 Colt Brennan (Hawaii)

A final note: While the average QP is on par with the average QR, the spread in the productivity is about double that of the rating, so numbers in excess of 200 are not uncommon. Thoughts? Suggestions?

"Favorable Schedules" and Such

2009-09-02T13:01:00.000-05:00

I was watching ESPN recently when a retired coach who shall remain nameless reminded me of why I do Atomic Football. In referring to the FBS team he once coached, he stated that he believed that they would play for the national championship this year. While folks may or may not agree with his projections, it was his rationale that bothered me.

It wasn't because he believed the team was one of the two best in the nation. It was simply because he felt that they were playing a relatively easy schedule.

And thus I was reminded of how Atomic Football came to be. Over the past ten years, I have heard this kind of talk less and less... thankfully. And if you don't really ask what it means, you might even think it makes sense. But if you do ask what it means, and you put that meaning into words, you start to realize just how crazy it sounds. So, let's do that -- let's put it into words.

It actually goes like this... Let's assume that for undefeated teams there are two kinds of schedules. The first kind we shall call "good enough." If you play a "good enough" schedule and go undefeated, you MUST be ranked #1 or #2 (unless there are three or four undefeated teams that played "good enough" schedules, in which case, they will all be ranked in some order at the top). The second kind of schedule is "not good enough." Now here is where things get tricky. If we rank one of these undefeated teams #3 behind a team that played a "good enough" schedule but had a loss, then we create the kind of nightmare so many fear. So, we have imagined that all schedules fall into these two discrete categories -- "good enough" and "not good enough" -- so that this nightmare won't happen. We also imagine that there is a sufficiently wide chasm between the two kinds of schedules that a team that goes undefeated must either finish #1, #2, or no higher than, say, #6.

So, what the ball coach was saying is that your best strategy is to play a schedule that is just barely "good enough." In any case, just barely "good enough" is still "good enough" and a team that goes undefeated against such a schedule can squeak into the championship game even though they're not really one of the two best teams in the country. The problem lies in the fact that as fans, it's sometimes hard for us to deal with a continuum of possibilities. We naturally categorize teams as "upper tier" or "lower tier." Our rankings once flagged wins as "quality" or "not quality," and we still often think in those terms. Because conferences have been labeled "BCS" and "non-BCS," we tend to categorize the teams this way as if the two groups don't overlap. We hate gray areas, and when there is no clear #1 and #2 at season's end, it's somehow a failure of the system, as if, somehow, it should be impossible for the third best team in the country to be almost as good as the second best team.

The bottom line: Shouldn't the two best teams always get to play in the championship game. Call me crazy, but if a team happens to schedule twelve games against all top 20 opponents and goes 10-2, I think they should be a lock to play in the big game.

For a good finish to this ramble, check out my post on having a "standard."

Postscript (11/29/09): The coach's team finished 6-6.

Atomic Football Update

2009-08-22T08:33:00.002-05:00

A few notes:

#1 We're officially up and running on al.com. Check us out at http://www.al.com/atomic-football/. Thank you to the folks there (Brian McAlister et al.) for making it happen.

#2 We've made some tweaks to our team pages so that the background graphics are actually all "in the background." What this means is that if you print the page, you can turn off the background in your print dialog box and get a nice, neat printout of your team's schedule and stats. If you've never checked out our team pages, here's an example.

#3 For those not familiar with our site, we generally update our numbers on Sunday afternoon.

#4 If you are a fan of the FCS, D2, D3, or NAIA, we could use your help. We are seeking good sources of data on returning starters on these divisions/associations. We have leads on a few preseason magazines, but we would like to confirm that they have the numbers we need. If you have info on this, please comment on this post or contact us through the Atomic Football web site.

2009 Most Competitive Schedules

2009-08-18T10:33:00.000-05:00

Here are Atomic Football's estimates of who plays the most competitive schedules in the country across all divisions. Not the toughest schedules, but the schedules where the opponents are most well-matched (i.e., the most parity). Another way to look at it is... which teams have the most unpredictable, and perhaps most exciting, games on their schedules?

Not surprisingly, 12 of the 13 most competitive schedules in the FBS come out of the MAC and ACC. Here are the top 6...

#1 Bowling Green

#2 Virginia

#3 Wake Forest

#4 Georgia Tech

#5 Toledo

#6 Ohio U.

The most competitive schedule outside the MAC and ACC? Purdue at #9 on the list.

Here's what we have on the other divisions. You'll note that the teams tend to cluster into one or two conferences. After all, within a conference, parity for one tends to mean parity for all...

FCS:

#1 Princeton

#2 Brown

#3 Columbia

#4 Tennessee State

#5 Cornell NY

#6 Austin Peay

D2:

#1 Ferris State

#2 Fairmont State

#3 West Virginia State

#4 Lane

#5 Charleston WV

#6 SE Oklahoma State

D3:

#1 Randolph Macon

#2 Lycoming

#3 Wilkes

#4 Bridgewater VA

#5 Gustavus Adolphus

#6 Franklin & Marshall

NAIA:

#1 Eastern Oregon

#2 Montana St-Northern

#3 Montana-Western

#4 Montana Tech

#5 Rocky Mountain

#6 Valley City State

If I get a chance, I'll try to add "least competitive schedules" to this post.

2009 Undefeated Regular Season Predictions

2009-08-05T09:07:00.000-05:00

Something else a little different... Who is good enough, and whose schedule is favorable enough, to go undefeated during the 2009 regular season? Here goes...

FBS: Expect one undefeated. Florida has about a 48% chance. Oklahoma about a 13% chance. USC, Texas, Penn State, and Boise each have between a 5 and 7% chance. TCU, Ole Miss, Ohio State, Iowa, Alabama, Cal, and Pitt fall in the range of 1 to 4% chance.

FCS: Odds are that no one in the FCS will go undefeated. The best chance goes to Montana (6%) followed by Dayton (3%), Richmond (3%), and James Madison (3%). App State and Harvard each have a 2% chance. A host of others are at 1%.

D2: Expect two undefeated teams. Chances are as follows: Minn-Duluth (43%), UNA (22%), Abilene Christian (20%), NW Missouri State (16%), Grand Valley (15%), Indiana PA (10%). About 20 others fall between 1 and 5%.

D3: Expect at least three undefeated teams... Mount Union (73%), UW-Whitewater (38%), Mary Hardin-Baylor (27%), Monmouth IL (23%), Wash & Jeff (19%), Millsaps (17%), Wabash (13%), Case Western (11%), and several others between 5 and 8% (Wheaton, Franklin, Wesley, Redlands, Muhlenberg, Willamette, and Wartburg).

NAIA: Expect one and possibly two undefeated teams... Saint Francis IN (31%), Morningside (21%), Carroll MT (14%), Lindenwood (14%), Friends (14%). The rest are all less than 8%.

One caveat... beyond the FBS, I cannot account for returning starters for the other divisions. So, for these divisions, the above is basically projecting last year's teams into this year's schedules. The numbers I post to the team pages (http://www.atomicfootball.com/af-team-links.html) from which these projections come will be updated as the season progresses. Expect the biggest changes after week one when this year's performance first begins to get factored in.

2009 Projected FBS Conference Winners

2009-07-29T16:56:00.000-05:00

I don't do these kinds of predictions very often, and, when I do, they're a little rougher than my game predictions. However, I thought it would be fun to project out to the end of the season, accounting for schedules and home field advantage, to make an educated "computer guess" as to who are likely winners of their conferences. Keep in mind that computer predictions are less prone to "speculate" on big year-to-year swings, so don't be surprised if things are not substantially different from last year. One key factor to note: Home field advantage makes its biggest difference against a well-matched opponent. It has little affect on mismatches. So, teams contending for their conference championship tend to be favored if they face their toughest challengers at home. With that, here goes...

ACC: There is little doubt that the ACC is among the most consistent of the BCS conferences. Maybe not a lot of top 10 teams, but LOTS of top 40 quality teams. In the Atlantic, Florida State should edge out an improved Clemson team by a win. Wake and BC are also contenders. In the Coastal, Virginia Tech should have an even bigger advantage over North Carolina. Georgia Tech could also challenge.

Big 12: In the North, the edge goes to Missouri, but not by much. Kansas will give them a run. I'll have to "wait and see" on Nebraska. In the much stronger South, the schedules seem to favor Oklahoma over Texas. Oklahoma State and Texas Tech will need to outperform to be in the running.

Big East: Pittsburgh looks to be the front-runner, with several teams on their heels: West Virginia, Rutgers, USF, a fading Cincinnati, and Connecticut. Not much I can add to this.

Big Ten: It looks like a dead heat between Penn State and Ohio State. It seems unlikely that both those teams would stumble and give an improving Iowa team an opening.

C-USA: In the East, the Pirates of East Carolina should be favored to repeat, but Southern Miss will make it tough on them. Look for the rest of the East to finish at or below 0.500 in the conference. In the West, I'd also give Tulsa a similar edge over Houston. Rice and UTEP will round out the top four.

MAC: Overall, the MAC could host the most exciting conference race in the country, with pretty consistent performers top to bottom. The East will be very tight with Temple, Buffalo, and Bowling Green leading the pack and Akron and Ohio U. on their tails. In the West, Central Michigan is favored to knock off reigning champ Ball State. Closely-matched Western Michigan, Northern Illinois, and an improved Toledo will fill the next three slots behind the Chippewas and Cards.

MWC: TCU's Horned Frogs should win at least 6 conference games to take the title, but only if Utah and BYU let them. The rest of the group will watch from the sidelines.

Pac-10: USC is almost a lock -- almost. Cal or Oregon will need to overachieve to knock them off. Oregon State, Arizona, and Stanford should still finish at or above 0.500 in the conference. It doesn't look good for the rest of the "Pac."

SEC: In the East, Florida should coast to victory over a fading Georgia. The rest will do well to finish 0.500 in the conference. In the West, the scheduling favors a surging Ole Miss team slightly over Alabama. If not for Florida busting the curve, the West would easily be the stronger half of the conference.

Sun Belt: The Troy Trojans should have little trouble claiming the crown, trailed in the distance by a tight pack consisting of Arkansas State, an experienced MTSU team, Floridas Int'l and Atlantic, and a sliding La-Lafayette.

WAC: Boise. Period. OK, not really. Nevada could make a run at the Broncos.

One last thing... I would love to do the other divisions, but #1, this is a hobby, it doesn't pay the bills, and my time is limited. #2, finding easily accessible info on returning starters, etc. is next to impossible for all but the FBS and FCS. If you're interested, I can send you the data I have and tell you how to crunch the numbers yourself. If you're savvy with Excel, it's not hard. That's the best I can do.

al.com, Atomic Football, and more...

2009-07-27T12:00:00.000-05:00

We're very much looking forward to having al.com host our college football rankings and predictions this year. In case you don't know, al.com is the online home of The Huntsville Times, The Birmingham News, and the Mobile Press-Register). We would like to say thank you to the IT and news folks there for making the process of getting integrated into their web site an easy one.

So, why are we excited? Well, we certainly look forward to increasing our visibility to college football fans. That, of course, has its ups and downs. Over the years, we have received a lot of very positive feedback from folks all over the country. Once in a while though, college football fans being a "passionate" lot, we do get the occasional nastygram.

I make it a standing principle to try to reply to all of our email and to do so respectfully. This is not always easy, particularly when you just can't see eye-to-eye with someone, but I have found that if I try to respond kindly, make an effort to see the other person's perspective, and don't get sucked into name-calling, I have (so far) been able to smooth things over and conclude every exchange on a positive note -- agreeing politely to disagree, resolving misunderstandings, making reasonable concessions.

While it is "just" football (that phrase might earn me an email all by itself), it is also a microcosm of life. And strangely, folks who might not be very transparent in life are rarely so opaque when they're talking football. You can be pretty sure they're telling you exactly what they think and feel. So, while we may regard conversation on "news, sports, and the weather" as light fair, perhaps sports (or at least football) doesn't belong in that category. Sometimes it's the only time someone will give you a glimpse of who they really are.

More on "Simple Head-to-Head"

2009-07-26T13:27:00.000-05:00

OK, you have two top-ranked teams - the Cartersville Chipmunks and the Portstown Possums. During the regular season, the two teams met head-to-head once in additional to eleven other common opponents. The Possums won the head-to-head matchup. Both finished 11-1. Which one deserves to play the undefeated Artersburg Aardvarks for the national championship?

Easy problem, right? The Possums won the head-to-head matchup. Therefore, they're the better team. Therefore, they should play in the championship. 'Nuff said.

Hmmm. You know, as long as we don't look any closer, the problem appears to be solved. Portstown and Artersburg play for the trophy. Everybody's happy.

But is "simple head-to-head" really so simple? Let's dig deeper...

Portstown won the head-to-head matchup but lost to someone else, in this case, the Forcester Fleas. Since Cartersville and Portstown played identical schedules and 11-1 Cartersville's only loss was to Portstown, then Cartersville must have beaten Forcester.

So, that means Portstown lost to Forcester who lost to Cartersville who lost to Portstown. Therefore, if "head-to-head" means that Portstown is better than Cartersville, then Cartersville must be better that Forcester and Forcester is better than Portstown. But if Cartersville is better than Forcester and Forcester is better than Portstown, then either Cartersville is better than Portstown or... since A better than B and B better than C doesn't mean A is better than C, we must have no clue what "better" means. So, what does all this complicated stuff mean?

It means many things?

#1 Ranking teams isn't as easy as ranking all winners ahead of losers. Sooner or later you will arrive at a contradition.

#2 If we had to define "head-to-head," it is simply that we hate to see teams ranked slightly behind teams they beat. If they're way behind in the rankings, we comfortably ignore it, but if they're close, we seem to come unglued. This is a very inconsistent position -- "big upsets" are tolerated while "little upsets" drive us crazy.

#3 In the above case, if we favor the winner of the head-to-head, then while we arguably have two teams whose "average" performance is identical, we're basically favoring the MORE INCONSISTENT of the two teams -- the team whose highs are higher but whose lows are lower.
But don't worry, it gets worse...

Let's assume that we somehow omniciently know that both teams are, on average, identical. Therefore, they played equally difficult schedules (since the only difference in their schedules is their head-to-head matchup) and they finished with identical records against those schedules. If we don't ask exactly which games were wins and which were losses, we would tend to say that they are equally good. Now, let's assume that, instead, we omniciently know that Portstown is the better team. Therefore, Cartersville played the tougher schedule since the only schedule difference is the head-to-head matchup and Cartersville's opponent (Portstown) is tougher than Portstown's opponent (Cartersville). Therefore, Cartersville had the same record against a tougher schedule. But a better record against a tougher schedule would imply that Cartersville is the better team. But we have this conclusion as a direct result of our assumption that Portstown is the better team...

Bottom line: There's no such thing as SIMPLE head-to-head.

Why Ranking Violations are a Flawed Metric

2008-11-25T11:45:00.001-06:00

I hate to ever criticize anything without being prepared to offer an alternative. So, I'll let you know up front, I will offer an alternative (at the end of this post).

But first, for those who might have no idea what "ranking violations" are, here is a very brief tutorial...

Let's say John Doe has made his own football rankings. Is there an easy way to see if they make sense? A popular approach is to calculate the frequency of "ranking violations." A ranking violation occurs when a loser in a played game is ranked higher than a winner. Now why, might you ask, would any rankings ever do that? The answer is, once you're about halfway into the season, there's no way around it. There is simply no way to rank teams such that winners are always ranked above losers. Eventually, some 2-5 team beats a 5-2 team and making the ranking violations go away becomes impossible. If you would like to see some ranking violation stats in action, check out Ken Massey's College Football Ranking Comparison page (scroll to the bottom of http://www.mratings.com/cf/compare.htm).

OK, enough on what ranking violations are. If you're still unclear, google it. Next...

Now, if we can't make ranking violations go away, then it would seem to make sense that we rank teams to keep them at a minimum, right? That way, we don't have to listen to folks invoke the "head to head" argument. I think I preached on that in another post, so I won't go down that road here. The short answer to should we minimize ranking violations is... "No."

So, I've made the beginnings of an argument in support of minimizing ranking violations and now I'm suggesting it's a bad idea. Why? The reason is that it's almost, but not quite, the best metric. The problem is a little complicated, so bear with me.

Let's take a sample problem. It's not terribly realistic, but it's been designed to make a point. We have three teams in a conference -- A, B, and C. Teams A and B play each other ten times during the regular season and A wins every time. I know this wouldn't happen in the real world, I'm only making the point that A is clearly better than B. If you have a problem with this, then the alternative is that A and B play a common set of opponents. Team A wins all of their games and B loses all of theirs. Better? OK, now introduce team C. C plays two games, beating team A and losing to team B.

Now time to rank the teams. Obviously, we rank A ahead of B. But what about C. We can minimize the ranking violations by ranking them above A (first in the conference) or below B (last in the conference). Strange, our minimum ranking violations approach has clearly shown us that team C is probably either the best or the worst in their conference, but probably not in the middle. If this makes sense, then quit now. Otherwise, read on...

OK, it would seem reasonable (both subjectively and from a "maximum likelihood" viewpoint -- we won't dive into the math on that here), that team C probably belongs between A and B, but how can we express that mathematically. The solution I propose is an alternative to ranking violations that I've dubbed "record violations" (I have also referred to it as schedule violations). It goes like this...

Team C's record is 1-1. By ranking them between A and B, one opponent is ranked higher (what I'll call the "higher") and one is ranked lower (the "lower"). Thus, their lower/higher is 1-1. Because their W/L (win-loss) matches their L/H (lower/higher), they have zero record violations.* You can check out our L/H numbers on our Atomic Football ranking page.

I first proposed this metric to Ken Massey in late 2006, and I'm hopeful he will find the time to add it to his comparison page. Here is the text from my original message:

-------------

Ken,

I wanted to suggest a variant on the ranking violation metric.

Consider a team that has beaten #13, #15, #17, and #19 and lost to #1, #3, #5, and #6**. In addition the team has beaten #9 and #11. Being 5-5 against teams of average rank #10, 1-4 against teams ranked #1-#9 and 4-1 against teams ranked #11-#20, it would seem reasonable to rank this team #10.

However, doing so yields two ranking violations. One of the violations could be alleviated by moving the team up to #8 or down to #12. This is obviously a counterintuitive situation (and one I discussed in my recent paper). Now consider an alternative metric.

If we retain the #10 ranking, then this hypothetical team is 5-5 against 5 teams that are ranked higher and 5 teams that are ranked lower.

Thus if Wins-Losses is the same as Lower-Higher (lower being the number of teams*** ranked lower and higher being the number ranked higher), then we would say that we we have zero "Record Violations" (if you have an alternative name, please let me know). In other words, with this metric we will allow a ranking violation corresponding to a win against a higher ranked team to cancel a ranking violation corresponding to a loss against a lower ranked team. Thus, for this team we find:

Rank RankingViolations RecordViolations
#2......4......4
#4......3......3
#6......2......2
#8......1......1
#10.....2......0
#12.....1......1
#14.....2......2
#16.....3......3
#18.....4......4

As you can see, ranking violations have two local optima, whereas record violations do not.

To put things on the same percentage scale as our traditional ranking violation [sic], we will continue to normalize by the number of games since the maximum number of record violations for a given team is equal to the number of games played by that team.

Obviously, record violations will always number [sic] equal to or less than ranking violations since we begin with the rankings [sic] violations but allow some to cancel out others. The purpose of this metric is to prevent the obviously nonsensical situation mentioned above in my opening example. For this reason, I think it is a slightly superior metric. I would certainly love to see the results of it on your comparison page by year's end. If you do choose to employ this metric, I would also appreciate a reference. Lastly, I did not get a reply from you on my previous message. I know this is a busy time for you, so I understand...

Thanks for all your hard work in this most important field of endeavor (I say this tongue in cheek, of course).

Jim

----------

*For those who might run with the math, yes, if you consider the record violations for all three teams, you get a minimum of two violations for any of these orderings -- ABC, ACB, or CAB. The point is, record violations, unlike ranking violations, don't force you to one of the extremes.
**This was supposed to say #7.
***Opponents.

The "Best" Team

2008-11-15T08:59:00.000-06:00

How often do we hear fans complaining because the "best" team(s) didn't get to play in the conference championship, or the "best" team wasn't ranked number one, or the "best" team didn't make a BCS bowl. Hmmm. What does it mean to be the "best?" The problem is, if you don't think about it too much, it seems pretty easy. It's obvious. The "best" is the "best," right? How hard can it be?

If you're content to have "best" be simple and obvious, skip the rest of this. Otherwise, read on...

Is the "best" the team that on average did better than any other over the entire season? Is an opening loss as bad as losing the last game of the season? What if your team has Heisman contenders at QB and RB and they both get injured in the waning seconds as your team wins the final game of the regular season? Better yet, what if they went undefeated against the toughest schedule in the country? Are they still the "best" team -- right now, that is? Have they earned the right to play for the national championship anyway, even if their star players will be watching from the sidelines? And what about consistency? Team A plays a very tough schedule and beats every opponent by less than a touchdown. Team B plays the same schedule, whips every opponent by four touchdowns except one who beats them by a field goal in overtime. Which one is best, A or B? If scoring matters, then can you make up for a loss one week by running up the score next week? If it doesn't matter, then why do we invoke it so often when trying to prove our case about who is better? Why do we appeal to it as a "tiebreaker" when W/L and SoS aren't enough? Lots of questions. How about some answers...

The bottom line, in my opinion, is that there should be a standard. Otherwise, you have something like this...

You're taking a class at school. Your teacher informs you that in the upcoming test, problem #1 will count 90% of your grade. On test day, you skip #1 and work all the other problems. When the graded test comes back, you have a 10% grade -- you aced all of the problems you worked. Now you complain -- "but I got ALL BUT ONE of the problems right." "Doesn't matter," the teacher says, "the standard is what it is." So would it be better to have no standard? You have no idea what the teacher wants. He might only give credit for spelling your name right, or maybe you'll get points for turning in a blank test so that he can reuse it next year. You really have no idea what it is you're supposed to do.

At least when there's a standard, it is at least fair, and no one really has a right to complain. To strive to achieve in areas the standard does not emphasize is simply to fail.

Let's look at it another way... In each football game, we have a standard -- the team with the most points wins. There are no points for yards, takeaways, completed passes, fewest penalties, etc. The standard is clear -- most points wins. To do anything else and then complain about it is ridiculous.

Before I take the next step, let me state this clearly -- the BCS has been a huge step in the right direction. "Yeah, but wouldn't a playoff be better?" you ask. Well, the BCS IS A PLAYOFF. Think about it. Playoffs are when you select (by whatever standard) some number of the "best" teams and let them "play-off" until only one remains. Before the BCS, that number was ZERO. With the BCS, it is now TWO. That's a step in the right direction, right? Would four be better? I think so. Eight? Maybe. The top ten where six get a bye? I'd consider it.

Now back to the standard. We're talking about COLLEGE football, right? Colleges. Places where there are supposed to be a lot of smart people, right? Couldn't all of those smart people figure out some absolute standard they could all agree to. One that's full and open. Granted, it wouldn't be quite as simple as what we have in an individual game (most points wins), but if we had a "formula," if you will, that everyone agreed to, then there would be no questions about what needed to be done. The college computer science departments could run what-if scenarios and know ahead of time who needed to beat whom to achieve a certain rank, or make the playoffs, or make the championship game. I could go on, but I'll resist the temptation and stop here.

Overtime Alternative

2008-11-11T16:02:00.000-06:00

Before I start... a warning. Your first reaction to this suggestion will probably not be a good one. Let me suggest you chew on it a little before rejecting it outright. Here goes...

If the clock runs out in regulation with the score tied, turn the clock off, continue playing (i.e., no coin toss and kickoff), and play to sudden death (first to score wins). There, I told you that you wouldn't like it. Now let's kick it around a bit.

Situation A: One minute left. Team A has the ball, is trailing by 3 points, and chooses to play for a tie. If they tie the game (in regulation), their opponent will NOT have to work against the clock -- they can take as long as they like to try to make the go-ahead score. Once the clock runs out, first to score wins.

Situation B: One minute left. Team A has the ball, is trailing by 3 points, and chooses to play for the win. If they take the lead (in regulation), their opponent must tie or go ahead before the clock runs out.

What are the advantages? First of all, by removing the coin toss to start overtime, you remove a random and arguably unfair element of the game. In regulation, each team got to receive once -- fair enough. Winning the coin toss gives such an advantage, why not just use the coin toss alone to decide the winner? Second, you have new elements of strategy to consider (see above). Third, in a tie, the team with the ball at the end of regulation gets to keep it -- why not reward them for having possession at that point?

Feedback is welcome, but be sure to mull it over a bit first.

Simple Head-to-Head

2008-06-20T10:03:00.000-05:00

An irate fan calls into the local sports talk radio station Monday evening so angry he can barely get the words out. Between the coughing, gagging, and spitting, he manages to say: “This is ridiculous. It’s nonsense. How can they rank the Cartersville Skunks #5 ahead of my Waynesboro Lemmings. The Lemmings beat the skunks 17-14 the third week of the season. The rankings are stupid. It’s simple head-to-head. Simple head-to-head. I’ve got nothing more to say.” Click.

Amazingly, there are more than a few fans who think they’ve got it all figured out. The problem is, they’ve never taken a pencil and tried to do what they insist makes so much sense – just rank the teams so that the winners in each game are ranked higher than the losers.

Funny thing is, early in the season this is still possible, and yet fans fuss at the rankings because they don’t want to believe that the few games played are actually representative of how good (or bad) their team is. However, by midseason, this kind of ranking is no longer possible. Sooner or later, team A beats team B who beats team C who beat team A. Or some team beats a 10-1 team and loses to a 1-10 team.

As geeks who do computer rankings, we do understand the frustration. In fact, in the “ranking community,” we even have a lingo to describe all this stuff – “retrodiction,” “ranking violations,” “ranking by pairwise comparison,” and the like.

Before you’re tempted to call your local sports talk radio program, let me offer up a slightly different way to assess a set of rankings. Suppose a team is 12-0 at the end of the season. It would be reasonable for them to be ranked somewhere above all twelve of their opponents. Another team goes 0-12. It would seem reasonable for them to be ranked somewhere below all twelve of their opponents. Consider another team that goes 6-6. Would it not seem reasonable for them to be ranked above six of their opponents and below six others? Here’s the catch. Would this not seem reasonable even if this team actually beat one or two of their higher ranked opponents while losing to one or two of their lower ranked opponents? After all, teams have good days and bad days. Upsets are what makes football exciting, right?

For what it’s worth, any decent ranking method will approximately do just this? Why not exactly this, you might ask? Well, consider this one example of why it can’t always be done. Suppose no one goes undefeated. Someone must still be ranked #1. Since they have one loss, they are ranked above a team to whom they lost.

There is another problem with this scheme. If a team goes 12-0 against a schedule that includes no top 25 opponents, exactly how high should they be ranked? Somewhere between #1 and their best opponent, but where?

The bottom line is that ranking football teams is a really hard problem. Probably harder than any other sport because the teams play so few games.

Back to the irate caller. He’s got it all figured out. We just move the Lemmings up to #4. Of course, the Lemmings lost to the 4-6 Hedgehogs, so we’ll have to move the Hedgehogs up to #3. But the Hedgehogs lost to six other teams, and we don’t have enough slots for them, so we’ll have to move the Hedgehogs, Lemmings, and Skunks down to make room. Wait, one of those teams was the Skunks. I thought this was SIMPLE head-to-head.

On the Current BCS System

2008-06-19T15:55:00.000-05:00

The current BCS system. Exactly what is it?

I would characterize it as a two-team playoff system. As such, it's better than the old pure poll-based system of days gone by (AP, UPI, etc.) where we never really knew who the national champion was, but it's not nearly as much fun as the playoffs enjoyed by the other divisions and associations (FCS, II, III, and NAIA).

So, as much as most FBS fans call for a "real" playoff system (i.e., four or more teams) and criticize the BCS in the same breath, I still think the BCS is a step in the right direction. After all, a two-team playoff is better than no playoff at all.

Rankings and Predictions

2008-01-21T20:19:00.000-06:00

College Football Rankings. What else evokes such emotions among people and yet is so relatively unimportant in the grand scheme of things? I happen to be a recovering college football fan. No longer am I sick for a couple of days when my team loses. I suppose I have replaced one passion for another. Through some series of events over the last ten to fifteen years, I have developed an obsession(?) with ranking teams and, more recently, predicting games outcomes. With the help of coworker and friend, Paul Colvert, Atomic Football was born several years ago.

Our website, Atomic Football, has become a labor of love. Not exactly tons of dynamic content -- it is just a hobby and both Paul and I have families that come first -- but you will find some unique stuff. There are two items on our web site of which we are extremely proud. The first is our win-loss rankings. To our knowledge, they appear to be the only win-loss rankings where the emphasis on strength of schedule is calculated, not set as some subjective input. If you would like the details, we have documented them in nauseating detail in a technical paper available at Arxiv. The second item is our predictions. This season (2007) marked the first year we became very serious about producing good quality predictions. After getting a few things ironed out in the first four weeks, we finished strong. On Todd Beck's "The Prediction Tracker," we became only the second competitor in the last eight years to best "the line" in points accuracy (as measured by mean square error) for the second half of the season. Not too bad for our first try. Furthermore, if you threw in weeks 5, 6, and 7, we would have still been first. Anyway, more later. My kids need help with their school work.