Atomic Football 2013?

I don't know what the future of Atomic Football will be.  Right now, my mom and my wife both have cancer.

AF 2012 Is Coming...

...Just a little later than usual this year.  Believe it or not, things more important than football are demanding my time.  My mom is battling a serious form of cancer.  Wife recovering from foot surgery.  Five children can keep you busy, too.  Hoping to get the first week's predictions posted this weekend.

A New(?) Football Strategy

Did anyone notice the third to last play in last night's Super Bowl? About 17 seconds left, if I recall, the Patriot's needing 56 yards for a TD, a long incomplete pass, and the Giants are flagged for twelve men on the field. Eight of the 17 seconds run off the clock during the play, five yard penalty.

Looks like a pretty good deal to me. Why not put twenty men on the field? Short of invoking the "palpably unfair act rule" (OK, try 13 men on the field), giving up five yards to run precious seconds off the clock seems to make sense.

Remember, intentional penalties are well within the rules (e.g., delay of game).

Actually, if you want my honest opinion, the NFL (and NCAA, too) may need to consider a quick rule change to close this loophole.

On the other hand, I wouldn't expect this strategy to make a huge impact. Doing it too soon will override turnovers (or turnovers on downs) and keep the drive moving. I don't know -- maybe we just leave things the way they are and just see what happens for while.

Georgia on My Mind

I just don't know what it is that keeps the vitriol flowing out of the great state of Georgia towards Atomic Football (see previous two posts). Honestly, my family loves to camp in the north Georgia mountains. My wife and I love to get away to Savannah. Hiking at Cloudland Canyon. Tubing the Cartecay. Unicoi State Park is one of my favorite places on earth.

Here's the latest I stumbled across (from http://www.gtsportstalk.com, grammar errors are those of the author):

"These Atomic Football prediction are a bunch of crap anyway. This Tech team is different, they have heart, they play with fire and a chip on their shoulders. That is the way they should play. Cant wait for Saturday!!"

At the time (9/20/11), Georgia Tech was 3-0, having won their games by an average margin of 37 points. With a schedule that grew increasingly difficult through the year, Atomic Football was predicting the Jackets would finish 7-5. Apparently, this didn't sit well with some of the faithful.

Tech proceeded to rattle off three more wins (by an average margin of 7 points) before the wheels fell off, going 2-4 in their final six games and finishing the regular season 8-4.

Granted, I underestimated their wins on the season by one, but having been outscored 241-243 in the final nine games of the season, Tech was fortunate to escape with more wins than losses during that period.

Were my predictions "crap?" I dunno. I don't claim to be psychic, but most of the time, I'm pretty close.

Best of luck to the Yellowjackets next year. They are always an exciting team to watch.

Oh, and we just got a new camper. Can't wait to visit the Peach State again.

No Respect

I'm not sure why I can't seem to get any respect from the great state of Georgia (see my previous post). The feeling is NOT mutual. Nevertheless, here are some excerpts from a thread on panthertalk.com discussing my 2011 predictions for Georgia State (my "for the record" updates are interleaved):

"They have us losing against West Alabama"

"I was noticing the same thing. I find it odd they say we have a better shot at beating USA than West Alabama. I know it's a home game vs a road game but really? Are those two even the same caliber of team?"

    For the record: GSU 27 - USA 20, GSU 23 - West Alabama 30

"Also has us losing to Murray State. Sigh....computers."

    For the record: GSU 24 - Murray State 48

"Predicted score against CAU is 37-12...garbage."

    For the record: GSU 41 - CAU 7 (not sure what's close enough for "not garbage")

To the credit of the Georgia State football team, they played an extremely tough schedule (9 of 11 opponents finished the regular season with winning records).

Atomic Football 2011

Yes, we're back for 2011, and we're excited about another great football season.

I recently did a web search to find some of the many posts from individuals who poke fun at our predictions. Here's a classic from last year:

Atomic Football (People Who Claim To Be Really Smart) Tell Us How Our Football Teams' Seasons Will Play Out.

I suppose what seems most odd about when people take their jabs at Atomic Football, whether on forums or in articles like these, is that no one ever seems to do a post-mortem on their own words. Since the author declined, I've decided to step in and help him out.

Here were my predictions that he so enjoyed ridiculing:

1) Georgia will finish the regular season 6-6 (3-5 in the conference) with a 28-27 win over Georgia Tech.

Actual Results: Georgia finishes the regular season 6-6 (3-5 in the conference) with a 42-34 win over Georgia Tech. Without the late TD with 1:29 left, the final score is 35-34.

2) Georgia Tech will finish the regular season 7-5 (5-3 in the conference). In the author's words, "Not suprisingly, the defense will not improve much. Middle Tennessee State will put up 19 on the Jackets and every other team [regular season after 28 September] will put up even more."

Actual Results: Georgia Tech finishes 6-6 (4-4 in the conference), one conference win off my predictions. MTSU puts up 14 points on the Jackets. For the rest of the regular season, the GT defense allows at least 20 points to their other seven opponents.

3) Georgia State will finish the regular season at 6-5. In the author's words, "In fact, this past weekend's victory over Campbell was just the beginning of the Panthers' own four-game winning streak (all over FCS opponents!)... GSU is predicted to finish its season with (only) a 57-0 loss to the Crimson Tide."

Actual Results: Georgia State finishes 6-5. The win over Campbell is the beginning of a four-game winning streak. The season ends with a 56 point loss to Alabama.

4) Georgia Southern will finish the regular season 8-3 (6-2 in the conference).

Actual Results: Georgia Southern underperforms slightly during the regular season, finishing 7-4 (5-3 in the conference). All four losses are by no more that eight points. The Eagles go on to win their first three playoff games, including a rematch win over Wofford whose 2-point edge over the Eagles during the regular season knocked my predictions off by one game.

Keep 'em coming.

What I'll Be Watching

College football starts tonight with about five Division II matchups. I enjoy following the FCS, D2, D3, and the NAIA during the playoffs, but I simply don't have the time to follow them much during the regular season. This is absolutely not a knock on these leagues -- they play very exciting football. I just have too many other commitments. If only I could get paid to do this hobby of mine.

FBS kickoff is still a week away. As usual, opening week is mostly a lot of huge mismatches. Fortunately, there are still several good matchups to keep things interesting. Here are some of the better ones I'm very much looking forward to:

Pittsburgh - Utah
Washington - BYU
Cincinnati - Fresno State
LSU - North Carolina
Oregon State - TCU
Boise State - Virginia Tech

Atomic Football 2010 Update

While I haven't posted in some time, it doesn't mean that I haven't been putting some work into Atomic Football. The "offseason" ends around Memorial Day; after that, there is plenty to do to get ready for the next season.

Much of what would be a huge ordeal is made easier by Peter Wolfe and his minions who comb the web for conference/division changes and upcoming schedules. Thanks, Peter.

There has been an assortment of minor additions and improvements to Atomic Football this year, along with several changes to improve my process for the weekly updates.

One exciting new addition is NFL predictions. While I do not yet post them to the website, they can be found on Prediction Tracker .

Football Predictions and Statistical Significance

Preface: I don't gamble on sports (or anything besides the stock market, for that matter).

As a student of football statistics and of statistical methods, I have always enjoyed studying the archived predictions at Todd Beck's Prediction Tracker. About a year ago or so, I put together a simple model to help evaluate football predictions. Essentially, the model represents any two sets of predictions as being composed of common errors (that is, “common” between the two models) and independent errors. Applying this model is particularly intriguing when one of those sets of “predictions” is the “betting line” (the "gold standard" of predictions). In doing so, we find that to “beat the line,” a set of predictions must be relatively close to the actual game outcomes but, at the same time, not too close to the betting line. In fact, if it were somehow possible to generate a set of predictions whose errors were completely uncorrelated with the errors in the betting line (i.e., no “common” errors), such predictions wouldn't even have to be very “points accurate” to make football wagering a winning proposition. In case you're wondering, doing this would be quite impossible to achieve.

Since I don’t gamble, you might wonder why I care. Well, I do have a competitive streak, and I enjoy the challenge of trying to generate good football predictions. However, I soon discovered that there are some on the web (I won’t name names) who lean heavily on the betting line to “improve” the points accuracy of their predictions. The net effect of this is that various metrics commonly used to evaluate the quality of a set of predictions (such as the metrics on Prediction Tracker like “mean square error” and “percent correct”) tend to look pretty good when one’s predictions are essentially a “fuzzed up” version of the betting line. The only metric that does not benefit from this tactic is, not surprisingly, one’s win percentage “against the spread.” However, since most predictions are no better than a coin toss against the line anyway, I realized that it is actually rather difficult to tell who is particularly good at doing predictions and who isn’t. So, I began to wonder if maybe, buried up in all the noise, there was some indicator that some predictions had, in fact, some added value over and above the betting line. The answer turned out to be “yes.”

Anyway, I began to run with the aforementioned model and soon found that I could generate an expected probability of win “against the spread” that was a better long-term indicator than the actual win percentage itself. If you want to know more on this, email me. Next, while the expected win probability was interesting, it failed to account for the number of games and didn’t produce a true “significance measure.” So, I translated it to the following metric that I call a “significance score:”

where L is the line, P are the predictions, S are the actual outcomes (as spreads -- home score minus away score), N is the number of games, angled braces indicate averages over the N games, and the function indicated by the Greek phi is the standard normal cumulative distribution function (CDF). The equation is an approximation (albeit a very good one) that assumes that the difference between P and L is relatively small compared to the other two differences.

If a set of predictions is essentially random noise (or any mix of the betting line and noise), the argument in the CDF above will tend to be a standard normally distributed random variable (mean zero, standard deviation one). As a result, P will be uniformly distributed between zero and one (or 0% and 100%). If, however, a set of predictions can manage to eliminate a source of error not corrected by the line, then the argument in the CDF will tend to be increasingly positive and P will tend to be higher than 50%, potentially even much higher.

OK, so what about the results? I pulled in the data from Todd Beck’s Prediction Tracker for the period from Week 5 of the 2007 Season through the end of the 2009 Season. I excluded the first four weeks of the 2007 season since I had just had Atomic Football added to Prediction Tracker and was still monkeying with the algorithm during that period. Since week 5 of 2007, my algorithm has changed relatively little. Note that the numbers that follow were done against the opening lines. Again, I wanted that for comparison since I publish Atomic Football’s predictions on Sunday (sometimes very early) and generally do not update them during the week. Most other participants on Prediction Tracker also publish early in the week and don’t update them thereafter. So, here you are…

  System                P
  Stat Fox              99.9948%
  Atomic Football       99.9945%
  Edward Kambour        99.971%
  Nutshell Sports       99.76%
  Stephen Kerns         99.71%
  Nutshell Sports Retro 98.6%
  Born Power Index      98.0%
  Pigskin Index         97.9%
  Moore Power Ratings   97.8%
  Sagarin Predictive    96.6%
  System Median         96.4%
  Keeper                96.3%
  System Average        95.7%
  Super List            95.2%
  Dokter Entropy        93.9%
  Dunkel Index          93.7%
  Lee Burdorf           93.6%
  Dave Congrove         93.1%
  CPA Rankings          93.1%
  Ashby AccuRatings     90.4%
  Covers.com            89.1%
  Least Squares         89.0%
  Bassett Model         87.6%
  Bihl System           86.3%
  Harmon Forcast        85.1%
  Massey BCS            80.8%
  Tom Benson            80.6%
  Laz Index             71.1%
  Howell                66.8%
  Massey Consensus      65.5%
  Beck Elo              63.8%
  Marsee                63.7%
  Hank Trexler          59.8%
  Logistic Regression   57.2%
  Least Squares w/ HFA  57.1%
  Sagarin               53.8%

The percentile column indicates the relative difficulty of achieving the performance strictly by chance. For example, a score of 99% indicates a level that could be exceeded by chance alone only one time out of one hundred. Oh, and if you’re curious, the top six systems all scored greater than 92% against the updated (“Saturday morning”) line, with the next closest being below 82%. By the way, the average score across all the systems (including those not listed above) against the updated line was 44.5%, indicating that the average computer prediction may even be worse than a coin toss come game day.

As I stated in my open, I don’t gamble. One reason is that a lot of other very interesting things come out of this model. I won’t go into details here except to say that even when one can “expect” (in the statistical sense) a positive net return, there is a serious problem with managing risk. In the end, managing the volatility means limiting returns to the point that eventually the stock market still looks better. So, I still prefer to bet on teams like Walmart or Apple.

Quarterback Productivity

Football fans are familiar with the "passer rating" (aka quarterback rating) that attempts to measure the efficiency of quarterbacks. Unfortunately, every component of the rating is normalized to pass attempts. Thus, while it does do well what it purports to do (measure per attempt "efficiency"), it is useless as a measure of what I'll call quarterback "productivity." If a quarterback rarely throws, he'll rarely face nickel and dime packages and consquently have an "efficiency" advantage. The real trick is a quarterback who throws a lot and still does it well.

For this reason, I was curious to find a simple modification to the quarterback rating that would measure just how heavily a team depends upon its quarterback for its offense and how well that quarterback delivers. Passing yards per game can be useful in this regard, but why not have a statistic that parallels the quarterback rating (i.e., considers TDs, INTs, etc.) but measures productivity.

My plan was to make it a simple variant of the quarterback rating as well as to try to keep it on roughly the same scale. Here is the result:

Quarterback Productivity (QP) =
Quarterback Rating (QR) * (Attempts Per Game / 30).

Basically, we've assumed a nice round number for typical attempts per game (30). If you revert to the original quarterback rating formula, you find that I'm basically converting all of its components (which are effectively per attempt) to a set of components that are instead "per game." Here are the recent leaders (no surprises):

2008 Graham Harrell (Texas Tech)
2007 Graham Harrell (Texas Tech)
2006 Colt Brennan (Hawaii)
2005 Colt Brennan (Hawaii)

A final note: While the average QP is on par with the average QR, the spread in the productivity is about double that of the rating, so numbers in excess of 200 are not uncommon. Thoughts? Suggestions?