Molecularbear's Blog

Last year I wrote Fantasy football non-lessons learned in 2008 so maybe this is becoming an annual tradition. My sorry draft this season has hopefully taught me a few lessons that I can learn from going forward:

Avoid one-hit wonders

RB Matt Forte exploded out of nowhere in 2008 and has comparatively disappeared in 2009. I was a little scared of a “sophomore slump”, but I bought into the hype that Forte could be even better with Jay Cutler at the helm. In the end, I chased after a one-hit wonder instead of using my 7th overall pick for a reliable stud (e.g., QB Drew Brees).

Don’t bank on wide receivers

Starting with the second round, I went Andre Johnson, Greg Jennings, and Jason Witten. Johnson has produced, but Jennings and Witten have both fallen well short of expectations. The WR position in general lacks consistency, so basing your team around a core of receivers isn’t necessarily the best idea. I think it might be better to go after the late-rounders or free agents who invariably wind up rising to the top (e.g., Miles Austin, Mike Sims-Walker, Sidney Rice, the Giants’ Steve Smith).

Two upside quarterbacks do not equal one stud

Last year I had an injury prone Matt Schaub and an inconsistent Jake Delhomme. This year, my QB corps of Carson Palmer and Matt Cassel (later traded for Matt Hasselbeck) is not much of an improvement. Since his bye week, Palmer’s FP production has been a pathetic {15, 7, 16, 11, 9}. Those performances from Palmer are a big reason why I’m out of playoff contention.

Don’t break strong RB monopolies

I had the RBs for the defending Superbowl champs in Willie Parker, Mewelde Moore, and Rashard Mendenhall. The second it looked like I had a three-headed RB monster on my hands, I started dropping these guys. Currently Mendenhall is playing for another team and has more fantasy points than Matt Forte or any of my other crummy backs. I need to think very carefully in the future before I give up an RB monopoly on a good team. Though with the way the Steelers have been playing lately, I’m not sure that we can call them “good”…

Next season I am almost certainly going to take a QB in the first round. After two straight seasons of flaky QBs and 1st round busts, I want some reliable production. Aaron Rodgers, say hello to your new team for 2010: the Shambling Corpses!

In my last post, I talked about grading the rankings of fantasy football experts using the root mean square error (RMSE), and listed some results for Week 9. This time, I took the data from ten analysts and examined their performance for weeks 1-9. The numbers seem to imply a certain amount of group think:

The results are pretty discouraging… all the analysts are basically the same! The biggest difference is 1.1 between the WR performance of ESPN’s Matthew Berry and Yahoo’s Scott Pianowski. So if Pianowski tells you that Randy Moss is the #1 WR for the week, then Moss is pretty much guaranteed to be in the top 21; if Berry tells you that Andre Johnson is the #1 WR for the week, then Johnson will fall somewhere in the top 22. Is that a big enough difference to say that Pianowski is a better analyst than Berry?

Amusingly, if you sum up the errors for each analyst, you’ll find that Matthew Berry, face of ESPN’s fantasy football team, is the worst. Still, they are all so close that being best or worst doesn’t have much meaning.

All this homogeneity had me wondering if there was a bug somewhere in my code. As a sanity check, I decided to see how different the rankings of the analysts were from one another. To do this, I took the average of the standard deviation of the rankings for each player. The result:

So you end up with an average standard deviation of about 4 for each player ranking. That is a pretty small number when you consider that each of these analysts is ranking 40 running backs and 50 wide receivers. If you include only rankings in the top 10, then the number plummets to 1.42704758.

Even more interesting is the very clear illustration of group think. The ESPN guys have an average deviation of about 3 amongst themselves, as do the Yahoo guys. But combining the two groups, the number increases by about 15%. It’s all very fascinating; the conclusion I draw is that these analysts are not significantly different from one another.

Notes about methodology

When calculating the RMSE table, I leveled the playing field by using the top N rankings from each position; see the table below:

The other thing to note about the scoring is that I capped the maximum error for situations where a player’s actual output put them outside the top N rankings. If they were outside of N, then I squashed their ranking to N+1. This was to avoid the situation where a player posts a goose egg and therefore winds up amongst the hundreds of other players with zeros.

Was it worth it?

I had fun doing this as a hobby project, but I was disappointed not to uncover a “super analyst”. All these guys are doing better than if you were to draw names out of a hat, but I wonder how much better they are than the average Joe Football. I am half-tempted to try ranking players myself and see if an amateur can compete with the experts.

UPDATE: added the ESPN guys.

Last year I considered writing some code that would grade the performance of fantasy football analysts. This year I’ve actually finished up the code and can post a few results. What I am doing is very simple:

1. Download an analyst’s weekly rankings.
2. Download the actual player rankings (based on fantasy point production).
3. Compare the two for each position using the root mean square error (RMSE).

The following table shows the RMSE of CBS Sports’ Jamey Eisenberg and Dave Richard for Week 9. Similar to the Netflix Prize, the lower the number the better. In this case, the RMSE tells you how many rankings off the mark the analyst tends to be.

EXAMPLE: Using the table below, we see that Dave Richard’s RMSE for quarterbacks is about 8. So if he says that a QB is going to be ranked #10, then you can be pretty confident that the QB’s actual rank will be between #2 and #18. That may seem like a large error, but keep in mind the random, uncertain nature of pro football: Tampa Bay beat Green Bay last week; Cedric Benson is the 7th top RB; Smith Smith of the Giants is the 7th top WR; the Titans are 2-6. Predicting this stuff is nearly impossible, but I have a hunch that some analysts will be better than others.

• ^C = CBS Sports
• ^E = ESPN

I will look to post data covering more analysts later this week!