Tweet
The Rookie Yardstick
In Part 1 of his NFL draft preview, Bill Barnwell has a new approach to measuring the success (and failure) of the picks
By Bill Barnwell on April 24, 2012
It's just about impossible to statistically measure the return that each draft pick provides after being selected in the NFL draft. The problem is that, unlike baseball and basketball, there's no easy way in football to value a player's contributions across different positions. Sure, you can probably guess that the running back who produces 4,000 yards in his first three seasons was a better draft pick than the one who produced 1,800 yards, but how do you figure out whether one draft pick on the offensive line is better than another? Or, even more distressingly, how do you compare the running back to the offensive lineman?
The reality is that you end up having to use the vaguest of vague statistics: appearances. Without reliable snap data available to the public, to measure a player's relative performance on the field, any sort of draft analysis has to employ games played and games started. That allows us to begin comparing the offensive lineman against the running back, but that process creates its own biases. Take the Steelers, a perennial contender who often leave talented young players on the bench during their rookie year as part of their successful developmental process. Pittsburgh star LaMarr Woodley has only started 57 games through his first five seasons, while former Jaguars castoff Justin Durant is at 54 starts over that time frame. Nobody in their right mind would choose Durant over Woodley, but a draft analysis that only incorporates games started would say that they had roughly similar production at linebacker. That sort of analysis isn't helping anybody.
We've done draft-performance studies like this in the past (as have others), and while we tried to control for things like draft position, merely using games started and games played created wonky results. Did you know, for example, that David Diehl is the greatest draft pick of all time? He's started 140 of 144 games in his nine-year career despite being drafted in the fifth round, which looks great if you forget that Diehl's often been a starter of last resort and a weak link on his own offensive line. Diehl was a good pick, but he shouldn't be the answer to that question.
What we needed to do, then, was produce some way to incorporate a player's performance beyond merely stepping onto the field and then use that to answer some important draft-related questions. Is there a position-general measure of performance that's universally accepted as a measure of success? Why, of course there is! In fact, there are two: Pro Bowl and All-Pro appearances. They're far from perfect (hi, Brandon Meriweather), but they're at least a step in the right direction in separating stars from mere starters.
With that in mind, we've put together a simple point system to measure a player's performance as a draft pick. We're going to use that today to evaluate draft picks throughout time, and tomorrow, we'll use the system to answer some questions about player scarcity and when teams should be splurging for certain types of players.
So here's how the system works: We wanted to measure how a player did during the general length of his rookie contract, so we're only going to consider how he performed during the first five years of his NFL tenure after being drafted. That means we can only start the player pool at 2007, since they were the last draft class to play five full seasons, so we'll go back to 1997 in order to have a full decade of draftees. We then gave the player credit for achieving various tasks as follows:
Each game started: 1 point
Each game played as a substitute: 0.5 points
Each Pro Bowl appearance: 10 points
Each All-Pro appearance: 20 points
The All-Pro award superseded the Pro Bowl award if a player made both teams, so the maximum number of points a player could earn for one season's work is 36 (a 20-point All-Pro appearance and a full 16-game slate as a starter).
Finally, we then compared each player's point total over his five-year stretch to every player from 1997-2007 who played the same position and was taken within 10 selections of the player in question. So Tom Brady, for example, started 62 games (with two additional substitute appearances) during his first five seasons and made it to two Pro Bowls, producing 83 points of value for his team.1 He was taken with the 199th overall pick, so our study compared him to every other quarterback from that 11-year stretch who was selected between 189th and 209th in the draft. Those quarterbacks produced an average of 6.4 points of value, so Brady's selection resulted in 76.6 points more than the average quarterback taken in that range of the draft. You get it, right? Imperfect, but it can serve as a reasonable starting point for these sorts of discussions.
As you might suspect, Brady rates out as the best selection at quarterback over that decade-long stretch, narrowly beating out Peyton Manning, who was at 71.9 points above his expectation as the first overall pick. Manning was better than Brady during their first five years, making three Pro Bowls, two All-Pro teams, and starting from Week 1 without missing a game, but Brady was selected in the sixth round, where virtually no quarterbacks turn into valuable properties. Our system says that he was relatively the better selection.
What about the rest of the best, though? And you'll probably want to know who the worst draft picks over that time frame were, too, right? We thought you might. So let's go position-by-position and point out who our little method suggests to be the best and worst draft picks from 1997 through 2007.
Quarterbacks
Best: Tom Brady (76.6 points above expected return)
Worst: Akili Smith (-45.9 points)
We've really forgotten as a culture just how bad Akili Smith was. He's basically the Ryan Tannehill nightmare; after just two years of action in college, Smith's ideal size and arm strength pushed him up draft boards during the spring of 1999. He eventually settled in as the third franchise quarterback in a three-man group, and the Bengals made him the third overall pick behind fellow signal-callers Tim Couch and Donovan McNabb. McNabb was good, and Couch was middling, but Smith was abysmal. He completed 46.6 percent of his passes as a pro, threw 13 interceptions against just five touchdowns, and was out of football after 17 starts in four years. In his one year as the regular starter in Cincinnati, the Bengals offense averaged 11.6 points per game. Tannehill can't possibly be that bad, right?
Running Backs
Best: LaDainian Tomlinson (88.9 points above expected return)
Worst: Curtis Enis (-55.1 points)
Both fifth overall picks, Tomlinson and Enis are the perfect examples of the risk-reward in taking a running back at the top of the draft, as somebody will do with Trent Richardson this Thursday. Enis's scouting report neglects to mention any form of injury concerns, but after averaging just 3.3 yards per carry as a pro, he was out of football with a knee injury by 24. Tomlinson matched Enis's 1,497 career rushing yards after about 18 games, and has tacked on more than 12,000 additional yards since. We'll talk more about running backs on Wednesday.
Wide Receivers
Best: Randy Moss (98.2 points above expected return)
Worst: Charles Rogers (-61.8 points)
Moss's average seasonal line over his first five years: 83 receptions, 1,349 receiving yards, 12 touchdowns. That's the Miles Austin breakout season from 2009 (81/1,320/11), five times in a row. The Tennessee Oilers chose Kevin Dyson three picks before Moss. Dyson's best season saw him catch 54 passes for 825 yards and seven scores. That was a subpar decision, but it was a better selection than Rogers, who was literally incapable of falling to the ground without fracturing his collarbone and ended up as the worst draft pick of the Matt Millen era in Detroit. NFL general managers who are willing to risk a high draft pick on Janoris Jenkins: Sometimes, your weed-friendly draft pick becomes Moss. Other times, your weed-friendly draft pick becomes Rogers.
Tight Ends
Best: Jason Witten (85.2 points above expected return)
Worst: Bennie Joppru (-54.0 points)
Witten is going to end up being some cause célèbre for football statheads when he retires, because he's going to have the résumé of a Hall of Famer and nobody's going to vote for him. Since he entered the league in 2003, only Tony Gonzalez has more receiving yards than Witten, who has averaged 89 catches and 1,014 yards over the past five years. He wasn't quite as good during his initial five-year run, but he was also just the 69th overall pick in that year's draft. The Texans chose Joppru 28 picks before Witten that April, but the Michigan product spent his first three seasons as a pro on injured reserve — yes, you read that right — and failed to catch a single pass at the NFL level before retiring in 2007.
Offensive Linemen
Best: Jahri Evans (86.0 points above expected return)
Worst: Bruce Nelson (-57.9 points)
The Saints took Evans in the fourth round of the 2006 draft as a left tackle out of tiny Bloomsburg University, but quickly moved him to right guard and found a superstar. Evans has started up front during each game of the Sean Payton era, and he's made the All-Pro team in each of the past three seasons. If he's not the best guard in football, he's awful close. You would be forgiven if you didn't remember Nelson, whom the Panthers chose with the 18th pick of the second round in 2003. Nelson started one game as a rookie before tearing the labrum in each of his hips, prompting his retirement after just 14 appearances.
Defensive Linemen
Best: Jared Allen (82.9 points above expected return)
Worst: Jamal Reynolds (-72.2 points)
Allen didn't quite come out of the Culinary Academy, but Idaho State was an obscure enough locale to push him to the 126th pick and the fourth round of the 2004 draft. He had nine sacks in 10 starts as a rookie and averaged 12 sacks per season during his first five seasons, four of which were spent on that rookie contract, which probably paid him just under $2 million. Reynolds was taken by the Packers with the 10th pick of the 2001 draft and never started an NFL game, producing just three sacks in 18 appearances. Reynolds had microfracture surgery on his knee after his rookie season, but because we didn't know what microfracture surgery was back then, it was characterized as having "flakes of cartilage" removed before doctors "poked some holes in his knee to improve circulation." We were so young!
Linebackers
Best: Patrick Willis (93.6 points above expected return)
Worst: Andy Katzenmoyer (-50.2 points)
Is Willis the most productive draft pick ever? He just finished up his fifth season in the league, and during those five years, he's missed a total of four games while making the Pro Bowl and All-Pro teams in each season.2 No other player over this time frame was able to match that accomplishment, and the last player we were able to find who matched that mark was Lawrence Taylor, who made the Pro Bowl team for the first 10 seasons of his career and got onto the All-Pro team for eight of those years. Sure, middle linebacker might not be the most important position on the field, but what more could the Niners ask for from their defensive captain? Katzenmoyer, on the other hand, was a legend at Ohio State, but his NFL career was brief; after suffering a neck injury during his rookie season, Katzenmoyer struggled to stay healthy and left Patriots training camp after breakfast one morning in 2001, never to return. He only played 24 games at the professional level, leaving the game just months before the Patriots won their first Super Bowl.
Defensive Backs
Best: Sam Madison (90 points above expected return)
Worst: Charles Fisher (-59.1 points)
Jimmy Johnson made a lot of terrible draft picks for the Dolphins during his tenure in Miami, but he picked up two pretty impressive players back-to-back in 1997. There, he grabbed Madison in the second round and followed that up with Jason Taylor in the third.3 Taylor probably had the better career, but Madison developed quicker; he was a starter by year two and an All-Pro cornerback by his third season, during which he led the league with seven interceptions. He eventually moved on to New York and picked up a Super Bowl ring with the Giants in 2007. Fisher, taken with the first pick of the second round in 1999, tore up his knee during his first professional game and was never able to make it back onto the field. The Bengals are prone to making some awful selections, but that's terrible luck, not poor planning.
In Part 1 of his NFL draft preview, Bill Barnwell has a new approach to measuring the success (and failure) of the picks
By Bill Barnwell on April 24, 2012
It's just about impossible to statistically measure the return that each draft pick provides after being selected in the NFL draft. The problem is that, unlike baseball and basketball, there's no easy way in football to value a player's contributions across different positions. Sure, you can probably guess that the running back who produces 4,000 yards in his first three seasons was a better draft pick than the one who produced 1,800 yards, but how do you figure out whether one draft pick on the offensive line is better than another? Or, even more distressingly, how do you compare the running back to the offensive lineman?
The reality is that you end up having to use the vaguest of vague statistics: appearances. Without reliable snap data available to the public, to measure a player's relative performance on the field, any sort of draft analysis has to employ games played and games started. That allows us to begin comparing the offensive lineman against the running back, but that process creates its own biases. Take the Steelers, a perennial contender who often leave talented young players on the bench during their rookie year as part of their successful developmental process. Pittsburgh star LaMarr Woodley has only started 57 games through his first five seasons, while former Jaguars castoff Justin Durant is at 54 starts over that time frame. Nobody in their right mind would choose Durant over Woodley, but a draft analysis that only incorporates games started would say that they had roughly similar production at linebacker. That sort of analysis isn't helping anybody.
We've done draft-performance studies like this in the past (as have others), and while we tried to control for things like draft position, merely using games started and games played created wonky results. Did you know, for example, that David Diehl is the greatest draft pick of all time? He's started 140 of 144 games in his nine-year career despite being drafted in the fifth round, which looks great if you forget that Diehl's often been a starter of last resort and a weak link on his own offensive line. Diehl was a good pick, but he shouldn't be the answer to that question.
What we needed to do, then, was produce some way to incorporate a player's performance beyond merely stepping onto the field and then use that to answer some important draft-related questions. Is there a position-general measure of performance that's universally accepted as a measure of success? Why, of course there is! In fact, there are two: Pro Bowl and All-Pro appearances. They're far from perfect (hi, Brandon Meriweather), but they're at least a step in the right direction in separating stars from mere starters.
With that in mind, we've put together a simple point system to measure a player's performance as a draft pick. We're going to use that today to evaluate draft picks throughout time, and tomorrow, we'll use the system to answer some questions about player scarcity and when teams should be splurging for certain types of players.
So here's how the system works: We wanted to measure how a player did during the general length of his rookie contract, so we're only going to consider how he performed during the first five years of his NFL tenure after being drafted. That means we can only start the player pool at 2007, since they were the last draft class to play five full seasons, so we'll go back to 1997 in order to have a full decade of draftees. We then gave the player credit for achieving various tasks as follows:
Each game started: 1 point
Each game played as a substitute: 0.5 points
Each Pro Bowl appearance: 10 points
Each All-Pro appearance: 20 points
The All-Pro award superseded the Pro Bowl award if a player made both teams, so the maximum number of points a player could earn for one season's work is 36 (a 20-point All-Pro appearance and a full 16-game slate as a starter).
Finally, we then compared each player's point total over his five-year stretch to every player from 1997-2007 who played the same position and was taken within 10 selections of the player in question. So Tom Brady, for example, started 62 games (with two additional substitute appearances) during his first five seasons and made it to two Pro Bowls, producing 83 points of value for his team.1 He was taken with the 199th overall pick, so our study compared him to every other quarterback from that 11-year stretch who was selected between 189th and 209th in the draft. Those quarterbacks produced an average of 6.4 points of value, so Brady's selection resulted in 76.6 points more than the average quarterback taken in that range of the draft. You get it, right? Imperfect, but it can serve as a reasonable starting point for these sorts of discussions.
As you might suspect, Brady rates out as the best selection at quarterback over that decade-long stretch, narrowly beating out Peyton Manning, who was at 71.9 points above his expectation as the first overall pick. Manning was better than Brady during their first five years, making three Pro Bowls, two All-Pro teams, and starting from Week 1 without missing a game, but Brady was selected in the sixth round, where virtually no quarterbacks turn into valuable properties. Our system says that he was relatively the better selection.
What about the rest of the best, though? And you'll probably want to know who the worst draft picks over that time frame were, too, right? We thought you might. So let's go position-by-position and point out who our little method suggests to be the best and worst draft picks from 1997 through 2007.
Quarterbacks
Best: Tom Brady (76.6 points above expected return)
Worst: Akili Smith (-45.9 points)
We've really forgotten as a culture just how bad Akili Smith was. He's basically the Ryan Tannehill nightmare; after just two years of action in college, Smith's ideal size and arm strength pushed him up draft boards during the spring of 1999. He eventually settled in as the third franchise quarterback in a three-man group, and the Bengals made him the third overall pick behind fellow signal-callers Tim Couch and Donovan McNabb. McNabb was good, and Couch was middling, but Smith was abysmal. He completed 46.6 percent of his passes as a pro, threw 13 interceptions against just five touchdowns, and was out of football after 17 starts in four years. In his one year as the regular starter in Cincinnati, the Bengals offense averaged 11.6 points per game. Tannehill can't possibly be that bad, right?
Running Backs
Best: LaDainian Tomlinson (88.9 points above expected return)
Worst: Curtis Enis (-55.1 points)
Both fifth overall picks, Tomlinson and Enis are the perfect examples of the risk-reward in taking a running back at the top of the draft, as somebody will do with Trent Richardson this Thursday. Enis's scouting report neglects to mention any form of injury concerns, but after averaging just 3.3 yards per carry as a pro, he was out of football with a knee injury by 24. Tomlinson matched Enis's 1,497 career rushing yards after about 18 games, and has tacked on more than 12,000 additional yards since. We'll talk more about running backs on Wednesday.
Wide Receivers
Best: Randy Moss (98.2 points above expected return)
Worst: Charles Rogers (-61.8 points)
Moss's average seasonal line over his first five years: 83 receptions, 1,349 receiving yards, 12 touchdowns. That's the Miles Austin breakout season from 2009 (81/1,320/11), five times in a row. The Tennessee Oilers chose Kevin Dyson three picks before Moss. Dyson's best season saw him catch 54 passes for 825 yards and seven scores. That was a subpar decision, but it was a better selection than Rogers, who was literally incapable of falling to the ground without fracturing his collarbone and ended up as the worst draft pick of the Matt Millen era in Detroit. NFL general managers who are willing to risk a high draft pick on Janoris Jenkins: Sometimes, your weed-friendly draft pick becomes Moss. Other times, your weed-friendly draft pick becomes Rogers.
Tight Ends
Best: Jason Witten (85.2 points above expected return)
Worst: Bennie Joppru (-54.0 points)
Witten is going to end up being some cause célèbre for football statheads when he retires, because he's going to have the résumé of a Hall of Famer and nobody's going to vote for him. Since he entered the league in 2003, only Tony Gonzalez has more receiving yards than Witten, who has averaged 89 catches and 1,014 yards over the past five years. He wasn't quite as good during his initial five-year run, but he was also just the 69th overall pick in that year's draft. The Texans chose Joppru 28 picks before Witten that April, but the Michigan product spent his first three seasons as a pro on injured reserve — yes, you read that right — and failed to catch a single pass at the NFL level before retiring in 2007.
Offensive Linemen
Best: Jahri Evans (86.0 points above expected return)
Worst: Bruce Nelson (-57.9 points)
The Saints took Evans in the fourth round of the 2006 draft as a left tackle out of tiny Bloomsburg University, but quickly moved him to right guard and found a superstar. Evans has started up front during each game of the Sean Payton era, and he's made the All-Pro team in each of the past three seasons. If he's not the best guard in football, he's awful close. You would be forgiven if you didn't remember Nelson, whom the Panthers chose with the 18th pick of the second round in 2003. Nelson started one game as a rookie before tearing the labrum in each of his hips, prompting his retirement after just 14 appearances.
Defensive Linemen
Best: Jared Allen (82.9 points above expected return)
Worst: Jamal Reynolds (-72.2 points)
Allen didn't quite come out of the Culinary Academy, but Idaho State was an obscure enough locale to push him to the 126th pick and the fourth round of the 2004 draft. He had nine sacks in 10 starts as a rookie and averaged 12 sacks per season during his first five seasons, four of which were spent on that rookie contract, which probably paid him just under $2 million. Reynolds was taken by the Packers with the 10th pick of the 2001 draft and never started an NFL game, producing just three sacks in 18 appearances. Reynolds had microfracture surgery on his knee after his rookie season, but because we didn't know what microfracture surgery was back then, it was characterized as having "flakes of cartilage" removed before doctors "poked some holes in his knee to improve circulation." We were so young!
Linebackers
Best: Patrick Willis (93.6 points above expected return)
Worst: Andy Katzenmoyer (-50.2 points)
Is Willis the most productive draft pick ever? He just finished up his fifth season in the league, and during those five years, he's missed a total of four games while making the Pro Bowl and All-Pro teams in each season.2 No other player over this time frame was able to match that accomplishment, and the last player we were able to find who matched that mark was Lawrence Taylor, who made the Pro Bowl team for the first 10 seasons of his career and got onto the All-Pro team for eight of those years. Sure, middle linebacker might not be the most important position on the field, but what more could the Niners ask for from their defensive captain? Katzenmoyer, on the other hand, was a legend at Ohio State, but his NFL career was brief; after suffering a neck injury during his rookie season, Katzenmoyer struggled to stay healthy and left Patriots training camp after breakfast one morning in 2001, never to return. He only played 24 games at the professional level, leaving the game just months before the Patriots won their first Super Bowl.
Defensive Backs
Best: Sam Madison (90 points above expected return)
Worst: Charles Fisher (-59.1 points)
Jimmy Johnson made a lot of terrible draft picks for the Dolphins during his tenure in Miami, but he picked up two pretty impressive players back-to-back in 1997. There, he grabbed Madison in the second round and followed that up with Jason Taylor in the third.3 Taylor probably had the better career, but Madison developed quicker; he was a starter by year two and an All-Pro cornerback by his third season, during which he led the league with seven interceptions. He eventually moved on to New York and picked up a Super Bowl ring with the Giants in 2007. Fisher, taken with the first pick of the second round in 1999, tore up his knee during his first professional game and was never able to make it back onto the field. The Bengals are prone to making some awful selections, but that's terrible luck, not poor planning.
Comment