Why it doesn't pay to trade up

[(P - shoulder)]

Thought this was interesting enough to deserve its own thread. It addresses a lot of the biases we tend to bring to the drafting process and tears them down analytically, and also shows one more reason BB is way ahead of the curve. So read this and remember it when the Pats don't make a move for your binky (or move back out of a spot where they could've had him)

http://www.vox.com/2015/4/30/8516007/nfl-draft-economics

A series of papers by economists Cade Massey and Richard Thaler has shown that at any given position, historically, the odds of the top player picked (Watkins) being better than the third player picked (Beckham) is just 55 percent or so.

"It's basically a coin flip," Massey, who serves as a draft consultant with several NFL teams, told me last year, "but teams are paying a great deal for the right to call which side of the coin."

Because teams just aren't that good at evaluating a player's chance of success, Massey and Thaler's analysis says in the current trade market, teams are better off trading down whenever they get an offer — that is, trading one high pick for multiple lower ones, in order to diversify risk. But many teams, like the Bills, become overconfident in their evaluation of one particular player and do the opposite: they package several slightly lower picks for the right to take one player very early.

Lots of teams, like the Bills, will trade up when they identify a player they prefer at a needed position: they need a wide receiver, and a few highly rated ones are available, but they trade up because they're certain Sammy Watkins is the best. But the data says that teams just aren't very good at figuring out when this is true.

On average, the chance that first player will start more games than the second one picked at his position: 52 percent. Compared with the third player, it's still only 55 percent, and compared with the fourth, it's just 56 percent.

Again, the data was unequivocal. On average, trading down and getting two players gave a team five more starts per season and slightly more total Pro Bowls.

You could chalk this up to the simple fact that more players start more games, but it's more than that. Even if you imagined that the team trading down could only keep the better one of the two players it drafted, it'd still get slightly more total starts and the same number of Pro Bowls. The truth is that teams are imperfect talent evaluators, so having two later picks is better than a single early one. It's just risk diversification at work.

The most straightforward piece of proof for all this is the fact that trading down correlates with more wins on the field.

Massey and Thaler came to this conclusion by looking at the number of wins a team had in any given season between 1997 and 2008, and the total value of all picks they'd made in the previous four years (the amount of time, on average, for which a rookie is under contract).

They found that one standard deviation in pick value translated to 1.5 more wins per season on the field. Sure, it's a small sample size, and there's a lot of chance and other factors built into the system, such as a coach's strategy. But trading down correlates with a significant amount of victories, given that there are only 16 games in a season.

...more at the link...

 

[(P - shoulder)]

One thing I think is interesting about this is that it puts the lie to the idea that some teams are significantly better talent evaluators than others (something BB the GM gets criticized for, especially wrt certain positions). I am sure there is some difference to a point, but the study shows it's not much, and it's really those who play the draft better are the winners, rather than those who are (slightly) better talent evaluators.

 

[Hawg73]

I agree with their point.

While I could cite several examples of teams trading up and wishing they hadn't, such as RGIII, it is harder to think of the opposite.

One notable exception is the Pats trading down with SF and they took some guy named Jerry Rice with the pick. We ended up with the forgettable Trevor Matich.

To be fair, Rice played at Mississippi Valley State and ran a 4.6 forty. And the rest is history.

Incidentally, Belichick has made 34 moves on draft day since he took over here. 17 up and 17 down. That surprised me a little. I would've thought he traded down significantly more than up.

 

[(P - shoulder)]

Incidentally, Belichick has made 34 moves on draft day since he took over here. 17 up and 17 down. That surprised me a little. I would've thought he traded down significantly more than up.

Intristing. I would also have put the total at higher than 34 if asked. I wonder what the breakdown is of what rounds those ups and downs come in. One of their points is that the curve is very steep in the first round (as far as perceived value vs. actual value), but it drops way off in the later rounds, where it can actually make sense to move around a little more.

 

[dchester]

One thing I think is interesting about this is that it puts the lie to the idea that some teams are significantly better talent evaluators than others (something BB the GM gets criticized for, especially wrt certain positions). I am sure there is some difference to a point, but the study shows it's not much, and it's really those who play the draft better are the winners, rather than those who are (slightly) better talent evaluators.

I didn't see anything in that article about specific teams, so I don't agree at all with your point. The article was about the league as a whole.

It's always easy to pick apart draft decisions in retrospect. But this mistake was utterly predictable — and it remains a mistake whether Watkins ends up a better player than Beckham or not.

I found this statement (above) in the article to be rather short sighted (at best). If Watkins ends up being clearly the better player, then it might have been worth it in this specific case.

Their point is valid that in general (or playing the odds), you pay too big of a price to move up, but in no way does that prove that it never works out, nor does it address (in any way, shape, or form) that some teams might be better at it than others.

 

[Dwight Schrute]

We moved up for Jones & Hightower.

That didn't seem to hurt.

Though in doing so we bypassed the 43rd ranked DB in the 2nd.

But hey, everything in life has its costs.

 

[HSanders]

It's really bad with QBS. More than a 50% suck/meh rate of QBs who were trade-ups in the first.

 

[tonyto36]

This is all good and true but it misses why teams often mistakenly do this. The to of the draft has the highest upside guys. Julio jones as a prospect was capable of heights that guys like Baldwin could never hope to become. Jj watt was capable of being a historic player which is why he went so high- his incredible raw potential that in that circumstance happened to pan out.

When you don't have a Tom Brady sometimes you have To take risks to get top end talent to keep up.

 

[BostonTim]

I didn't see anything in that article about specific teams, so I don't agree at all with your point. The article was about the league as a whole.
I found this statement (above) in the article to be rather short sighted (at best). If Watkins ends up being clearly the better player, then it might have been worth it in this specific case.

Their point is valid that in general (or playing the odds), you pay too big of a price to move up, but in no way does that prove that it never works out, nor does it address (in any way, shape, or form) that some teams might be better at it than others.

All I think that means is that the odda of doing better by moving up are statistically low, and this is true whether your certain move turns out beautifully. The fact that it might turn out or does turn out is implicit in the whole notion of statistics. I would also like to see a coach by coach comparrison oh their own individual success rate moving up. It might didclose that certain coaches operate above the mean odds while others sre below. In fact that must happen by definition. That might have some validity as a measure of talent evaluation chops.

Cheers, BostonTim

 

[(P - shoulder)]

I didn't see anything in that article about specific teams, so I don't agree at all with your point. The article was about the league as a whole.
I found this statement (above) in the article to be rather short sighted (at best). If Watkins ends up being clearly the better player, then it might have been worth it in this specific case.

Their point is valid that in general (or playing the odds), you pay too big of a price to move up, but in no way does that prove that it never works out, nor does it address (in any way, shape, or form) that some teams might be better at it than others.

Well right, exactly - it works out 54% of the time. But that's the point when you're dealing with statistics - you can always point to an example that defies the statistics, but if you use that same approach over the long run, it will fail more often than it works out. So the smart thing to do is to assign a value to trading up that is in line with the 54% likely success rate, in which case you'll be making an offer that probably won't be accepted by the team that holds the pick you want.

You are correct that they do not address whether some teams are better than others. It is implied that there are no significant differences, but they don't specifically address that, and it would be a very interesting set of data to have in regards to this question. I'd be willing to bet that there is not that much difference between the best drafting teams and the worst, and that the main difference is that teams like Balt play the draft better than teams Buff, rather than being significantly better evaluators of talent.

I guess one thing you could do to improve your odds when trading up is make your qualifications on who's worth it much more stringent. So for example BB hardly ever trades up in the first round or two, but the few exceptions (Jones & Hightower were mentioned above) are hits. That would be another data point that would be interesting to see - if teams that trade up less often have a higher success rate.

So yes, there are a number of conclusions one could draw from this (especially if we saw more data), but I still feel well convinced that playing the odds is the correct way to go, and I'd be surprised if any teams are head and shoulders above the rest in talent evaluations (more than, say, a couple percentage points worth of success rate).


Edit: Or what BT said, more concise, less hot air

 

[AZPats]

BB is a master in trading around. He simply must and has to be superior, because his record proves that, despite drafting at the tail end of most draft selections.

It seems to have a different modus operandi when ever BB trades up. Except to position by a few draft positions, at most. His trades down are bigger distances early, obtaining multiple good picks.

Late rounds, he seeks more picks rather than specific players, IMHO.

 

[dchester]

Late rounds, he seeks more picks rather than specific players, IMHO.

This makes a lot of sense, as not many late round picks will make it. If you are able to get several, the odds are better that one of them will succeed.

 

[O_P_T]

Well right, exactly - it works out 54% of the time. But that's the point when you're dealing with statistics - you can always point to an example that defies the statistics, but if you use that same approach over the long run, it will fail more often than it works out. So the smart thing to do is to assign a value to trading up that is in line with the 54% likely success rate, in which case you'll be making an offer that probably won't be accepted by the team that holds the pick you want.

You are correct that they do not address whether some teams are better than others. It is implied that there are no significant differences, but they don't specifically address that, and it would be a very interesting set of data to have in regards to this question. I'd be willing to bet that there is not that much difference between the best drafting teams and the worst, and that the main difference is that teams like Balt play the draft better than teams Buff, rather than being significantly better evaluators of talent.

I guess one thing you could do to improve your odds when trading up is make your qualifications on who's worth it much more stringent. So for example BB hardly ever trades up in the first round or two, but the few exceptions (Jones & Hightower were mentioned above) are hits. That would be another data point that would be interesting to see - if teams that trade up less often have a higher success rate.

So yes, there are a number of conclusions one could draw from this (especially if we saw more data), but I still feel well convinced that playing the odds is the correct way to go, and I'd be surprised if any teams are head and shoulders above the rest in talent evaluations (more than, say, a couple percentage points worth of success rate).


Edit: Or what BT said, more concise, less hot air

I'd bet that a major factor that differentiates the "good" drafting teams from the "bad" drafting teams has as much, if not more, to do with scheme as the ability to evaluate talent.

Very few players can excel in any and all schemes. Most will be good to very good in a limited number of schemes, but simply don't have the physical or mental skills to be anything but meh in others.

If your team is constantly rotating coaches, and thus potentially changing schemes, then you likely won't have as many guys who can execute the plays the coaches draw up.

The exception being BB who doesn't have a "scheme" but game plans extensively and so tries to get guys who can play more than one scheme.

Also, if you run a scheme that is closer to what was used at various college teams, then it is easier to project how a player will perform in your scheme, than if you run something very different.

 

[O_P_T]

This was an article from Pats Pulpit before the draft that I forgot to post.

The graphic certainly supports the OP.

How Belichick Wins The Draft, Before It Even Begins

By spiketv on Apr 29, 2015, 9:15a


Every night I fall asleep, comforted by my faith that Belichick won't sleep until he finds a way to do every part of his job just a little bit better than the rest of the NFL. And he has, in all facets of the game (except rule-bending). The draft is no exception. So let's take a look at how Belichick wins the draft every year, before it actually happens.

There's a lot of hoopla around the draft these days. I don't know exactly where it came from, but like many of you, I love watching film, analyzing players' stats, regurgitating pundit promises, and making predictions about who's going to draft the next All Pro player. I guess it's not that we stopped dreaming about playing ball, we just want to do it from the box seats now. But through the countless hours we've devoted to these labors, we know that draft day is unpredictable — even for real coaches, GMs and whatever ethereal title we're designating Belichick with these days.

That's why so much of draft day success comes from its preparation. I'm not an insider and therefore don't know how the Pats, or any NFL front office and teams of scouts prepare, or how that ultimately impacts their picks. But the Pats seem to stand out from most teams by accumulating draft picks, and of course by their consistent success in the league each year. But how unique is this approach?

The following is a graph that compares Average Team Performance with Average Draft Capital over the years 2008-2015. Try to find the Patriots dot...

Jump to figure to skip methodology.

Average Team Performance Ranking is each team's "end of season ranking" from the previous season. The ranking is defined by the inverse of the draft order (determined before trades). For example, having just won the Super Bowl, the Patriots draft 32nd, and their 2014 performance ranking is 1. The Seahawks are 2, and so-on until the Bucs at 32. For teams knocked out in the same round of playoffs, the order is determined by season record, with a bunch of other tiebreakers that can be applied. Even if a team didn't have a first round pick, or any pick at all in a draft, their Performance Ranking was still calculated.

Average Draft Capital is the cumulative "Approximate Value" (AV) of picks per draft, for each team. So not just the number of picks they have in that draft, but the sum of their values. Pick value is determined by a formula that reflects a single value for each pick number, independent of players that will actually drafted. Jimmy Johnson famously pioneered this method to evaluate packaged picks in a trade. For example, in this year's draft the Patriots have the 32nd pick, which has a value of 29.6, compared to the draft's first three picks, which have values of 73.2, 64.5, 59.4 respectively. Picks in the 7th round range from 3-5 points in value. Find out more about the formula, which was built by Chase Stuart.

The red line reflects the expected average cumulative draft capital based on end of year ranking. For example, if a team finishes last in the NFL (ranked 32nd) every year, they would be assigned the first pick of every round, and should accumulate the highest draft capital each year. The opposite is true of a team that finishes first (wins the Super Bowl) every year. That's why the line slopes downward.

Teams to the left of the line have less draft capital than expected, based on performance, whereas teams to the right have more. The distances between each team and the red line reflect the impact of management / front office decisions, primarily trades and the accumulation of compensatory picks.

The fact that the Patriots have accumulated draft capital, through their frequent trades and savvy management of free agents, is no surprise. The extent to which they've outperformed their peers, and dramatically increased their expected draft capital, is mind-boggling. Since 2008, Belichick has gone into drafts with the 4th highest draft capital at their disposal among all teams, despite ending the season ranked better than 5th, on average.

It was always safe to say Belichick makes something out of nothing (see Akeem Ayers maneuver), but on average, he's accumulating draft capital that equates to the addition of the 16th pick, of the First Round — in every draft! The next closest team, surprisingly are the Bengals, who have added, on average, 27 value points to their drafts, or the equivalent of the 39th pick (7th pick in the 2nd round). The Rams have done well, primarily from their fleecing of Washington in the RG3 trade in 2012. Read more about how trades have impacted the Pats and other teams here and here.

 

[BostonTim]

Synopsis. Pats are a mile further than anyone else from the red line. Thus they are a mile better than all the rest.

Great graph.

Cheers, BostonTim

 

[BostonTim]

Synopsis. Pats are a mile further than anyone else from the red line. Thus they are a mile better than all the rest.

Great graph.

Cheers, BostonTim