Thursday, May 03, 2007

Finding hockey's bargains

Toronto-based magazine Canadian Business has gone hockey crazy today, posting a massive database of NHLers ranked by what they consider the best bargains in the league.

It's all very interesting, although I must admit that I'll have to question their methodology given I have to scroll to Page 4 of forwards just to find Sidney Crosby, who somehow finds himself behind world-class pluggers Blair Betts and Arron Asham.

Still, there's a lot of useful data there, and CB even allows you to download their information to a spreadsheet so that some nutty bloggers can play around with the numbers.

Labels: ,


At 5:09 PM, May 03, 2007, Anonymous Karina said...

Wow. I couldn't make it far in that article - valid point on McCabe being a pylon, but low blow on the Leafs goalie situation!

At 5:10 PM, May 03, 2007, Blogger J. Michael Neal said...

The methodology is simple, but pretty screwy. For forwards, it's simply salary divided by number of points. For defensemen, it's an unspecified combination of salary divided by minutes played, and plus/minus. For goalies, it's just salary divided by minutes played. They assume (incorrectly) that number of minutes played is a good proxy for quality, figuring that if the goalie/defenseman wasn't so good, he wouldn't play so many minutes.

I think the flaws in those methods are pretty self-evident. However, there's a more subtle, but at least as bad, flaw. Each point, and each dollar spent aren't equal. Start with the latter. There is a minimum salary that a team can pay a player. That provides a floor that a team must spend. Every dollar below that is a fixed cost, and including it in this sort of calculation is misleading. Rather than looking at the salaries on a gross basis, they should be looked at from a marginal basis. Figure out how much it would cost to put together a team where you're paying everyone the minimum salary, and subtract that from the total payroll. We're only interested in the effect of decisions that the team makes, and they don't make a decision about that first increment of salary.

The second flaw is similar, but more complicated. Much as there is a minimum salary that a team must pay each player, there is a minimum level of talent that the best player at each position that doesn't make a team represents. If, for instance, Sidney Crosby goes down with a season ending injury in November, the Penguins don't lose all of his contributions; they lose the difference between Crosby's production and the production of whoever they can find in the AHL or the waiver wire that takes his roster spot. This makes a big difference. Starting from a basis of zero points overvalues the marginal players on the roster, by implying that if they went down, the team would lose all of their contribution. That's nuts. If your thirteenth forward is lost to injury, the cost to the team is almost nothing, since thirteenth forwards are all over the place looking for NHL jobs. Finding one isn't hard. For this reason, your fourth line players really shouldn't be paid much, if anything, over the minimum salary. Why would you, since you can get almost as much production if you hire someone else at the minimum?

The above lays out the concept of a replacement level player. The goal should be to figure out how many points a team made up exclusively of thirteenth forwards, seventh defensemen and third-string goalies would earn over the course of a season. When your comparing salary to points, only count the number of points earned above what this theoretical replacement, minimum salary, team would earn. Again, we're interested in the effect of decisions that a front office makes in assembling its team, and it doesn't take any effort to produce a replacement team. Counting those points distorts the measure of how successful the GM is.

If you really want to dig deep into this, you also have to understand that some points are more valuable than others. The point that gets you into the eighth spot in your conference is the most valuable point you can earn, so the range of points where that cutoff falls each year is more valuable than points outside that range. Meanwhile, the value of getting 70 points is not much greater than the value of collecting 65 points, so those points have almost no value at all. The same is true, though to a lesser extent, of the points between 105 and 110; all they'll gain you is the difference between a one and a two seed, and often not even that.

Canadian Business approaches this question in a manner so simplistic that it's just about worthless. Now, as far as I know, no one has actually done the work to figure out what replacement level is at each position. We also don't have a good way to measure the overall contribution of a player. So, any such study is going to have some necessary flaws. Even given that, though, CB doesn't know what it's doing.

At 5:13 PM, May 03, 2007, Blogger Michael said...

I do not think people can count Sid in with the best bargains because his contract has been handicapped by the "entry level" status. Had he been able to negotiate more, then maybe the value would be more equal.

Holy cow! Mr. Neal... are you writing a book over there?

At 5:15 PM, May 03, 2007, Blogger James Mirtle said...

Long, well thought-out comments are always appreciated.

Some good analysis, JMN.

At 8:38 PM, May 03, 2007, Blogger Jeremy said...

Can't really comment on the quality of the methodology, but I couldn't help but notice the number of Anaheim skaters near the top of that list. They've got top-notch guys like Penner, Getzlaf, Pahlsson, Perry, Kunitz and Beauchemin all under a million bucks...most of them by a fair margin. I didn't get to see much of them this year until the Vancouver series, but they appear to be excellent players.

By comparison, Vancouver's only got two players in that category (younger, lots of playing time, good stats, and cheap): Pyatt and Bieksa.

At 8:40 PM, May 03, 2007, Blogger The Forechecker said...

"CB even allows you to download their information to a spreadsheet so that some nutty bloggers can play around with the numbers."

Hey, I resemble that remark...

JMN has already eviscerated the methodology, but there's another online tool that folks can look at as well. IBI (Information Builders) built an NHL database as an example of what their WebFocus reporting tool can do. They plugged in 2006-7 regular season data, along with a few extra items like salary and a proprietary rating value.

It's a neat online tool, as you can apply your own filters and sorts, and export the data in a variety of formats.

At 1:04 AM, May 04, 2007, Anonymous David Johnson said...

They put a bit of work into this and it has a certain amount of 'curiosity' factor to it but it is more or less useless because you can't evaluate a player based solely on points or on minutes played. Their procedure of evaluation is far too simplistic. Plus, they use the players actual salary rather than the more important number (for many teams) of salary cap hit. This hurts a guy like McCabe who made $7.15 million this season but only had a salary cap hit of $5.75 million.

In the end the whole process is kind of useless.

At 1:05 AM, May 04, 2007, Blogger J. Michael Neal said...

Holy cow! Mr. Neal... are you writing a book over there?

I not only have a university degree in Statistics, but my father proudly raised me to be both an intellectual snob and an incorrigible pedant. You should see what my long diatribes look like.

That said, I should also admit that none of that analysis is original to me. It's all cribbed pretty directly from a set of baseball researchers who are holders of Statistical degrees, intellectual snobs, or incorrigible pedants.

At 1:19 AM, May 04, 2007, Blogger J. Michael Neal said...

I do not think people can count Sid in with the best bargains because his contract has been handicapped by the "entry level" status. Had he been able to negotiate more, then maybe the value would be more equal.

I'm responding to this in a second post mostly because I was too stupid to do it in one ...

In fact, you have to include Sid in this, for exactly this reason. GMs work within the context of a particular Collective Bargaining Agreement. Building around players who are earning less than market value due to the restraints of that CBA is a valid strategy for running a team. As they say, the goal of running a business is to buy cheap and sell dear; this is an example of the former.

The introduction of a salary cap serves to make this an even more valuable approach. Drafting well is more crucial now than it has been since the introduction of free agency, and probably even before that. A GM should be praised for having three guys like Crosby, Malkin and Staal under contract for less than they are worth.

Of course, in the long run, the organization should be punished for being so bad for so many years that they have the opportunity to pay those three guys less than they're worth. That's why this sort of analysis works even better if you not only do it right, but look at multiple seasons rather than just one. As I said when I pasted my first comment into my own blog, the folks running hockey teams seem even more afraid of anything involving actual math, or new ideas, than baseball teams were twenty years ago. I think that there is a huge competitive advantage to be gained by the first couple of teams who try to really understand numbers, and create more accurate numbers. It'll be interesting to see if someone takes up that challenge.


Post a Comment

<< Home

Links to this post:

Create a Link


Free Page Rank Checker
eXTReMe Tracker