The short answer, according to the Hardball Times: Pretty accurately, but not perfectly. I wanted to link to this article by Colin Wyers because I thought it was important. Yet I didn’t see much discussion about it when it came out and I hang out online in some of the some places that statheads do. Here’s the important part:
Some things to take away from the chart:
Power hitters are far more “inconsistent” than other kinds of hitters. Doubles, triples and home runs by far have the highest standard error.
Walks are very consistent, showing very little standard error compared to other events. Outs are also very consistent. So, all else being equal, players with either very high or very low on-base percentages would tend to have less error in their linear weights estimates.
Colin was writing about hitting. That’s the portion of baseball that’s easiest to analyze statistically. If there’s fuzziness there, what about when it comes to analyzing the rest of the game? None of this should suggest that Duane Kuiper was a better hitter than Barry Bonds or anything like that. But I am curious to hear what others like Tom Tango have to say about this article. Maybe I missed the discussion.