Which roster is stronger?

So, this morning, my girlfriend's father emailed my girlfriend and me a copy of the 2013-'14 playoff roster (and basic stats) side-by-side with the 2011-'12 edition, accompanied with a simple question: "Which roster is stronger?" A normal person would have replied with a sentence or two to start a conversation. Instead, I wrote this.

Oh, wow. Great question. Strap in.

Let's start with an easier one: which team is better? I don't know how much of hockey's "advanced stats" you're familiar with, so here's some context. If you already know what "Corsi For Percentage" is, then please skip this paragraph. For our purposes, let's accept some basic things the "advanced hockey stat nerds" have been more or less statistically proving for a few years:
1) The best way to measure how "good" a team is is to measure its puck possession. Scoring goals (and not allowing goals) is how you win, but goals are sometimes fluky, and "how much you have the puck" is a better long-term indicator of likelihood to win than actual past goal differential. Put another way: scoring goals may or may not mean you're likely to keep scoring goals; having the puck is a pretty good indicator you're going to be scoring goals.
2) Measuring "how much you have the puck" is really hard given the stats the NHL tabulates. No one tracks actual zone time or possession time. So, we have to try to represent possession time with what we do have. And what we do have is shots on goal, shots blocked (meaning shots you take that the other guys block), and shots that miss the net. Adding these up gives us a crude but decent indicator of how much you have the puck: you can't be attempting shots without having the puck. In a just world, we would add these 3 things up and call them "shot attempts," but instead we call them "Corsi," to confuse everyone.
3) So, a really good simple dirty way to tell how good a team is is to compare its Corsi to its opponents' Corsi (in games against them). To make it easy to compare across different games, we represent it as a percentage: (our Corsi) divided by (our Corsi plus our opponents Corsi). We call that "Corsi For Percentage," or "CF%." All it means is: what percentage of the total shot attempts in the games were taken by our team? For example, in one game, I attempt 10 shots and you attempt 30 shots. My CF% for that game would be 25% and yours would be 75%. This is very simple, it works as a good stand-in for possession time, and so it works as a good indicator of how good a team is. As a rule of thumb, anything under 50% means "worse than their opponents," and above 50% means "better than their opponents."

So, the first thing to measure is: which team is better? At 5-on-5, the 2011-'12 Rangers had a CF% of 47.7%. The 2013-'14 Rangers had a 5v5 CF% of 52.4%. That is a very significant improvement (for reference, the range tends to be from mid-to-lower-40s to mid-to-upper-50s for a team for the season). The stat "Fenwick" is like Corsi but it subtracts out the shots that were blocked. The Rangers' Fenwick For Percentage saw a similar increase: 2011-'12 had a 5v5 FF% of 49.4%; 2013-'14 had a 52.6%. Finally, the 2013-'14 Rangers' 5v5 shooting percentage (percent of shots on goal that go in) was an abysmal 6.66% - third-worst in the league. Meanwhile, the 2011-'12 squad shot at 8.30% 5-on-5, for 8th-best in the NHL. It's generally accepted and statistically supported that shooting percentage, except in a few rare cases, will regress to the mean over time, and is largely attributable to luck. So, there's an argument that the 2011-'12 team was luckier than this year's team, in terms of goals actually going in the net, which means their success was a little less based on skill than 2013-'14. All in all, yes, it's clear that the 2013-'14 team is better.

But that's not exactly what you meant, right? The interesting question is "is the new team better because of the new roster?" The alternative, I suppose, is "is it better because of improvements in the players who were on both squads?" Let's start by looking at the roster changes you sent:

2011-'12 F: Callahan, Gaborik, Anisimov, Fedotenko, Prust, Dubinsky, Mitchell, Rupp
2011-'12 D: Del Zotto, Bickel, Eminger

2013-'14 F: St. Louis, Brassard, Zuccarello, Pouliot, Carcillo, Moore, Nash, Dorsett
2013-'14 D: Klein, Moore, Diaz

(I am throwing out Miller and Fast because they played so little time compared to the rest of the list.)

At a glance, the lists look pretty even, with a slight advantage to 2013-'14 squad:
Scoring up front: The only pure scoring talent on the 2011'12 list is Gaborik, while this season boasts both St. Louis and Nash. Advantage: 2013-'14
Middle of the pack: Callahan, Anisimov, Fedotenko, Prust, and Dubinsky stack up pretty evenly with Brassard, Zuccarello, and Pouliot. 2011-'12 has more of those guys, and they hit more, but we know how good the Brass-Zucc-Pou line has looked. Advantage: Even
Guy who's better than everyone thinks he is: Dominic Moore is our new John Mitchell. Advantage: Even
Dumb idiots: 2013-'14 has 2. 2011-'12 has 1. Advantage: Even
Defensive scoring: Michael Del Zotto was sometimes an actual threat from the blue line, when he wasn't a liability. 2013-'14 has no equivalent. Advantage: 2011-'12.
Defensive actual defense: 2011-'12 boasts 3 occasional liabilities. 2013-'14 is 3 completely adequate guys. Advantage: 2013-'14
Net advantage: 2013-'14

But let's see how true that is by looking at the 5-on-5 CF% of each player on these lists. This is just like the team's 5v5 CF% calculation, but it only counts when that player is on the ice. Much like the team calculation, as a dumb rule of thumb, above 50% is good and below 50% is bad.

2011-'12
53.4 Mitchell
50.3 Dubinsky
48.4 Anisimov
47.9 Gaborik
46.8 Callahan
44.7 Prust
44.2 Fedotenko
41.4 Rupp

47.9 Bickel
47.2 Del Zotto
45.1 Eminger

2013-'14
55.1 Pouliot
54.2 Nash
53.8 Zuccarello
53.5 Brassard
51.9 St. Louis
50.5 Dorsett
48.2 Moore
47.5 Carcillo

58.0 Diaz
51.6 Moore
50.5 Klein

Wow! Those numbers are not even close. On the 2011-'12 squad, only Mitchell and Dubinsky had scores above 50; only the 2013-'14 squad, only Dominic Moore and Carcillo were below 50. To explain those numbers further, we can look at a statistic called "Zone Start Percentage." This measures what percentage of your shifts you start in the offensive zone. The lower the number, the more often you're on the ice in the defensive zone (which may, in some cases, explain why your score is lower). If your ZS% is high AND your CF% is low, then you're used in the offensive zone all the time but your opponents still attempt more shots than you whenever you're on the ice. That'd be really bad.

Dominic Moore's ZS% was 25.2% this season, meaning he started 3/4 of his shifts in the defensive zone. In that light, his 48.2% CF% doesn't sound so bad. Similarly, Carcillo's ZS% was 32.6. So, Moore and Carcillo may not have been dragging their team down as much as it seems. (On the flip side, this helps explain Diaz's high score - his ZS% was 75.6%.) Over on the 2011-'12 squad, Dubinsky's ZS% was 41.7%, and Mitchell's was 45.8%, meaning they both started more in the defensive zone than the offensive zone: their higher scores are not explained away by their zone starts; they really were that much better than the rest of their team.

But something smells in this analysis: if the numbers on the changeover players are that much worse, the numbers on the common players are probably worse also. Maybe the changed players aren't entirely to blame. So let's look at the 5-on-5 CF% numbers for the roster players who stuck around, from 2011-'12 to 2013-'14

Boyle 48.1 -> 46.9
Hagelin 52.6 -> 54.5
Richards 48.2 -> 54.1
Stepan 46.5 -> 53.1

Girardi 48.4 -> 49.9
McDonagh 49.0 -> 51.1
Stralman 48.3 -> 56.5
Staal 44.0 -> 54.4

(I'm eliminating Kreider here because I don't know what to do with him. He didn't play nearly enough in 2011-'12 for his numbers to be useful, but it doesn't seem fair to count him as a "roster change" for 2013-'14, since he was on the team in 2011-'12. So, I just left him out entirely.)

Remember, what we're trying to learn is how much each of these players is now bringing the team up, as compared to the roster change. Boyle's CF% went down (requisite with getting harder zone starts), so it wasn't him. Hagelin, Girardi, and McDonagh each improved a little bit, but probably not enough to make a huge difference - and, all three of their changes were aligned with getting significantly more offensive zone time. Hagelin's ZS% went from 51.9 to 62.1, Girardi's from 44.7 to 47.1, and McDonagh's from 43.1 to 48.3. Those are right in line with their CF% increases, so it's probably not them either.

That leaves Richards, Stepan, Stralman, and Staal. All 4 improved significantly, quite possibly pulling the rest of the team up with them. A look at Richards's ZS% shows an increase from 54.1% offensive in 2011-'12 to 66.4% in 2013-'14. That's a healthy boost, and it probably explains some, but not all, of Richards's CF% increase. The other 3, on the other hand, all saw harder zone starts in 2013'14 and nonetheless but up significantly better CF%. So, the team's improvement can certainly at least be attributed to an improvement in Stepan, Staal, and Stralman, and probably also, to a lesser degree, to Richards.

This brings up a question for me. Stralman and Staal are paired together this season, so I'd expect their numbers to be pretty similar. But, because we watch the games, we feel like Stralman is probably dragging Staal down, and Staal bringing Stralman up. This could help explain why they're both on this list. Are we right? To help find out, we can look at each player's 5-on-5 CF% when he's on the ice without the other one, compared to his CF% with. When we do that, we find a surprising result:

2011-'12
Staal without Stralman: 43.9%
Stralman without Staal: 49.0%
Together: 45.0%

2013-'13
Staal without Stralman: 48.6%
Stralman without Staal: 56.1%
Together: 56.2%

I did not see that coming, and I don't immediately know how to explain it. Regardless, the original point stands: improvement has come from within the retained players, in Stepan, Staal, and Stralman, and to a lesser extent Richards.

So, how much of the Rangers' improvement can be attributed to these 4 dudes, and how much to the roster changes? That's a harder question to answer, but we can make some smart guesses by comparing the individual players' CF% to the team's. If the player's individual CF% is higher than the team's, it stands to reason he is bringing the team up; otherwise, he is bringing the team down. So let's go back and look at the roster changes again, with the team average for that season inserted into the list.

2011-'12
53.4 Mitchell
50.3 Dubinsky
48.4 Anisimov
47.9 Gaborik
47.9 Bickel
47.7 Team Average
47.2 Del Zotto
46.8 Callahan
45.1 Eminger
44.7 Prust
44.2 Fedotenko
41.4 Rupp

2013-'14
58.0 Diaz
55.1 Pouliot
54.2 Nash
53.8 Zuccarello
53.5 Brassard
52.4 Team Average
51.6 J. Moore
51.9 St. Louis
50.5 Klein
50.5 Dorsett
48.2 D. Moore
47.5 Carcillo

Counting the players on each side of the line doesn't do us much good, but looking at who is on which side does. In 2011-'12, outside of Mitchell and Dubinsky (who were genuine assets, as we saw), the only 3 players bringing the team up were Anisimov, Gaborik, and Bickel. Guys whom we thought of as producers on that team - Del Zotto, Callahan, Prust, Fedotenko - are all below the line. In 2012-'13, the main names bringing the number down are Dorsett, Dominic Moore, and Carcillo. All 3 of those guys had ZS% below 35%, so we expect lower numbers. (To contrast, Rupp, Fedotenko, and Prust averaged a ZS% of 37.8; Dorsett, Moore, and Carcillo averaged 30.8).

St. Louis's number is weird, because it was generated mostly on a different team. Tampa Bay's 5-on-5 CF% this season was 51.0%, which St. Louis actually brought up. So it's hard to measure him against the Rangers' average meaningfully. That leaves only John Moore and Kevin Klein as the new roster guys bringing us down. Compare that to the list of guys bringing the 2011-'12 number down, and I think it's safe to conclude that 2013-'14's roster is better.

So, I'd conclude that the 2013-'14 team is definitely better than the 2011-'12, and that that's due in part to the improvement of Stepan, Staal, Stralman, and Richards, and in part to the improvement of the roster, which is itself on average better.